The Verdict on 2017 Central Florida’s Mythical National Championship

Before 2014, there was really no way to crown a definitive ‘national champion’ of college football. Because of the lack of a playoff, oftentimes many teams claimed to be on top creating a long history of the mythical national championship in college football. Many unbeaten teams who weren’t given the chance to prove themselves often claim a mythical national championship for their school. The most recent example – and as we’ll soon find out together one of the most compelling examples – is the University of Central Florida in 2017.

I am going to present my take on resolving the ‘strength of schedule’ problem in order to study the quality of teams in individual seasons. By doing this, I can truly answer how good were Blake Bortles and the UCF Knights in 2017? Would they have beat the best teams in the country? Should they have been given a shot? Is the UCF mythical national championship a warranted and righteous claim?

Let’s dig in.

The Root Cause of the Mythical National Champion Fiasco

College football, much like college basketball, is plagued by the term strength of schedule. The league consists of 100s of teams, so there is a very wide range of abilities and team qualities. Even worse, because college football teams play a large majority of their games within their own conference, it can be quite difficult to compare teams from different conferences based on records alone.

This problem is even more obvious when there is actually zero overlap between leagues. For instance: NBA teams often have difficulty gauging the quality of players coming from Europe. The European teams play only against themselves so there is almost no way to know if a dominant player from Eastern Europe will turn into a superstar or will flounder around the league without ever being that good.

College basketball is in better shape than college football because teams play somewhere from 10-15 out of conference games against the rest of the league. This is a pretty large sample size. This allows us to gauge the relative quality of NCAAM teams by looking how they did against common opponents.

However, in college football (especially in 2020) the problem is pretty bad. Because of how short the season is, each football team really only has the time to play against a few out of conference teams. Even worse, when Alabama spends their time beating Mercer 56-0 instead of playing meaningful regular season game against Ohio State or Wisconsin or Oregon or Notre Dame, we have even fewer comparisons to make.

Back to our specific problem. Central Florida went undefeated in 2017. Before the Peach Bowl, UCF’s only quality wins were against the other top teams from the American: Cincinnati, Memphis, etc. However, because these teams play equally weak schedules, it can be difficult to determine whether Central Florida benefited from a weak schedule or if they really deserve to claim their mythical national championship.

Why is Strength of Schedule Hard to Analyze?

Most simply put, strength of schedule is hard to analyze because it is self-referential.

If we want to determine how good a team is, we look at the quality of their opponents. However, determining the quality of their opponents requires knowing both their opponents’ records and their opponents’ strengths of schedules.

To put that another way, we need to know everyone’s strength of schedule in order to be able to compute strength of schedule in the first place. This is like the chicken and the egg problem cast into the sports world.

Some rating systems like RPI try to address this by including opponents’ records and opponents’ opponents’ records in the formula to act as a first approximation of opponents’ strength of schedule. However, as discussed in our article introducing ensemble ratings, this has some pretty serious flaws.

We need some way to solve for everyone’s strength of schedule simultaneously. Luckily some mathematical tools from optimization theory and statistics tell us how to set up this problem while the language of linear algebra lets us solve systems of ‘interlinked’ equations simultaneously. In the next section we’ll discuss how to use these ideas for college football to discuss the mythical national championship.

An Unbiased, Schedule Independent Rating System

The idea in this section is something we have used before – for NBA ratings and for fantasy football defense rankings – and has proven to be quite successful and powerful. We use the ordinary least squares technique to assign each college football team a rating.

The ratings are designed so that the difference between two teams’ assigned values predicts the margin of victory. If Ohio State is a 20, Wisconsin is a 12, and home field advantage is worth 3 points, we would predict that Ohio State playing at Wisconsin would win by 5.

We determine each team’s assigned value by looking at the games that have already been played and giving out the ratings that explain the previous scores with minimal error. That is, the ratings are chosen optimally so that the predicted margins of victory and the actual margins of victory are as close as possible over the entire set of games this season. Setting this up leaves us with a least squares problem.

Why does this answer the strength of schedule problem?

Because each team only gets credit based on who they beat and how much they beat them by. If you played 12 games against a team with a 20 rating and lost by 5 each time, you would have the same assigned rating as if you played against a team with a 10 rating and won by 5 each time. In this sense, the model doesn’t care about records. It only cares about quality of opponents and quality of win (margin of victory).

So, using this method we should actually get an apples-to-apples comparison between teams from different conferences. Using this method, we are no longer comparing records while having to take into account strength of schedule. The number, a team’s rating, we compute already takes that into account. If team A has a better rating than team B, then team A should be better than team B. There is no ambiguity.

The 2017 College Football Season

The first thing we need to do in order to determine if UCF actually deserved their mythical national championship is to compute the CFB ratings that year. So, we took into account every game from that season and computed our end-of-year CFB rankings for 2017. Before presenting the rankings (because they might be surprising), we will validate that our model works very, very well.

Quick Model Validation

Using our model, the team with the higher end of season rating won over 81% of the games they played. Be aware, this is not the same as saying my model can correctly predict the winners 81% of the time, but simply that the ratings correctly explain 81% of what happened that season. These distinctions are subtle but important. Using this source, the best model accuracy for that CFB season was around 74-75% of games correctly predicted. I am not saying my model is 6% better than the best model out there, I am simply trying to point out that this 81% is nothing to sneeze at.

What about using our model for predictions? For example, if we want to use our technique to predict winners/losers in week 6 of college football, we should only be allowed to use data up to and including week 5. So, we did just that. We predicted the wines and losers of every game. Our season-long accuracy is about 72%, within 1 or 2 points of the best method here.

However, our model takes a few weeks to ‘learn’ enough about the teams it is ranking. This is because we don’t impart any bias, any outside rankings, or any other information. So, at the beginning of the season all the teams are ranked exactly the same. It is only after week 5 or week 6 that we have enough games to actually tell how good everyone is.

The plot below shows how accurate our model is in another way. The plot below has dots relating day number and accuracy. If there is a dot on say, day number 20, with accuracy around 77% that should be interpreted as ‘If we only count predictions made on or after the 20^th day of the CFB season, our prediction accuracy is 77%’. Basically, if we ignore the first half of the season while our model is still learning, our method becomes extremely accurate. One could argue it is even more accurate than those presented before.

2017 College Football Final Rankings

These rankings are an important step in helping us determine how valid Central Florida’s mythical national championship is. Before presenting them, I wanted to remind everyone of their accuracy per the previous section and that they are entirely unbiased, data-driven rankings. The CFB polls suffer greatly from bias. One of the biggest sources of bias is a recency bias.

An 11-1 team that lost week 1 is almost certain to be ranked higher in the various polls than an 11-1 team that lost just last week. Is this fair? Probably not.

Moreover, my system doesn’t care about conferences, it doesn’t care about school history. It doesn’t take into account narratives, Heisman races, or any other external noise. It is just data. That is why it looks significantly different from the final coaches/AP polls in 2017.

Here is what we found:

Rank	Team	Rating
1	Alabama	47.4
2	Penn State	44.9
3	Ohio State	44.6
4	Georgia	44.1
5	Clemson	42.3
6	Wisconsin	39.8
7	Oklahoma	39.3
8	Auburn	38.8
9	Notre Dame	36.9
10	Washington	34.3
11	Central Florida	33.9
12	Oklahoma State	33.9
13	Iowa	32.6
14	Texas Christian	32.1
15	Virginia Tech	30.9
16	Miami (FL)	30
17	Mississippi State	29.8
18	Northwestern	28.6
19	Iowa State	28.1
20	Stanford	28
21	North Carolina State	28
22	Louisville	27.9
23	Wake Forest	27.8
24	Michigan	27.7
25	Michigan State	27.4
26	Southern California	26.8
27	Louisiana State	26.6
28	Texas	26.5
29	Memphis	26.3
30	Boston College	25.4
31	Florida State	25.4
32	Purdue	25.3
33	Georgia Tech	23.8
34	South Carolina	23.3
35	Duke	23
36	Kansas State	22.4
37	Utah	21.2
38	Texas A&M	20.9
39	South Florida	20.7
40	Missouri	20.6
41	Florida Atlantic	20.4
42	Washington State	20.1
43	Oregon	19.8
44	Texas Tech	19.8
45	Indiana	19.6
46	Boise State	19.3
47	West Virginia	19.3
48	Navy	18.5
49	Pittsburgh	18.2
50	Minnesota	17
51	Arizona	17
52	Arizona State	16.7
53	San Diego State	15.2
54	California	15.1
55	Houston	14.8
56	UCLA	14.7
57	Syracuse	14.5
58	Florida	14.3
59	Fresno State	14
60	Kentucky	13.8
61	Mississippi	13.6
62	Samford	13.1
63	Army	13
64	Nebraska	12.9
65	Virginia	12.7
66	Ohio	12.5
67	Appalachian State	12.2
68	Toledo	11.8
69	Troy	11.7
70	North Carolina	11.4
71	Arkansas	10.5
72	Southern Illinois	10.3
73	Colorado	10.2
74	Maryland	10.1
75	Temple	9.9
76	Northern Illinois	9.9
77	Southern Methodist	9.7
78	Wyoming	9.1
79	Baylor	8.9
80	Youngstown State	8.2
81	Colorado State	7.9
82	Nicholls State	7.9
83	Tulane	7.7
84	Tennessee	7.7
85	Marshall	7.6
86	Vanderbilt	7
87	Utah State	6.9
88	Rutgers	6.2
89	Louisiana Tech	6
90	Western Michigan	5.7
91	Arkansas State	5.7
92	Mercer	5.1
93	Eastern Michigan	4.8
94	Buffalo	4.6
95	Central Michigan	3.9
96	Villanova	3.9
97	Stony Brook	3.7
98	Air Force	3.1
99	Tulsa	2.9
100	Illinois	2.6
101	Middle Tennessee State	1.5
102	North Texas	1
103	Delaware	0.9
104	Southern Mississippi	0.9
105	Miami	0.8
106	Massachusetts	0.3
107	Brigham Young	0.3
108	Nevada	0.1
109	Nevada-Las Vegas	-0.6
110	Weber State	-0.9
111	Texas-San Antonio	-1
112	Florida International	-1.1
113	Cincinnati	-1.5
114	New Mexico State	-1.8
115	Akron	-2.1
116	Connecticut	-2.4
117	Louisiana-Monroe	-3.3
118	Georgia State	-4.8
119	Western Kentucky	-5.1
120	East Carolina	-5.3
121	New Mexico	-5.4
122	Alabama-Birmingham	-5.5
123	Oregon State	-5.8
124	Jacksonville State	-6.2
125	Bowling Green State	-6.2
126	Kansas	-6.3
127	Idaho	-6.5
128	Georgia Southern	-7.5
129	South Alabama	-7.5
130	Furman	-8
131	Coastal Carolina	-8.1
132	William & Mary	-8.3
133	California-Davis	-8.8
134	Old Dominion	-10.5
135	Hawaii	-10.7
136	Alcorn State	-11.1
137	Eastern Kentucky	-11.1
138	Chattanooga	-11.4
139	Tennessee-Martin	-11.4
140	Northern Colorado	-12.8
141	Louisiana	-13
142	Montana State	-13.9
143	Kent State	-15
144	Maine	-15.7
145	Rice	-16.5
146	Texas State	-17.4
147	Charlotte	-17.4
148	Alabama State	-18.3
149	Citadel	-18.7
150	Murray State	-20.1
151	Gardner-Webb	-20.9
152	San Jose State	-21.1
153	Eastern Illinois	-21.1
154	Colgate	-21.4
155	Texas-El Paso	-21.4
156	Ball State	-23.6
157	Grambling State	-24.3
158	Montana	-24.7
159	Albany	-27.5
160	Eastern Washington	-29.2
161	Houston Baptist	-29.4
162	Indiana State	-30.3
163	Jackson State	-33.9
164	Stephen F. Austin	-37.3
165	Delaware State	-37.7
166	Towson	-38.9
167	Southern Utah	-39.2
168	Tennessee Tech	-41.6
169	Southern	-42.5
170	Cal Poly	-45.1
171	Arkansas-Pine Bluff	-48.2
172	Morgan State	-61.8

That is right, Notre Dame tops the chart. Perhaps that is a bit surprising. However, returning to the subject of our article, Central Florida is all the way up at number 6.

To complement these ratings, we computed alternate ratings in an ever-so-slightly different manner. Here we give teams the same amount of credit no matter how much they win by (after taking into account a 3 point home field advantage). In this world, the Alabamas don’t get a huge boosts from beating the Vanderbilts of the world by 59 points.

In this setting, Central Florida comes in as the fifth best team in the country.

My Verdict on UCF’s Mythical National Championship

Central Florida had a stellar year. They went undefeated but were snubbed from the championship consideration by the selection committee even after the playoffs had expanded to a 4 team bracket. However, Central Florida was simply too far behind everyone else to warrant careful consideration for a national championship.

The Knights would have been implied 13.5 point underdogs against Alabama (you can see an example of just how unlikely this is here). Underdogs this big win extremely rarely. But, even worse, they would have had to win twice because of the playoff format.

Central Florida did themselves a huge favor by beating Auburn in their bowl game, but I don’t think it was enough. So, for me, no the Central Florida Knights do not get to claim a mythical national championship in 2017.