A More American College Football Playoff System
Like many sports fans, I was shocked by the College Football Playoff (CFP) Committee’s decision to leave out Florida State, the undefeated ACC champion. Florida State did what anyone would have expected would be enough to earn their way in, but despite what they accomplished, they are no longer, after the injury of their starting quarterback, one of the four best college football teams according to the committee.
While someone want to argue about what criteria the committee should have used instead to make their decision, I personally dislike the entire idea of having a committee pick the teams who qualify. As I put it on Threads:
(For those on Threads, I’m here by the way.)
The CFP Committee presumably made sense when it was created because it fixed the problems of what came before. The BCS system that preceded it used computer rankings, and those rankings also caused controversy when they disagreed with consensus. However, in that case, the reason for its errors was easy to describe: the computer lacked the “common sense” that only a human can provide. Hence, we get a committee.
Of course, human judgement is no panacea. Humans have biases and make mistakes as well. This week’s events are a demonstration of that.
I bring up this topic not to vent my displeasure but rather to offer suggestions for improvement. My posts here mainly focus on investing, but one frequent element in those posts are ideas on how to use data analysis and machine learning to improve results. And I think there is an easy way to remove the need for the CFP committee with only a tiny amount of data analysis, certainly nothing as sophisticated and black-box as the BCS rankings, let alone modern machine learning.
I’ll describe that idea below. First, however, we need to think about what criteria we should use to judge potential systems for picking playoff teams.
A Matter of Principle
The BCS rankings and the CFP committee each have their own advantages and disadvantages. I think that is to be expected. The search for a ranking system that is perfect in every dimension is probably futile. Instead, choosing between ranking systems comes down to the importance we place on the various dimensions.
Most arguments I have heard focus on two particular areas: does the system usually pick the 4 teams most likely to win (the “best” teams) and does the system usually pick the teams that were the most accomplished during the season (the “most deserving” teams).
Implicit in my comment on Threads is that I put equal if not more weight on a different principle: every team knows what needs to happen for them to win the championship. Both the BCS rankings and CFP committee fail in this area. Both the black-box computer rankings and the committee meeting behind closed doors make decisions that can’t always be understood or predicted. Under neither system could Florida State know, going into its final game, whether a win would get them into the playoff.
Some believe the move from a 4- to a 12-team playoff will solve this problem, but I think it simply replaces one controversy with others. Even with 12 teams included, the top 4 teams gain a special advantage — a bye week — so who is ranked 4th still matters. Furthermore, we now get argue over which team was left out at 13th rather than at 5th.
Below, I want to propose a system that lives up to this third principle. A system with clear rules about who gets in and who is out, where every team know exactly what they need to do to make the playoff.
As I said above, however, no system is perfect. I don’t contend that this system will always put the four teams most likely to win into the top four rankings, but I can live with that. The four best teams will know what they need to do to win, and if they are truly the best, then they can prove it on the field.
A Conference-Focused Ranking System
My system starts with the following idea: to be the best team in America, you first need to be the best team in your own conference.
Each conference has its own rules for picking a winner (and none of them involve committees!). Each conference lays out, at the beginning of the season, what needs to happen for a team to win. We will lean on those existing rules (which most seem to be happy with) as the first part of our ranking system. Unlike the BCS and the CFP, we will not override the conference’s rankings based on the “eye test” or anything else. The best team in the conference may not always win the conference, but they know what they need to do win, and it’s up to them to do it.
We allocate the 12 playoff spots between conferences as follows. The top 4 conferences each get two teams into the playoffs: their winner and the runner up. The next 4 best conferences get the remaining 4 playoff spots for their winning teams.
Of course, this begs the question, how do we decide which are the top four conferences? After the Pac-12 merges into the Mountain West next year, I think there is a clear consensus on which are the top four conferences: they are the SEC, the Big Ten, the Big 12, and the ACC. A consensus is not enough, though, I want to have clear rules for this part as well.
This is where some simple data analysis is useful. Let me describe a simple way to rank the conferences.
Every year, each college team plays 3–4 games against teams in other conferences. Ignoring the individual teams, just focusing on the conferences, we can build a simple logistic regression model to predict the outcomes of these games. When I do that on all inter-conference games from 2018–22 (5 years of data), I get the following model:
The results, I think, are very reasonable. The “power 5” conferences are clearly better than the rest, just as the consensus believes. With 2 Pac-12 teams merging with 11 Mountain West teams, a simple 2–11 combination of the coefficients above gives a coefficient of 0.114 for the new conference, making it clearly the 5th best after the SEC, Big Ten, Big 12, and ACC.
Applying the system described above, with two playoff spots for the top four conferences and one playoff spot for the next four conferences, and then seeding the conference winners by the rankings above, we get the following 12 team playoff bracket:
Again, to my eye, this looks very reasonable. The highest seeding goes to teams that will quite often be the best.
You could reasonably complain that even this logistic regression is a black-box for many people. We could replace it with something like a conference Elo rating to make it more transparent.
In either case, however, it is important to note that these rankings are done before the season starts. Going into each season, the playoff seeding is already set. Each team knows what it needs to do to make the playoffs: finish in the top two of your conference rankings, if your conference is top four, or win the conference, if your conference ranks 5–8. (Those in the Mid-American conference also know where they stand: they need to win some inter-conference games to improve their ranking.)
Another nice thing about this system is that all the games still matter. The regular season conference games matter for getting to the playoff. The inter-conference games matter for the future seeding of your conference.
Again, I don’t claim that this system is perfect in every way, but I do claim that it accomplishes the goal of letting every team start the season knowing what needs to happen for them to win the championship. And while the model used to rank conferences isn’t trivial, it gives the analysts something to write about. Sports junkies can spend the offseason arguing about how the conference ranking system could be changed to help their conference, but when the whistle blows to start the first game, the rules for that season are clear to everyone.
That should matter to every college football fan since this is, after all, American football, not association football (soccer) or rugby football. Replacing the committee with this system would not only make the playoff rankings more predictable, it would make them more American because this is a system based on “laws, not men”.
