The CFP's Rankings Exposed a Fundamental Problem With How We Evaluate College Football, and the Ravens Just Proved the NFL Has the Same Issue

The College Football Playoff selection committee released its initial rankings on Tuesday, and what we witnessed was not a surprise. It was a tell. A massive, unmistakable tell about how both the sport of college football and professional football approach the evaluation of talent and performance in ways that are fundamentally broken.

Let's start with what everyone wants to talk about: the teams that landed higher or lower than expected in those initial rankings. But before we go chasing that narrative, we need to understand what those rankings actually represent. They represent a subjective process managed by a committee of people whose incentives are not always aligned with accuracy or fairness. They represent tradition, name recognition, conference affiliation, and yes, occasionally, actual performance. The surprise is not that the rankings were controversial. The surprise is that we continue to treat them as though they are science.

The biggest revelation from those rankings is not where Ohio State landed or whether a one-loss team from a particular conference deserved to jump ahead of a one-loss team from another conference. The biggest revelation is that the eye test still dominates decision-making in football at the highest levels, and the eye test is a profoundly flawed instrument. It is subject to availability bias. It is vulnerable to recency bias. It rewards style points in ways that have nothing to do with wins and losses. A dominant performance against a bad team is somehow weighted differently than a close performance against a good team, and the weighting system is never made explicit because it cannot be defended if it were.

This matters for professional football in ways that people who cover the NFL often miss. Because while we sit here parsing the methodology of college football playoff selection, the NFL is simultaneously making evaluation errors that cost teams millions of dollars and affect competitive balance in ways that ripple through entire seasons. And the Ravens just provided a perfect case study.

The Ravens are starting an undrafted rookie quarterback on Thursday Night Football. Let that sink in. In 2024, with the salary cap, with all the money teams have to spend, with all the scouting resources available, a professional team is going to place its championship aspirations in the hands of a player who went undrafted. This is not a heartwarming story about opportunity. This is an indictment of how NFL teams evaluate talent at positions where evaluation actually matters.

The conventional narrative will be sympathetic. The undrafted kid got his shot. He impressed in practice. The team believes in him. These are all things that might be true. But they mask a deeper problem: if an NFL team's quarterback situation is so compromised that it has to turn to an undrafted player, that team had evaluation failures long before this decision was made. Either the team's scouting process failed to identify viable talent in the draft. Or the team's decision to let previous quarterbacks walk or trade them away was made without a credible contingency plan. Or the organization's roster construction was so neglectful that there was no investment in the position group. Probably all three.

The parallel to the CFP rankings is direct and unavoidable. Just as college football committees rely on subjective impressions formed by watching games, NFL teams rely heavily on the eye test when evaluating players. And just as the CFP process produces results that seem arbitrary and indefensible under scrutiny, NFL teams make personnel decisions that seem baffling in retrospect because they were made based on incomplete information filtered through human bias.

The Ravens situation is actually more complicated than it appears on the surface. Lamar Jackson has been dealing with injuries. The team needed a contingency plan and apparently lacked one. But here is where the NFL business reality intersects with evaluation failure: teams make these personnel decisions in the full knowledge that they might blow up in their face. They do it anyway because of cap constraints, draft capital limitations, and past mistakes that cannot be undone. The Ravens are not starting an undrafted rookie because they carefully constructed a scenario where that was the optimal solution. They are doing it because they are out of better options, and those options were lost or squandered in previous decision cycles.

This is where the college football comparison breaks down slightly, but only because the incentive structures are different. In college football, a coach can recruit, build, and develop a roster knowing they will have four or five years to see the results. In the NFL, the decision-making window is much shorter. Mistakes compound faster. A bad draft pick in year one affects cap space in year three. A failed free agent signing in year two creates dead money in year four. The evaluation errors made under uncertainty eventually become problems that force bad decisions made under duress.

The initial CFP rankings produced controversy because people have strong opinions about how teams should be ranked. But the deeper issue is that the rankings are a symptom of a system that has no objective measure of quality. In football at every level, from youth leagues to the professional ranks, teams are ranked and evaluated on criteria that are partially subjective and partially objective, with the weighting between the two categories constantly shifting based on narrative convenience.

What we saw with the CFP rankings and what we are seeing with the Ravens are part of the same phenomenon. Both are examples of institutions trying to make high-stakes decisions in environments of genuine uncertainty while operating under constraints that limit the information available and the options accessible. The CFP committee cannot watch every snap of every game for every team. The Ravens cannot rebuild their quarterback position instantaneously. Both are making decisions based on partial information and hoping the risks pan out.

The Ravens decision is more defensible from a practical standpoint because they are not trying to claim that their choice is scientifically validated. They are acknowledging that they have limited options and are taking a calculated risk. The CFP committee, by contrast, is presenting subjective rankings as though they are the output of a rational process that should withstand scrutiny. They are not claiming that the rankings are perfect, but they are claiming that the process is sound. The evidence suggests otherwise.

What emerges from both situations is a discomfort with the reality that football evaluation at high levels is still fundamentally dependent on human judgment, imperfectly informed and influenced by factors beyond pure performance. The CFP wants to pretend it has solved this problem through committee consultation. The Ravens are simply acknowledging that they have not.

The real story here is not that the CFP rankings produced surprises or that the Ravens are rolling with an undrafted quarterback. The real story is that both situations reveal how much of what happens in football remains dependent on the subjective assessments of people making decisions with incomplete information. That is not going to change. But acknowledging it honestly, rather than dressing it up in the language of system and process, would at least have the virtue of clarity.