NFL Draft Accuracy: Three Questions and Answers

When analyzing pick values in creating my recent NFL draft value chart, I was led to ask some questions related to accuracy in the NFL draft. When most people talk about NFL draft accuracy, they look at how good different experts’ mock drafts are. However, I was curious to study:

1. If NFL teams are better at drafting now than they were in the past
2. If there are certain colleges where players tend to be consistently drafted lower than they should be
3. If any NFL teams are consistently better at drafting than others

We’ll go through each of these questions using draft data back to 1980 and pro-football-reference’s approximate value stat to support our answers to these questions with data. You can use this link to skip ahead to the discussion about over and underrated colleges and this link to skip ahead to the discussion about each team’s NFL draft accuracy.

NFL Draft Accuracy Over Time

Measuring accuracy over time is kind of a difficult thing to do, but there is one simple guiding principle. A perfectly accurate draft is one in which the best player is drafted first, the second best player is drafted second, and so on. A perfectly inaccurate draft, on the other hand, would be one in which the worst player in the draft was taken first, the second worst player was taken second, and so on.

For each draft since 1980, we went through each draft pick and looked at that player’s overall rank in the draft class. Then, a perfectly accurate draft class would look like:

Pick Number	1	2	3	…
Career Rank	1	2	3	…

If we compute the average error between where a player was taken and where they should have been taken, then we get a measure of how accurate an individual draft class was. Looking at this accuracy measure over time from 1980 until 2015, we will be able to see if teams have gotten better at drafting over time as scouting departments have gotten “more advanced”.

This chart shows in a fairly definitive way that teams are getting better at drafting. Using the regression line as a guide, the root mean squared error in the draft order has decreased by about 10% over the last 35 years. If you ever wonder why sports teams spend so much on mathematicians and computer scientists, look no further than this chart.

While there are other ways to measure draft accuracy over time, this way captures fairly well the effect that a higher prevalence of analytics has had on the ability of NFL teams to scout and acquire talent through the draft. Next we’re going to look if team’s consistently miss on evaluating talent from certain colleges or from certain conferences.

NFL Draft Accuracy by College

One of the lingering debates leading up to the NFL draft is whether or not dominant players from small schools stand a chance at succeeding in the NFL. Sometimes, dominant players from small schools transition well to the NFL (like, for example, AJ Bouye) and sometimes they don’t (like, for example, Blake Bortles). And, as evidence by the fact that Bouye and Bortles were on the same college team, you can’t predict which players will succeed just by looking at how good their college team was.

NFL teams do in general evaluate prospects relative to the quality of their competition. What I am curious about, though, is how well the teams do this. Do NFL scouts consistently over or under-correct for players from certain schools or certain conferences?

First, I looked at the set of all players drafted from each of the power 6 conferences. Then, I looked at those players’ draft positions and computed how much value you would expect to get from a player at that draft position. Then, by comparing the actual value the players had over their career to the expected value based on where they were drafted, we can see if the players should have been drafted earlier or later. This lets us see if some schools or conferences have been historically underrated. The results for each of the division one conferences are contained in the chart below.

To me, this result is fascinating. We see that the ACC and Group of 6 have been undervalued while the other 4 P5 conferences have been overvalued.

I find this particularly interesting because the ACC is regularly and rightly viewed as the weakest of the traditional power five conferences. Thus, the fact that players from the ACC and G6 schools are undervalued means that scouts have historically placed too much value on the quality of opposition in evaluating college talent.

Let me say that again in a slightly different way: good players from small schools are likely better than we have historically given them credit for. The effect is small, but clear and consistent.

I also broke down within each P5 conference how over or under-valued each team was. The results of this analysis are in the five plots below. I’m not going to have any interpretation for these plots but they’re certainly fun to look through.

Across all teams that have had at least 20 players drafted, the three teams that were the most consistently underrated were Iowa State, Purdue, and Boston College. On the other hand, the three most consistently overrated teams were Wyoming, BYU, and Oklahoma.

If we restrict ourselves to looking at only Group of 6 teams, the three most underrated schools were Marshall, Cincinnati, and Tulsa. If we look at all schools and not just those who have had at least 20 players drafted, the two teams that have delivered the most average value over expectation were Savannah St. (because of Shannon Sharpe) and Sonoma St. (because of Larry Allen). Both of these schools have generated a hall of famer and not really anyone else.

In summary, it seems as if scouts tend to underrate players who played their college career against lower quality opponents. NFL draft accuracy could be increased by taking this idea into account, but by being careful to not overcorrect too far in the other direction.

NFL Draft Accuracy by Team

Finally, I wanted to look at which NFL teams were the best and worst at drafting over the period from 1980-2015. The methodology in determining which teams had the highest NFL draft accuracy over this period was largely the same as the previous section. We looked at the value that each team accrued relative to the expected value at each pick. Then we tallied things up to find the chart below.

Best 5 Drafters (1980-2015)	Worst 5 Drafters (1980-2015)
PIT (+3.6)	CLE (-3.2)
BAL (+2.8)	STL/LAR (-2.3)
SF (+2.6)	CIN (-2.0)
SEA (+2.0)	OAK/LV (-1.7)
GB (+1.95)	DET (-1.6)

This table isn’t terribly surprising. On the left – in the good drafters section – we see a lot of super bowl winning teams. On the right, we see the teams that have been consistently bad in the period from 1980-2015. This just goes to show you that championship teams really are built in the draft. Just a few good picks or a few bad picks can dictate how your team will look over the next decade.

Use the subscription form below to subscribe when more analysis and articles are posted!