The previous article in this series had a look at the number of “elite” players taken at a small set of pick numbers spanning the first three rounds of the NFL draft over the last 41 years. My original intentions were to evaluate fans’ expectations regarding the likelihood of drafting an elite player against the historical outcomes, and to get a sense of how the probability of drafting an elite player differs as we get further into day one and day two of the draft. But along the way, I made an interesting discovery.

For purely illustrative purposes, I chose to look at the Redskins’ #2 pick and Miami’s first seven picks which might be involved in a trade and nicely span the first three rounds. An interesting thing happened when I plotted the numbers of “elite” players taken at each of these picks against the Jimmy Johnson Trade Value Chart (TVC) on the same scale. In contrast to what we saw in the first article of the series, comparing the TVC to average player career value, the TVC values seemed to do a very good job of tracking the numbers of elite players taken at the same pick numbers. And the same pattern was repeated using three different measures of elite player status: All-Pros, three-time Pro Bowlers and Hall of Famers.

It has previously been shown by others that the TVC overvalues early first round draft picks relative to measures of average player quality, and I saw the same pattern emerge in my first article. My apparent new finding, if true, adds an interesting twist by suggesting that the TVC does provide a good account of the relative value of early and later draft picks based on the probability of selecting an elite player. While this could have interesting implications for draft trades, the apparent fit was based on only eight sample points, and there was considerable jitter in the data, so it’s possible that it was just a coincidence.

---

Testing the Association Between the TVC and Probability of Selecting Elite Players

To determine if the initial finding is real, I will borrow a bit of methodology from my day job. I have mentioned previously that what initially caught my interest in the NFL draft is that it has a lot in common with problems I have experience with through my work in medical research. One of the common problems I help the doctors and scientists I work with is development of new diagnostic tests for cancer and other diseases.

Development of an improved diagnostic test typically starts with discovery of a novel biomarker. A biomarker is any type of “signal”, like expression of a particular protein or gene sequence, that correlates with a disease state or responsiveness to a drug. A medical scientist who has an idea for a new biomarker will usually perform an initial test in a relatively small “discovery set” of patient samples. If he is working on a new biomarker for early detection of ovarian cancer, for example, he might test the expression level of a particular protein in blood samples from patients with diagnosed ovarian cancer relative to a matched set of samples from healthy controls. If the protein is expressed at higher levels in the cancer patients than controls, it might be a useful biomarker for early detection of cancer.

Unfortunately though, most initial biomarker discoveries fail because they turn out to be specific to the particular discovery set that was used, and don’t generalize to patients at different hospitals around the world. That is pretty much where we are with my initial discovery about the apparent correlation between the TVC and probability of selecting elite players. It looks like it might be interesting, based on results from a small sample of picks. But that might just be an erroneous result of selecting the wrong draft picks to look at, since I’ve shown previously that there is a huge amount of variability in historical outcomes at different pick numbers in the draft.

The standard approach that medical scientists use to determine whether their initial results from the discovery set are real or not is something called an independent cohort validation study. That might sound complicated, but all it really means is testing the finding in samples from a new patient cohort, ideally from different hospitals in different parts of the world.

That’s all well and good for clinical research since there are hospitals everywhere (actually, it’s incredibly difficult, time consuming and expensive), but there is only one NFL, and I’ve used up 41 years of draft data to get this far. The closest approximation to a true independent cohort validation study I can achieve using available data is to test the finding in the other 88 draft picks in the first three rounds over a comparable period. To make the validation set as independent from the discovery set as possible, I’ll also use a different measure of elite status, career average value (CarAV), which I used in the first article.

The first figure shows the numbers of “elite” players selected at each pick from #1 to #96, from 1986 to 2016, except that I’ve left out the pick numbers in the discovery set (2, 5, 18, 26, 39, 56, 70) to make it as independent a cohort as possible. If you missed my first article, CarAV is an advanced statistic developed by Pro Football Reference, which quantifies the career value of NFL players, regardless of position, based on a combination of measure including numbers of starts, All-Pro and Pro Bowl selections, and other awards. It is not entirely independent of All-Pro and Pro Bowl status, since those go into it, but it is different enough for this purpose.

In this graph, “elite” players are defined as players with CarAV > 65. I initially settled on a ballpark range of thresholds from 55 to 70 by eyeballing the CarAV data to get a sense of where it transitions from very good players to household names. Then I plotted the results and picked the threshold that gave the best fit to the TVC, which settled on a value of 65. To give you an idea what a CarAV value of 65 means, some players on either side of the elite threshold are:

68: Khalil Mack, NaVorro Bowman, Robert Mathis, Max Unger, Gerald McCoy, Charles Tillman, Jason Kelce, A.J. Green

67: Laveranues Coles, Terry Glenn

66: Trent Williams, Nick Mangold, Elvis Dumervil, Amani Toomer, Aquib Talib, Mike Vrabel, Tony Boselli, TY Hilton, Chandler Jones, Zach Martin

65: Samari Rolle, Cameron Heyward, Nate Solder

64: Antonio Freeman, Jamaal Charles, Mike Iupati, Ray Rice, Muhammad Wilkerson

63: Justin Houston, Osi Umenyiora, Ryan Tannehill, Chris Samuels, Kirk Cousins, Ryan Kerrigan, Demarco Murray

The red line is the Jimmy Johnson TVC, scaled to the maximum number of elite players (14 at pick #1). It provides a very good fit to the central trend of the elite players scatter. To illustrate just how good a fit it is, I also plotted the best-fit trend line of the elite player scatter (dotted blue line). It is a logarithmic curve, and accounts for 75% of the variance of elite player numbers across draft picks. It’s not an exact fit to the TVC, but it’s so close it’s hard to split them.

Visually, this result shows that there is a strong association between the TVC value and the number of elite players historically selected at each pick number in the draft. To quantify the strength of that association, in the next figure I plotted TVC values against elite player numbers at every pick from #1 to #96 from 1986 to 2016 and calculated the correlation between the two.

As expected, numbers of elite players are strongly correlated with TVC value across all draft pick numbers in this range. The strength of the correlation is quantified by the correlation coefficient, which ranges from 0, indicating no correlation, to 1, indicating exact correlation. In this case, the coefficient (R) is 0.87, indicating a very strong correlation. The square of the correlation coefficient (R²) is the proportion of the variance in elite player numbers across draft picks that is explained by the TVC values. It is a healthy 0.75, meaning that TVC values explain 75% of the variance in elite player numbers across draft picks. Assuming that future drafts will be similar to recent past ones, this means that the TVC is highly predictive of the probability of selecting an elite player at each pick number in the draft.

Getting back to the biomarker discovery example, the usual result of independent cohort validation studies is that the association between the biomarker and the disease state disappears when it’s tested in a large, independent sample. When the effect instead becomes clearer when it’s tested in a larger sample, as we have seen here, it means you are on to something.

The TVC was developed by Jimmy Johnson’s colleague Mike McCoy (not the same Mike McCoy who was an OC for the Chargers, Broncos and Cardinals) in the early 1990’s as a tool to determine fair value in draft pick trades. The only public statements I have been able to find about its origin indicate that it was developed by looking at past draft trades. But it does not appear that anyone involved has ever explained how the outcomes of those trades were evaluated. The very strong correlation between the TVC values and past probability of selecting elite players, measured in a variety of ways, suggests that McCoy might have based it on some measure of frequency of drafting elite players. If not, it’s an amazing coincidence.

As I mentioned above, to achieve the near exact fit that I showed in this analysis, I had to tweak the threshold used to define elite players. At CarAV = 55, the TVC sits slightly below the central trend of the elite player scatter and at CarAV = 70 it sits slightly above. But CarAV did not exist when Johnson and McCoy were working, and the association I’ve shown seems to be fairly robust across different metrics, so I suspect they would have found it if they had been looking at something like multiple Pro Bowlers as I did in the last article.

---

“Sure Thing” Draft Picks

The strong correlation which I demonstrated between TVC values and the probability of selecting elite players in the draft means that we can now get back to putting some meaningful numbers around trade-down scenarios. But before we do that, it is worthwhile reflecting on what teams aim to achieve in the draft.

It might appear obvious that teams would want to maximize their chances of selecting elite players with the limited number of picks available to them. But elite players are elite for a reason. They are rare, which means the chance of hitting on one at any position in the draft is low. In the previous article, for example, I showed that the probability of selecting a three-time or greater Pro Bowler maxes out at a modest 0.44 (18/41 players) at the first overall pick and declines from there as the draft progresses. That is hardly the “sure thing” proposition that fans often talk about as the reason that it would be crazy to trade down for more picks later in the draft.

Even the most hardcore stick at #2 advocates would have to admit that no college prospect is 100% guaranteed to be successful at the NFL level. But what would it take to achieve a 90% chance of selecting an elite player? We can use my discovery about the TVC to get an idea.

Since the shape of the TVC closely approximates the distribution of elite talents in the draft, we can convert its pick values into probabilities of selecting elite players through a simple scale conversion. If we divide all the values in the TVC by 3000 (its arbitrary maximum value), we get a curve with exactly the same shape but with values ranging from 1 at pick #1 to a number close to zero at the last pick. Now, if we multiply each value by the probability of selecting a three-time Pro Bowler at #1 (0.44), voila, we have a chart showing the relative probability of selecting an elite player at each position in the draft.

This calculation gives the following probabilities at the draft picks we examined in the previous article (#1, Redskins #2 and Miami’s first 6 picks):

A team holding the first overall pick enters the draft with a 0.44 chance of selecting an elite player. It can increase its chance of success by adding more picks. For the purpose of this exercise, imagine it added the #2 pick. Now the chance of picking at least one elite player is equal to one minus the probability of missing out on both picks. The probability of missing out at #1 is 1-0.44 = 0.56; and the probability of missing out at #2 is 1 - 0.38 = 0.62. The probability of picking an elite player is then 1-(0.56*0.62) = 0.65. That’s better, but still well short of 0.9. Adding pick #3 increases the probability of a hit to 0.765. To get the probability of a hit up to 0.9, the team would have to hold picks one through six, which of course is never going to happen.

The Bengals hold the #1 pick this year, as well as #s 33, 65, 107, 147, 180 and 215. If they don’t trade or add picks, their chance of selecting an elite player in the first four rounds is 0.51, and their 5^th through 7^th round picks don’t add much to that. What this demonstrates is that, based on historical success rates, not only is there no such thing as a sure thing in the draft, in realistic draft scenarios, it’s pretty hard to get the probability of selecting an elite talent much higher than 0.5.

---

Trade Down Scenarios Targeting Elite Players

With the team holding the #2 pick this year, the hot topic of discussion is whether the team should stay put and pick the best overall prospect in the draft, since the Bengals are expected to pick the best QB at #1, or trade down for more picks. Let’s have a look at how trading down changes the team’s chances of picking a game-changing talent.

Miami and Los Angeles, holding picks 5 and 6, respectively, would make the best trade partners, because the Redskins could trade down for additional picks and still have a shot at one of the top prospects in the draft. A swap of the Redskins’#2 pick for Miami’s #5 and #18 is dead even value according to the TVC (2600 = 1700 + 900). The probability of selecting one elite player with the #5 and #18 picks is 1 – (1- 0.25) * (1 - 0.13) = 0.35. That is just slightly lower than the probability if they stuck at #2 (0.38).

Any real math or stat heads reading this might wonder at this point why the probability isn’t exactly the same at #2 and #s 5 + 18, since I’ve shown that the TVC which we’re using to value trades closely tracks the probability of elite players. The answer is that there’s a bit of slop in it. Jimmy and Mike’s TVC isn’t exactly a smooth curve, it just closely approximates one. And it’s close enough that the difference is essentially negligible. If you are worried about the 0.03 difference between 0.38 and 0.35 (0.03/0.38 = 8%) you are reading too much into this. The point is that trading down to add a pick did not change the probability of picking an elite player by very much.

While the chance of selecting an elite player is just slightly lower in this scenario, trading down does create additional benefits compared to sticking at #2. First, it creates a new possibility of picking two elite players. The probability of that happening is the product of the hit probabilities at picks 5 and 18, or 0.25 x 0.13 = 0.03. That’s about the same as the probability of selecting an elite player at pick #76, small but real. There are also a few other new possibilities that we won’t dwell on now, because we’re focused on finding elite players. But it’s worth mentioning in passing that this trade down also creates new potential outcomes including picking one elite player and one quality starter, or two quality starters. I’ll come back to that in the next article.

In addition to making it possible to address more than one roster spots, trading down also helps to manage injury risk by spreading it across more than one player. To illustrate that effect, I’ll use NFL injury statistics that were published in a Football Outsiders article by Zach Binney that Hogs Haven contributor Eboracum wrote about last year. According to the figures in that article, the probability that an NFL player will miss half a season or more to injury each year is about 0.09. If we trade down and pick two players, then the probability that both the players will lose half a season or more becomes 0.09 x 0.09 = 0.0081. In other words, by trading down and selecting two players, the risk that all the players selected using our piece of draft capital is reduced to less than a tenth of what it was if we had stuck with our original pick.

What I’ve shown thus far is that trading back using the TVC produces roughly the same probability of selecting an elite player, spread across more than one draft pick. Trading back also provides additional benefits such as reducing the risk of completely wasting the pick through loss to injury. The next table shows how that works across several possible trade down scenarios, two involving Miami’s first and later round picks. The TVC differential is the difference in TVC points between the Redskins’ and the trade partner’s picks, with positive values favoring Washington. And injury risk is the probability that all players selected will miss 8 or more games due to injury.

The first two trades with Miami have the same probability of yielding an elite player, while the third has a somewhat lower chance, despite being more favorable to Washington in TVC value. This little dip is the result of what I mentioned above, the TVC having some slight deviations from a smooth curve. But by far the biggest effect here is the dramatic reduction in risk of total loss to injury as more players are added. While fans of the Washington Redskins over the last few seasons might not appreciate this, it’s just really unlikely that any two or more players will all miss significant time to injury in the same season.

Los Angeles doesn’t have nearly as much draft capital as Miami to work with, so in order to make a trade up offer that’s close to parity on the TVC, they had to offer their first four picks in 2020 and throw in a 2021 2^nd rounder, which is valued for trade purposes as a third round pick. We don’t know where the Chargers will pick in the second round next year, so I valued it as the middle pick in the third (81) for purpose of the trade value calculation, and the middle pick in the second (49) for purposes of the hit probability calculation. If we are confident that Los Angeles will be terrible next season, this could be a better deal than I’ve shown.

So, by trading down using the TVC to determine fair trade value, the Redskins are able to maintain approximately the same chance of selecting an elite player, while significantly reducing the risk of wasting their pick due to injury.

---

Summary and Conclusions

In the previous article, I observed that the TVC seemed to correlate very well with the chance of selecting elite players across a small set of draft picks spanning the first three rounds. In this article, I put that relationship to the test, by comparing the TVC to the probability of selecting elite players across all the other pick numbers in the first three rounds. The results confirmed the preliminary finding, and demonstrated that the TVC values do appear to reflect the relative chance of selecting an elite player at different pick numbers.

At this point, I have to confess that this finding was a complete surprise to me. When I started writing this series, I was convinced I would end up debunking the TVC because of its poor correlation with average player value. But the finding that the TVC correlates strongly with the chance of selecting elite players makes the story a lot more interesting than a simple hatchet job.

Because of the strong, but not exact, correlation, the TVC provides a somewhat coarse tool to compare the probability of selecting elite players with different combinations of draft picks. Using this approach, I showed that it would require more picks in the first round than any team could plausibly acquire to get close to a “sure thing” (90% chance) of drafting an elite player. The closest a team holding the first overall pick might come to picking an elite player in the whole draft in realistic scenarios is a little over 50%.

I also had a first look at trade-down scenarios for the Redskins. As expected, trade -downs which achieve fair value according to the TVC yield comparable chances of selecting elite players, while reducing the risk of loss to injury. There are also additional benefits to trading down, which will become apparent in the next article where I will take a look at the probability of selecting quality starters instead of elite players.

---