Now that we are past the disappointment of yet another failed season, excitement is starting to build in the lead-up to the draft, where the Redskins hold the #2 pick. I have decided to have a look at how hopes and expectations for the draft square with reality, by examining the basic statistical properties of past draft results.

In the first article, I showed that frequently expressed expectations of landing an elite to generational talent with the second overall pick align with the best outcomes that teams have achieved over the last three decades, rather than the average outcome of #2 draft selections. The misalignment between fans’ expectations and actual outcomes is an example of survival bias, a psychological phenomenon in which we selectively remember winners and overlook losers of a selection process, resulting in false conclusions.

I also showed that, although the average quality of players drops as the draft progresses, draft outcomes at each pick number are highly variable, with very good players lasting on the board well into the later rounds, and quite a few not very good players selected in the top five. This type of outcome reflects the operation of a highly probabilistic process, in which teams are able to influence the outcomes through the scouting process and player evaluations, but a large element of unpredictability remains.

Now that I have introduced the basic statistical realities of the draft, I would like to take a look at some beliefs about the draft that are commonly expressed in discussion threads on Hogs Haven. For those readers who found the first article hard going, this and the following ones will be shorter, and I’ll move on from using esoteric analytic metrics (CarAV) to measures of elite status that most football fans are comfortable with.

I would like to start by taking a further look at the expectation that players drafted number 2 overall should be more or less guaranteed to be truly elite talents in the NFL. I’ll also have a look at how pick #2 compares to later first through third round picks held by possible trade partner Miami, who hold several picks nicely spanning the first three rounds. Elite players are frequently defined as future Hall of Famers, perennial All-Pros and Pro Bowlers. So, let’s take a look at where those players are selected in the draft.

---

All-Pros, Perennial and One-Timers

To set the scene, it is useful to look at where All-Pro players are selected throughout the whole draft. The All-Pro teams are selected annually through voting by large panels of football experts. While many of you may already know this, I was surprised to learn that there are three All-Pro teams, none of which has official status with the NFL. The AP uses a voting panel of fifty NFL journalists to pick a first and second team. The Sporting News All-NFL team is voted on by NFL players and executives; and the PWFA team is selected by its more than 300 member journalists. There is a very high level of overall agreement between the AP1&2, SN and PWFA teams, so to get a slightly different perspective, and add more players to get the numbers up for statistical purposes, I also included the analytics-based Pro Football Focus All-Pro team, which is the most likely of the four to pick players that the other three leave out.

The first figure shows the number of players selected to at least one All-Pro team from 2015 to 2019 (AP1, AP2, SN, PWFA, PFF) by round in which they were originally drafted. As we might expect, the proportion of players who became All-Pros roughly halves from the first to second and second to third rounds, then seems to flatten out in the fourth and fifth before dropping to zero in the seventh. Then there is quite an uptick for undrafted free agents (UD), who are almost as numerous as third rounders.

The high number of UDFAs might seem paradoxical at first, until you consider that the annual UDFA class is more than ten times the size of any draft round. I haven’t kept precise count, but it seems that the Redskins bring in at least 10 UDFAs per year for preseason workouts, multiplied by 32 teams, compared to about 32 players per draft round give or take a few comp picks. The high number of UDFA All-Pros makes sense in this context, because the best strategy to improve your chances of picking a winner in a highly probabilistic system like NFL prospect selection is to take more shots.

Since we’re holding the #2 overall pick, what Redskins fans are most focused on, as reflected in the polling in my first article, are the later first and second-round picks we might pick up in a trade down. With that in mind, I had a closer look at All-Pros selected in just the first round:

Here I’ve broken the first round into quarters. As you’d expect, teams picking in the top quarter of the first round have the best chance of landing a future All-Pro. There is a big drop between the number of All-Pro’s originally selected in the first and second eight picks, with picks 9 to 16 yielding only 0.6 as many All-Pro’s as picks 1 to 8. After that the proportion bottoms out at picks 17 to 24 (0.46) and rebounds back to 0.6 in the final 8.

Some astute readers of my first article predicted in the comments that we might expect to see an uptick in draft outcomes at the end of the first round, where the playoff teams generally pick, due to better coaching, player development and team management. That could be it, or it might just be an artefact due to drawing relatively small samples from a noisy distribution. What is most interesting here is that, after the first eight picks, the chance of landing a future All-Pro seems to be fairly flat throughout the rest of the first round.

So far, this is looking pretty good for the Redskins. The #2 pick is right at the top of the high hit rate range for future All-Pros. If they select a player at #2, instead of trading down, we should expect a multiple-time, maybe perennial All-Pro, right? If that’s what you’re thinking, you’d better sit down. The next figure shows the distribution of 1^st team All-Pro selections by players selected at pick #2 from 1979 to 2019. (I’ve narrowed the All-Pro definition to AP1 here, because I’ve pulled the data from Pro Football Reference and that’s all they list).

Exactly one player selected at number 2 in the last 41 years fits the perennial All-Pro description. That’s Lawrence Taylor (LT, 8x AP1), one of, if not the, greatest defensive players in NFL history. Eric Dickerson, with 5 appearances, might argue his way into the perennial category. Six other #2 picks in 41 years could be called multiple-time All-Pros (Mashall Faulk, Julius Peppers, Von Miller, Ndamukong Suh, Calvin Johnson and Tony Boselli), for a total hit rate of 8/41 = 19.5%. On the other hand, about two thirds of #2 draft picks in this period (27/41 = 66%) have not been selected as AP 1^st team All-Pros once.

The relative scarcity of multiple-time All-Pros at the second overall pick in the draft reinforces the main conclusion of my first article. Despite all the effort the best draft experts in the business put into evaluating prospects, teams struggle to identify elite NFL talents ahead of the draft based on college performance and scoutable attributes.

Getting back to our trade-down scenario, what we really want to know is the probability of landing an elite player at the later first and second round picks a team like Miami is holding. The previous analysis shows that even a single AP first-team All-Pro selection is a reasonable criterion for elite status. If anything, it may be too stringent as we’ll see later when we get to Pro Bowlers.

The next figure shows the numbers of AP1 players selected at pick #1, Redskins pick #2, and Miami’s picks numbers 2, 5, 18, 26, 39, 56 and 70 from 1979 to 2019:

The first thing that stands out is that teams picking at numbers two and five have scored more All-Pros than teams picking #1 overall. What could be going on at #1? A few commenters on my first article suggested that teams reaching for quarterbacks at the top of Round 1 might be skewing draft outcomes in that range. Of the 41 players selected #1 overall in this period, 21 (51%) were quarterbacks, so if all positions were equally likely to make the AP1 team, we would expect 4 of the 8 All-Pros selected #1 overall to be quarterbacks. In fact, only two of the eight All-Pros selected #1 overall were quarterbacks (Peyton Manning and Cam Newton), half the expected rate, but that’s based on a tiny sample. So it might appear that QBs are driving down the numbers at pick #1. But that is probably an anomalous result of AP1 being too strict a criterion for elite status, as we’ll see in the next section.

The orange line in the plot is the Jimmy Johnson Trade Value Curve (TVC), scaled to the maximum player number. In my first article, I showed that the TVC seems to overvalue high first-round picks and undervalue later picks relative to observed player outcomes as measured by average CarAV. But here, aside from that dip at #1 overall, and some jitter, which is normal when working with highly variable data sets, the TVC appears to be doing a fairly good job of tracking the probability of selecting All-Pro players throughout the first three rounds. Before we get too excited about that, some caution is advised, since we are looking at a small number of picks, and there are a few pretty big deviations from a perfect fit. It would be good to get another perspective, such as Pro Bowl selections.

---

Ten-Year Pro Bowlers

I have to admit, I’ve always considered the Pro Bowl to be the Grammy Awards of football. It’s voted on by players, coaches and fans, and as soon as the teams are announced each year a flurry of articles goes up about which more deserving players were snubbed in favor of players from more popular teams and aging superstars. But it is the most commonly cited metric for elite players in discussion threads, and it does provide an independent metric for elite players, since it’s chosen by a different group of voters, so it’s worth a look. The next figure shows the distribution of Pro Bowl appearances by players picked #2 overall over the last 41 years.

More than half of the #2 picks (23/41 = 56%) have been to the Pro Bowl at least once, but some of the one-time Pro Bowlers are hardly elite players, like Mitch Trubisky, or only managed elite level play for a single season, like our own RG3. That’s clearly too lax.

“Ten-year Pro Bowlers” comes up from time to time in Chase Young discussions, but the only one here is Lawrence Taylor, and there are only four others across the eight draft picks we have been looking at in 41 years (Peyton Manning [14x, pick #1], Ray Lewis [13x, pick #26], Junior Seau [12x, pick #5], Bruce Smith [11x, pick #1]). That seems to be approaching a working definition of “generational talent” and is clearly too strict. Therefore, as a reasonable compromise, I set the “elite” threshold at three-time Pro Bowl selection. The distribution of three-time or more Pro Bowlers across our sample of picks is shown in the next figure.

A few things stand out here. First, the quarterback effect at pick #1 is gone. Quarterbacks “rescued” at #1 in this sample, who never made AP1, are: 9x John Elway; 6x Troy Aikman; 4x Eli Manning, Michael Vick, Andrew Luck, Drew Bledsoe; 3x Alex Smith, Carson Palmer. With a few possible exceptions, the teams drafting those QBs got good value from their first overall picks. It now appears that the apparent dip in AP1 selections at #1 shown in the previous section has more to do with AP1 selection being too limiting a criterion than a true reach effect.

The second thing that stands out is that 3x Pro Bowl selection allows in more players than first-team All-Pro selection. The numbers top out at 18/41 (44%) at pick #1 and drop to 14/41 (34%) at the Redskins’ #2 selection. But, aside from the #1 pick, the relative distribution of “elite” players across picks, as measured by Pro Bowl selections, is generally similar to that shown by All-Pro selections.

And third, the TVC continues to do a pretty good job of tracking the numbers of elite players across picks, if you allow for some jitter. The fact that a second, independent measure of elite status, voted on by different selectors and taking in some different players, replicates the same trend adds more confidence that it may be real.

---

Hall of Famers

The ultimate elite distinction for an NFL player is selection to the Hall of Fame. But Hall of Famers are so rare that the data are pretty sparse. Nevertheless, the historical results for drafting Hall of Famers are consistent with the trends we saw using All-Pro or Pro Bowl selections to define elite players:

Only four out of 41 players (9.8%) selected #1 overall have become Hall of Famers, but a few more are likely on the way, such as Peyton Manning. I doubt anyone genuinely expects to land a future Hall of Famer with the Redskins’ first pick in the draft this year, but if they do they are being unrealistically hopeful.

Again, the TVC seems to do a good job of tracking the elite of the elite. But here the relationship hits a dynamic range problem after pick #26 where generally only zero or one Hall of Famers appear at any pick number, and more often zero than one. Many Hall of Famers were taken after 26 (just not at picks 39, 56 or 70) but instead of the numbers taken at individual picks getting smaller, they tend to come one at a time and get further apart at later pick numbers. If we wanted to anlyze trends in this part of the draft, we’d have to use something called a logistic regression and, trust me, we don’t want to do that.

---

Summary and Conclusions

In this article I examined the underlying reality behind commonly expressed beliefs that players, such as Chase Young, who achieve consensus elite prospect status heading into the draft are guaranteed perennial 1^st team All-Pros, Pro Bowlers and possibly even Hall of Famers. I did this by examining the historical results at pick #2 over the last 41 years. The results are consistent with what I wrote in the first article of this series, that such expectations are unrealistic and reflect a common phenomenon known as survival bias in which we tend to remember the winners and overlook the losers. In fact, the majority of players selected at #2 overall, who generally had similar things written about them heading into the draft (exception for quarterbacks), never made the AP 1^st team All-Pro roster, and only 56% were selected to at least one Pro Bowl.

Clearly NFL teams, fans, and media analysts are not as good at identifying elite and generational talents heading into the draft as some of us seem to think.

That’s the simple part of the story. I also set out to examine how the probability of selecting an elite player at #2 differs from that at later picks the Redskins might acquire in a trade down with Miami. But I got a little side-tracked by an unexpected discovery. When I compared the historical All Pro, Pro Bowl and HoF outcomes at those picks to the famous Jimmy Johnson TVC, I found that the TVC values do a pretty good job of tracking the proportions of elite players selected at different draft picks. If true, that makes this a much more complicated story than I originally anticipated.

But these preliminary results, are based on looking at only eight points in a fairly noisy data set, so it’s possible that the apparent fit between the TVC and number of elite players is just a coincidence. In the next article, I will put this finding to a more rigorous test and, if it holds up, I’ll get back to comparing the value of different picks in a trade back scenario and discussing the implications of this new wrinkle to draft trade strategy.