This article takes its inspiration from a recent discussion in the comments section of Hogs Haven. We were discussing the merits of selecting a QB in the first round versus later in the draft, when one frequent commenter wrote:

“By and large, mid-round QB picks are a terrible investment, especially when viewed against the steep opportunity cost of talented starters on cheap contracts at other positions of need. The hit rate on non-first-round QBs is truly abysmal…”

Whether or not he is correct, the poster’s comment very concisely gets to the heart of the problem facing Ron Rivera and Martin Mayhew in their quest to find a franchise quarterback, while rebuilding the roster that Bruce Allen left in tatters. What is the best strategy to spend limited draft capital resources on finding the QB of the future without starving the team of talent at other crucial roster positions?

As the commenter points out, the hit rate for QBs drafted in the first round is much higher than that for those picked on day two and three. But so is the opportunity cost measured in terms of potential high-impact players that would have to be passed up. Thinking about that made we wonder whether the balance of QB hit rate relative to opportunity cost is actually better or worse in different parts of the first round, or in later rounds of the draft. This article represents a first stab at quantifying the balance of potential reward to opportunity in order to determine whether there are better or worse places in the draft to pick quarterbacks.

---

Methodology

For this purpose, I developed a novel metric, which I’ll call Risk Ratio (RR, Ron Rivera reference intended). RR is simply the ratio of the chance of a particular draft selection being successful (Success Rate, SR) to the Opportunity Risk (OR), defined as the chance that players who were passed up might have been successful:

RR = SR/OR

RR can be used to quantify risk-taking in a wide variety of draft decisions, depending on what variables are plugged in as SR and OR. In this article I will use the RR to quantify how the risk-reward balance of selecting a QB versus any other position changes as the draft proceeds. For this purpose, I define SR as the chance of selecting a long-term starting QB at different positions within the draft, while OR is defined as the chance of drafting a player, at the same point in the draft, who becomes a starter at any other position (non-QBs) within his first three years in the league.

Effectively, I am weighing the chance of finding a long-term solution at QB in the draft against the opportunity cost of passing up players who might provide more near-term starting help at other roster spots. I chose this comparison, because I think it describes a major facet of the decision facing the GM of a quarterback-needy team, with numerous holes and near-term needs at a variety of other roster positions. I will discuss the implications of the apples to oranges comparison when I present the results.

Draft Pick Ranges

Because QBs are fairly scarce commodities at most positions in the draft, it was necessary to group picks into ranges to get large enough sample sizes for meaningful analysis. Since success rates for drafting any position fall off more steeply at the start of the draft and then more gradually later on, I attempted to set sampling ranges which mirror that trend. I settled on the following sampling ranges, which roughly approximate a power of two sequence: 1^st overall pick, 2^nd – 3^rd picks, 4^th-7^th picks, 8^th-15^th picks, 16^th-32^nd picks, second round (picks 33 – 64), third to fourth rounds, fifth to seventh rounds.

Success Rate (SR) - QBs

Rivera has indicated that he is focused on finding Washington its next franchise QB. The first challenge is then to define “franchise QB.” The term implies a long-term starter that the team is happy with and would not be happy about losing.

Teams do occasionally misjudge what they have and let good drafted QBs go, such as Drew Brees in San Diego, Ryan Tannehill in Miami, and Kirk Cousins in DC. Therefore, to catch all the drafted franchise-worthy QBs, I settled on the following definition: drafted QBs who were extended as starters after their rookie contracts, or who were signed by another team as a starter.

A QB who started for the duration of his rookie contract and then signed somewhere as a backup is considered a miss. I don’t think Rivera or most fans would be happy with that outcome. Also, teams often hang on to their high-round draft picks longer than they should, which means that a fair number of QBs who will ultimately be declared busts are still starting three years after they were drafted, which might otherwise be more directly comparable to my three-year starter criterion for Opportunity Risk (see below). Therefore, I felt the second starting contract was the best criterion.

To calculate the QB draft Success Rate, I used the Pro Football Reference Draft Finder to identify all the QBs drafted from 2009 to 2018 and counted those within the designated draft ranges who met the criteria as long-term starters, then divided the counts by the total numbers of QBs drafted in the same range.

Draft Year Range

Because it appears that teams’ approach to drafting QBs is changing rapidly, I made an effort to keep this analysis as recent as possible to make it as relevant as possible. This type of analysis requires fairly large sample sizes, but using too long a time window risks pushing back into a previous era. I settled on a ten-year period from 2009 to 2018 to balance competing requirements, as I will explain here and later.

Inclusion of the 2017 and 2018 QB classes becomes a bit challenging, because not all of the QBs drafted in 2017 have reached the ends of their rookie contracts, and none of the class of 2018 have. However, I felt it was safe to make the call on the remaining QBs from 2017, and projecting the class of 2018 felt pretty safe, as well. By going out on a bit of a limb there, I was able to push the analysis window to within the last three years.

The sample sizes are still pretty small throughout much of the first round and, where this becomes a real problem, I will discuss how pushing the year range back to the beginning of the salary cap era (1994) to get more players would have changed things.

Opportunity Risk (OR) – Draft Success Rates at other Positions

Opportunity Risk was defined as the chance of selecting a player at a non-QB position who went on to become a starter by his third NFL season. The player’s third season was selected because it often takes players a few years to earn starting spots. I also used a games started criterion for non-QBs, as opposed to the second contract criterion used for QBs, because the workload involved in looking up contracts for the 2,200-plus players drafted in a 10-year span would have been overwhelming.

Defining a “starter” can also be challenging, because not all established starters start 16 games every season. Setting too low a bar for defining starters, such as starting one to three games, might catch too many backups doing spot starting duty. After testing criteria from five to eight games started in the third league season, I settled on six games started as a reasonable criterion to exclude backups without missing too many true starters who missed starting time to injury. This isn’t perfect, as some backups will have sneaked in and some true starters will have been missed. But I expect those types of errors to cancel each other out across the 2,200-plus players that went into the analysis.

---

Photo by Andy Lyons/Getty Images

Quarterbacks and Draft Ranges

Before we get to looking for the risk-reward sweet spot, it will be instructive to have a look at the quarterbacks picked at different points in the draft during the selected decade. I will break these down into QBs who met the criterion as “long-term starters” and those that didn’t, with comments on the few QBs who were difficult to categorize.

First Overall Pick

Seven QBs were picked first overall in the ten drafts from 2009 to 2018. This is by far the most QB-dense pick in the whole draft. Of those QBs, five met the criterion as long-term starters: Cam Newton, Matt Stafford, Andrew Luck, Jared Goff and Sam Bradford. Bradford pushed the boundaries of the definition, but he did start 14 games for Philadelphia in 2015, after completing his rookie contract with the Rams, and then started 15 games the following season in Minnesota. I am projecting Baker Mayfield to become a long-term starter based on his 2018 and 2020 seasons.

Only Jameis Winston failed to continue as a starter following his rookie contract.

Second and Third Overall Picks

The hit rate on QBs selected second and third overall in these ten drafts has been appalling, and represents a huge drop from the first overall pick. Of the six QBs selected at these picks only Carson Wentz met the long-term starter criterion, and there is reasonable doubt about whether he will turn things around in Indianapolis. The QBs who failed to catch on as long-term starters are Marcus Mariota, Blake Bortles, RG3, and Mitch Trubisky. I am projecting Sam Darnold to join the list of failures at the third overall pick, based on the horrible first three years of his career with the Jets (59.8% completion percentage, 45 TD, 39 Int, 6.6 Y/A, 2020 QBR 40.1).

The low QB hit rate at these picks does not appear to be just some kind of statistical anomaly of using a small sample, either. Reaching back to the beginning of the salary cap era (1994) brings the sample up to 14 drafted QBs, of which only Steve McNair, Donovan McNabb and Matt Ryan join Wentz as long-term starters. The others read like a list of famous draft busts including Heath Shuler, Ryan Leaf, Akili Smith and Vince Young. While that would bring the hit rate up to a more respectable hit rate of 29%, it is still far short or what one might expect, and what teams have achieved picking slightly later in the draft.

Fourth through Seventh Overall Picks

Only two QBs were picked across these four picks in the decade of interest. Mark Sanchez, picked fifth in 2009, signed to back up Nick Foles in Philadelphia following rookie contract with the Jets. I am projecting 2018 seventh overall pick Josh Allen to become a long-term starter. I would caution against drawing any conclusion from the apparent 50% success rate at this draft range, because it is only based on two players.

Once again, pushing the time window back to 1994 to get a larger sample boosts the hit rate by adding long-term starters Phillip Rivers, Kerry Collins and Trent Dilfer, who just barely meets criterion, and only Byron Leftwich, who doesn’t. Those additions would bump the hit rate up to 67%. We will soon get a few more data points in this range, as the QB classes of 2019 and 2020 develop, and what happens here in 2021 is anyone’s guess. In a few years this picture could look very different, but for now the estimates sit somewhere between 50-67%.

Eighth to Fifteenth Overall Picks

This seems to be the boom or bust draft range. Of the seven QBs selected here; Ryan Tannehill, Patrick Mahomes and Deshaun Watson have all become excellent to elite long-term starters, while Christian Ponder, Blaine Gabbert, Jake Locker and Josh Rosen were all busts.

Extending the sample back to 1994 wouldn’t have changed things much, since added long-term starters Ben Roethlisberger, Jay Cutler and Daunte Culpepper are mostly offset by busts Matt Leinart, and Cade McNown, boosting the hit rate modestly from 43% to 46%.

16^th to 32^nd Overall Picks

The second half of the first round (plus pick 16) was another dead zone for drafting QBs in this decade. Seven of the eight QBs selected in this range failed to become long-term starters: Teddy Bridgewater, Josh Freeman, Brandon Weeden, Tim Tebow, EJ Manuel, Johnny Manziel, Paxton Lynch. Lamar Jackson is an easy projection as a long-term starter based on the stellar start to his career in Baltimore.

Extending the sample back to 1994 would boost the hit rate here from 12.5% to 26.3% by adding long term starters Joe Flacco, Aaron Rodgers, Drew Brees and Chad Pennington, and six guys who didn’t make it, including Redskins’ experiments Jason Campbell, Rex Grossman and Patrick Ramsey.

Round 2

Three of the ten QBs drafted in the second round met criterion as long-term starters: Andy Dalton Derek Carr and Colin Kaepernick. The seven QBs who failed to meet criterion were Geno Smith, Brock Osweiler, DeShone Kizer, Jimmy Clausen, Pat White, Christian Hackenberg and Jimmy Garoppolo. Garoppolo was traded during his rookie contract in 2017 to join a QB carousel in San Francisco, where he did not catch on as the full-time starter until 2019. The 49ers trade up to the third pick in the 2021 draft signals the end of his starting opportunity in Santa Clara.

Kaepernick might strike some readers as a controversial inclusion as a long-term starter. However, he meets the criterion, because his rookie contract was extended in 2014 to a six year $114,000,000 contract ($19,000,000 APY), which was starting money at the time. It can be debated whether his career as an NFL starter ended three years later because of his performance on the field or other factors. His performance during his rookie contract was clearly better than during the second and third years of the extension.

Rounds 3 to 4

Only three of the 25 QBs drafted in rounds three and four from 2009 to 2018 became long-term starters: Russell Wilson, Kirk Cousins and Dak Prescott.

The list of “unsuccessful” QBs selected in this range is headlined by Super Bowl LII MVP, Nick Foles. Foles challenges my long-term starter criterion more than any other QB in this analysis. Three years into his rookie contract he was traded to St Louis, where he started 11 games in 2015. The two-year, $25,000,000 contract he signed with St Louis was a high-end backup/replacement starter deal, and he was released after the first season. While he has since gone on to sign low-end starting deals and high-end backup deals with Philadelphia, Chicago and Jacksonville, he has not started more than seven games in a season for any team after leaving St Louis. I can’t list him as a long-term starter in good conscience.

The other 21 QBs drafted in this range who did not meet criterion were: Jacoby Brissett, Colt McCoy, Mike Glennon, CJ Betahard, Logan Thomas, Cody Kessler, Matt Barkley, Bryce Petty, Tom Savage, Landry Jones, Ryan Mallett, Stephen McGee, Sean Mannion, Joshua Dobbs, Connor Cook, Cardale Jones, Garrett Grayson, Ryan Nassib, Mike Kafka, Davis Webb and Tyler Wilson.

Rounds 5 to 7

No long-term starting QBs were drafted after the fourth round from 2009 to 2018. There is little point in listing the 44 QBs selected in the later rounds. The best of the QBs selected in this range, in approximate descending order were: Tyrod Taylor, Trevor Simian, Jeff Driskel, TJ Yates, Brett Hundley, Brandon Allen, AJ McCarron, John Skelton, Nathan Peterman and Kevin Hogan.

---

Draft Success Rates

The first figure shows the past probability of picking a starter, as defined in the Methodology section, for QBs (orange) and all other positions (blue) from 2009 to 2018. The alternative QB success rates generated by including data from 1994 to 2008 are plotted as the gray line.

It should come as little surprise that teams are more likely to find a starter at any other position than QB at any point in the draft. There are a few reasons for that. First, QBs are relatively rare, compared to all other non-specialist positions in the draft, making this a choice between a shallow talent pool at QB and a much deeper one at all the other positions. Second, long-term starting QBs are even rarer and competition from other teams for the few good ones is more intense than any other position. Third, QB is harder to project to the NFL than most other positions.

Fourth, and probably most importantly, my choice of criteria for starters at QB and other positions rigs the competition in favor of the other positions since I’ve set a much lower bar for non-QBs than QBs. If I had based the definition of Opportunity Risk on the chance of picking long-term starters at other positions, or made some attempt to define non-QBs of equivalent value to long-term starting QBs, the bars would be a lot closer together.

Using very different criteria for QBs and non-QBs should be acceptable for this analysis since the aim is to compare the risk-reward balance for selecting QBs at different points in the draft. As long as the same definition of Opportunity Risk is applied in each Draft Pick Range, it is not likely to bias the conclusions. I think the choice of near-term starting help at other positions is defensible, but I could have easily chosen other comparisons, such as day one starters, or long-term starters (those might be topics for another day).

If the aim had been to take a deep dive into whether teams are really better off picking a QB or another position at any particular draft pick, then I would have needed to find a way to define non-QBs of equal value to a long-term starting QB. It is not clear to me how to do that, since existing metrics like AV are heavily biased by positional value, which would be expected to influence the outcomes.

Another thing to note is that the success rate for drafting near-term starters at non-QB positions does not fall off exponentially through the first round, like you might expect if you are used to thinking in terms of the Jerry Jones or Rich Hill trade value charts, the former of which I have shown tracks well with the probability of picking blue-chip players. Teams picking in the middle to later parts of the first round have done better than expected, which might have something to do with the fact that those tend to be the better teams. In fact, the variation in success rates from 0.75 to 0.86 across picks 2 to 32 is likely to be close to the margin of error. Therefore, the success rate for drafting near-term starters is probably best described as close to flat throughout the first round after the first overall pick.

Last of all, two numbers in this chart should be taken with big grains of salt. The 100% success rate for drafting non-QBs with the first overall pick is only based on three players: Myles Garrett, Jadeveon Clowney and Eric Fisher. Similarly, as noted above, the 50% success rate for picking long-term starting QBs at picks 4-7 is only based on two players: Josh Allen and Mark Sanchez.

Since the sample sizes were low at picks 4-7, as well as surrounding ranges 2-3, 8-15 and 16-32, I also generated alternative probability estimates by extending the range of draft years back to 1994. This increased the hit rates at all of these ranges but 8-15. This might provide more reliable statistical estimates, but it pushes the sample back to a previous era when competition for QBs might have been less intense.

---

Is there a sweet spot for drafting QBs?

With all those preliminaries out of the way, here is how the balance of Success Rate versus the Opportunity Risk played out for teams drafting QBs throughout the most recent measurable NFL drafts.

As expected, the best Risk Ratio falls on the first overall pick (higher values mean higher ratio of reward to Opportunity Risk). But describing the first overall pick as the “sweet spot” for picking QBs is kind of stupid. Over the last decade it has been the best place to pick any position. That does represent a bit of a change from previous decades, when first-pick buss were more common. That is great news for Jacksonville, holding the first pick in the 2021 draft, but what about everyone else?

After the first pick, the first round holds some surprises. Over the decade in question, there were two notable bad spots for drafting QBs. Teams picking QBs second and third overall did exceptionally poorly, given the options available to them, and in doing so passed over large numbers of close-to-sure-thing starting talent at other roster positions.

The second Risk Ratio dip was in the second half of the first round. That came as a bit of surprise to me, because it is where the better teams tend to pick. As it happened, that was not so much the case from 2009 to 2018, when perennial winner Baltimore was the only team to find a late-first-round bargain in Lamar Jackson. Unsuccessful teams in this range included Cleveland (Brandon Weeden, Johnny Manziel), Denver (Tim Tebow, Paxton Lynch), Minnesota (Teddy Bridgewater), and Tampa (Josh Freeman).

This is also a bit of a contrast with the prior decade, when Green Bay, San Diego (unknowingly) and Baltimore all managed to find future franchise QBs in this range. As with picks two and three, the second half of the first round is a great place to find starters at other roster positions, resulting in a poor Risk Ratio for selecting QBs in this range.

The QB draft Success Rates in the ranges of draft picks from two to 32 were all based on small sample sizes. To provide more reliable estimates, I also generated alternative figures by extending the sampling back to 1994. This boosted the Risk Ratios across this range, but it did not completely eliminate the dips at picks 2-3 and 16-32.

Perhaps the biggest surprise was how relatively modest the falloff of Risk Ratio is from first to the second round, and even from the second to the middle rounds. While the hit rate for drafting QBs declines as the draft progresses, with a few bumps and dips through the first round, so does the hit rate for drafting starters at other positions. As a result, the total cost of selecting QBs does not fall off as precipitously through the second through fourth rounds as one might expect, based on QB draft success rates alone.

Now, to answer the question, for teams not holding the first overall pick, the least risky places to spend draft resources on finding a QB of the future would appear to be the fourth through 15^th picks of the first round, followed by the second round.

---

Photo by Michael Zagaris/Getty Images

Discussion

Does this mean that teams should avoid picking QBs with the second or third picks, or in the second half of the first round, even when their scouts rate a prospect in that range? Of course not. NFL teams face a big challenge projecting college prospects to the pro game and the best tool available to them to help reduce the uncertainty of that decision-making process is their scouting departments’ player evaluations. When a GM is on the clock, the best decision is to select amongst the players that his scouts rated the highest, regardless of whether players at the top of his board play quarterback or other positions.

What I think these results do is to give some hints about how things might be about to change in response to the fact that starting QB contracts are escalating faster than the salary cap. The results I have shown here were drawn from drafts dating back to 2009, when teams could expect to extend a starting QB if they drafted a good one. Under those market conditions, teams could find reasonable value relative to opportunity cost when drafting QBs in the early-to-middle part of the first round and into the second round, with some value to be had in the third and fourth rounds.

The unprecedented demand for QBs on rookie contracts is showing signs of driving a panic rush on QBs in the top end of the first round in 2021, with QBs who might have previously carried a second round scouting rating being discussed as top ten prospects. The problem with this scenario, if it truly plays out like that, is that the supply of starting quality QB prospects is showing no signs of increasing to meet demand.

By trading into the top ten to target a prospect whom they might previously have expected to find in the late first to second round, they aren’t actually increasing their chance of drafting their QB of the future. But what they are doing is driving up the cost of looking for him, measured in terms of the opportunity cost of blue chip players passed up at other positions plus additional draft picks given away to trade up.

I can’t see how a situation in which teams are forced to trade up into the first round to have a chance at drafting second tier QB prospects is sustainable. My opinion is that a market correction is inevitable, once teams begin to fully appreciate what their search for new rookie QBs is doing to the rest of their rosters. But I appear to be in the minority on that.

In the meantime, a draft day feeding frenzy on QBs would be great news for teams without an immediate need at the position, who would expect to see top prospects at other positions pushed down the board by over-drafted QBs.

Acknowledgement

I always thank James Dorsett for his insightful comments and editorial assistance. This article benefited even more than usual from a discussion regarding the likely impact of escalating QB salaries on draft behavior, above and beyond his usual helpful input. I would also like to thank the HH commenter mentioned at the start for sowing the seed that led to this article.