The QB Sweet Spot Revisited

Prior to the 2021 draft, many Washington football fans thought it was a given that the team would select a quarterback somewhere in the draft. On opening night, an opportunity arose to draft one of the top three QB prospects, when Justin Fields slid out of the top ten. Rivera’s team did not take the bait, and stayed put at #19, where they selected linebacker Jamin Davis.

In an interview on Chris Collinsworth’s podcast, highlighted by Hog’s Haven’s KyleSmithforGM, Rivera explained his reasoning for not trading up to draft one of the top five QB prospects:

Collinsworth: “Such an interesting year. We saw Taylor Heinicke, we saw so many different people, Dwayne Haskins you let go. Lot of people thought this year might be a year that, despite the fact that you signed Ryan Fitzpatrick, you might have made a move in the draft and come up and take one of those five quarterbacks that everybody was talking about. Did you consider that at all?”

Rivera: “Oh yes. Oh, we did, Chris, we really did, but the thing also that stood out in our mind was the amount of draft capital, the price that we were going to have to pay to try to get the right one, the one we liked the most. That was hard, because we’re looking at our teaming saying, “God, we need a linebacker, we need a left tackle - for the future - we need so many other things that just getting the one guy might not have put us in that position.” Now, with what we’ve done, we’ve secured some of our positions. We feel good about those positions, and the rest of the team. And, I’ll say this, this is one of the things we talked about too was - other than Tom Brady - who’s won multiple Super Bowls? You know, you can say Peyton, you can say Eli, but then who else? You look at the other guys that have won them, and there are a lot of one-timers. And what do those one-timers have in common? Well, they’re guys that were very efficient, they managed the game, they didn’t turn it over a lot, and they made plays when they had to.

So, we’re just wondering if, is that the other formula? If you’re fortunate enough to find this guy, you ride him. But if you don’t, and you find these guys, what says you can’t win with them? Other than Tom Brady, think about this. So we went through that back and forth. Now I’m not saying that if the right guy, in our mind, is there, that we don’t ride with him. So we’ll see.”

My takeaway from Rivera’s comments is that, while the new Washington front office recognizes the value of having an elite quarterback, they are not willing to sell the farm in an effort to find one. Rather, Rivera’s thinking about drafting QBs involves weighing the opportunity to select a top prospect against the opportunity cost involved in passing up top players at other priority positions with the draft picks that would have to be given away to trade up.

In his comments to Collinsworth, Rivera points to another formula involving maximizing use of draft capital by spending it on other positions where the value aligns with players available and building a team that is capable of winning a Super Bowl with a more average game-manager type player at QB. In the current QB market, having a shot at one of the top QB prospects requires picking in the early part of the first round. However, if a team is willing to lower the bar slightly regarding the talent level of quarterbacks they are seeking that opens a wider range of options for finding a signal caller later in the draft or in free agency.

A few weeks before the draft, I followed a similar train of thought to that expressed in Rivera’s comments in an attempt to determine whether there is a “sweet spot” in the draft to optimize the balance of opportunity to opportunity cost. The results showed a sharp dip in value at pick numbers two and three that just didn’t make any sense. Since Rivera has indicated that he is actually thinking along these lines, I thought it would be worthwhile to revisit that analysis to see if I might have got something wrong.

Rethinking the Opportunity-Opportunity Cost Ratio

The original intention of this analysis was to measure the ratio of Opportunity to Opportunity Cost associated with selecting quarterbacks at different positions in the draft to see if there are any peaks and troughs. In this analysis “Opportunity” means the chance of selecting a long-term starting QB at the chosen range of draft picks. On first thought, it seemed reasonable to use past hit rates for selecting QBs to estimate the chance of selecting “the guy” at any particular draft position. The hit rates for drafting long-term starting QBs in the salary cap era (1994 to 2018, excluding more recent years that are too early judge) are as follows:

I decided to use the whole cap era in the revised analysis because the numbers of long-term starting QBs selected in some draft ranges got too small when I tried to keep it more recent. The more I looked at that dip in hit rate at picks two and three, the more it bugged me.

Then it finally hit me. That dip is almost entirely due to the teams making the selections at picks two and three, and has very little to do with the availability of long-term starting QBs at those picks. The reason that has to be the case is that all the long-term starting QBs who contributed to the higher success rate at picks four to seven were on the board when teams were making selections at pick numbers two and three.

Once I had that realization, it became clear that the correct measures of Opportunity and Opportunity cost are the relative availabilities of long-term starting QBs and equivalent-value players at other positions, respectively, rather than past hit rates for teams selecting them. Hit rates are a flawed measure because they conflate two things: availability of players that meet criteria and the ability of teams to identify the better players. This realization led me to change the specific variables used to measure Opportunity and Opportunity Cost, while keeping the overall concept the same.

Revised Methodology

As in the previous analysis, efficiency of draft resource utilization was measured as the ratio of Opportunity to Opportunity Cost, which I have renamed the OOCR:

OOCR = Opportunity at pick x/Opportunity Cost at pick x

Opportunity. In the revised analysis, Opportunity was measured as the Relative Availability of long-term starting quarterbacks throughout the draft. As in the previous article, a long term starting QB was defined as a QB who was extended as a starter, or signed a starting contract with another team following his rookie contract.

Relative Availability was defined as the proportion of long-term starting QBs within the total talent pool who were available at each pick in the draft over the period from 1994 to 2018. This measures the opportunity that was available to teams on the board at a given pick number, without the confounding influence of the decision making of the teams that actually picked QBs at that pick number.

Relative Availability was calculated by counting all the long-term starting QBs selected at a given pick number or later in the draft, and dividing by the total number of QBs selected at that pick number or later. Sticklers might argue that this overestimates true Relative Availability because it ignores all the draft-eligible QBs who went undrafted. I’m not going to worry about that.

Opportunity Cost. I also had a rethink about how to measure Opportunity Cost. In the previous analysis, I used past hit rates for drafting players who became starters by their third year in the league. That was a fairly arbitrary choice, and I was only looking for a reasonable standard of something else that a team might be looking for. Eventual starters at other positions seemed suitable. That was OK for that analysis because I was only looking to compare relative value across draft position.

In the revised analysis, I have attempted to make the absolute values of the OOCR more meaningful by comparing Opportunity, defined above, to Opportunity Cost, defined as the Relative Availability of players at other positions with equivalent-value to a long term starting QB. To do that, I decided to use Pro Football Reference’s statistic Approximate Value (AV), which is designed to provide a single number which rates the overall productivity of players across positions in a single season on the same scale.

I decided to use single season AV, rather than Career AV, because the latter is a cumulative statistic and can weight longevity as much or more than actual productivity. I settled on AV in a player’s third year in the league because it can often take players a few years to earn a starting spot, but if you are considering between drafting a QB and a player at another position, you probably want the non-QB to start fairly early in his career.

The real challenge here was to determine what AV value is equivalent to that of a long-term starting QB. After some trial and error, I decided to set the minimum equivalent AV value near the bottom of the range for long-term starting QBs, so we are selecting players at other positions who are at least as good as a QB who is just good enough to earn a long-term starting job.

To find the equivalent single-season AV, I calculated the median single-season AV over the playing careers of all the long-term starting QBs identified in the first article. I then ranked all the selected QBs by median single-season AV and looked for an appropriate lower cut-off. The bottom 5 QBs in the ranking were as follows:

• Trent Dilfer, median AV 6
• Kerry Collins, median AV 8
• Colin Kaepernick & Sam Bradford, median AV 9
• Joe Flacco & Carson Wentz, median AV 10

I decided to set the threshold at AV >7 to exclude Trent Dilfer and keep everyone else. To give you an idea of how good a player this criterion selects, Terry McLaurin just misses the cut, since he has posted a single season AV of 7 in each of his first two years in the league. That might seem crazy, but it is a reflection of the fact that a just good enough starting QB makes a bigger contribution to his team’s success than a very good player at most other positions.

To summarize, Relative Availability of equivalent-value non-QBs was defined as the number of non-QBs with AV >7 selected at each draft pick or later in the draft divided by the total number of non-QBs selected at that draft pick or later. This value was plugged in as the Opportunity Cost in the OOCR calculation.

Sampling. Because long-term starting QBs are very rare compared to other positions, the OOCR value does not change very much between picks where they are selected. Therefore, Relative Availability of QBs and equivalent-value non-QBs, as well as the OOCR, were just measured at picks where long-term starting QBs were selected in any draft from 1994 to 2018.

The QB Sweet Spots Revealed

The Relative Availability of long-term starting QBs (blue dots) and equivalent-value players at other positions (orange dots) is plotted in the next figure at each draft pick number where a long-term starting QB has been selected in the salary cap era.

The Relative Availability of both types of quality players drops as the draft proceeds, but Relative Availability of QBs drops more steeply than non-QBs. The steep drop in availability of starting-quality QBs through the early part of the first round makes intuitive sense, because QBs are so highly valued, the best players tend to get taken out of the talent pool earlier in the draft than other positions. The fall-off in availability of starting-quality QBs hits an inflection, becoming more shallow than steep, around the second half of the first round. The plot of QB Relative Availability follows a logarithmic curve (blue line), and the curve fit explains an impressive 97% of the total variance.

The surprising result is that the Relative Availability of equivalent-value non-QBs falls off more gradually than QBs throughout the first two days of the draft. As you can see, the non-QB line stays above the QB-line from about pick number four to five throughout the second and most of third round, until the curves start to come together near the start of the fourth round.

This suggests that teams should have a better chance of picking a high-impact non-QB than a long-term starting QB at any position in the draft from around the fifth overall pick to the end of the third round. After that point, both types of player are more or less equally rare. Let’s see how that translates into the ratio of Opportunity to Opportunity Risk in the next plot.

To orient you, an OOCR value of one indicates that a team is just as likely to pick a long-term starting QB as an equivalent-value player at another position. A value greater than one means they are more likely to pick a long-term starting QB. And a value below one means they are more likely to pick an equivalent-value non-QB.

The plot of OOCR vs. draft pick number reveals two sweet spots in the draft for picking QBs, where the Opportunity exceeds the Opportunity Cost. One is at the first overall pick, and the second is in the late fifth to sixth rounds where Marc Bulger (pick #168), Matt Hasselbeck (pick #187), and Tom Brady (pick #199) were selected.

However, I wouldn’t make too much out of the OOCR peak being greater than one in the late 6^th round. Looking back to the Relative Availability plot, you can see that in that range the numbers for QBs are fairly jittery around the trend line. This is likely a result of what statisticians call “sparseness”. Because long-term starting QBs are so rare in the sixth round, one occurrence makes a big difference. The main point is that the OOCR is higher in the later rounds of the draft than it is earlier on.

The next big takeaway message of the OOCR plot is that there is a big dip stretching from around the middle of the first round, where the points fall below about 0.8 to about the fifth round where the points finally start trending back up toward 1. At these positions in the draft, the Relative Availability of quality players at other positions is significantly higher than that of starting quality QBs, making this the costliest part of the draft to select a QB. Teams can therefore optimize their expenditure of draft capital on QBs by focusing their selections on the first half of the first round and the fifth through sixth rounds.

However, although, the OOCR is comparatively low from the second half of the first round through the third round, it is not really that low in absolute terms. The chance of drafting a long-term starting QB is only about 0.6 to 0.8 lower than the chance of drafting an impact player at another position in this range.

That brings me to my final takeaway. Depending on a team’s risk tolerance, the Opportunity Cost associated with passing on potential impact players at other positions may not be prohibitively high at any point in the draft, given the potential reward of hitting on a starting QB.

What cost would be too high? That depends on the GM’s risk tolerance. Despite the “Riverboat” moniker, Rivera’s decision on draft night suggests his tolerance limit might be somewhere around a doubling or tripling of the Opportunity Cost relative to Opportunity, which is what would have happened if the WFT had traded away one or two first round picks to move up for Justin Fields or Mac Jones.

Conclusions

Ron Rivera’s comments on the 2021 draft indicate that the team weighs opportunity against opportunity cost in its decisions about whether to use draft capital on QBs or other positions. Calculating the ratio of Opportunity (Relative Availability of starting quality QBs) to Opportunity Cost (Relative Availability of equivalent-value players at other positions) revealed two sweet spots for drafting QBs which they might want to take advantage of in the search for a franchise QB.

At the first overall pick, and continuing into the first half of the first round, the availability of starting quality QBs is highest, and compares fairly favorably to that of impact players at other positions. In this range, teams have the highest probability of picking a long-term starting QB.

The second sweet spot is in the late fifth to sixth round, where teams really have nothing to lose by spending a draft pick on a QB, because the chance of picking a quality QB or non-QB is about equally low. While the chance of finding a starting QB in this draft range is very low, there is an additional possibility of finding a viable backup, which has significant value. And the Opportunity Cost of selecting a QB is low, since impact players at other positions become rare this late in the draft.

From the middle of the first round through the fourth round, the Relative Availability of quality players at other positions is higher than that for QBs. The Opportunity Cost associated with drafting QBs is highest in this range. However, even at its least favorable point, in the second round, the Opportunity Cost does not become so high that selecting a QB could be seen as a wasted pick.

What Does This Mean for Rivera’s “Other Formula”?

In his comments to Chris Collinsworth, Rivera revealed that the team actually discussed the possibility of trading up for a QB on draft night, but eventually decided that the cost of moving to select the one they liked would have too high an impact on their ability to build the rest of the roster. Was it Justin Fields or maybe Zach Wilson? We may never know.

The main point is that Rivera is keenly aware of the potential pitfalls of going all in to pursue a franchise QB, and has pointed to another path to success: building a team that’s capable of winning a Super Bowl with an average talent, game manager at QB. That player might already be on the roster, someone they draft in the next season or two, or an affordable veteran free agent.

This offseason is about the most hopeful I can remember the fanbase being since, maybe, 2012. If things unfold as many of us expect, the WFT could once again become a victim of its own success with respect to positioning itself to draft one of the top QB prosects. My analysis suggests that picking outside of the top 15 should not be a barrier to selecting a QB, as decent to good value can still be found later in the draft when measured in terms of the balance of Opportunity to Opportunity Cost. This is particularly the case if, as Rivera has indicated, the team is open to drafting a competent starter, rather than fixating on finding a superstar.