Quarterback Cap Hit Doesn’t Affect Team Performance…Yet

Now that Washington Football Team’s season is over, the conversation on Hogs Haven has returned to one of the usual topics amongst fans of teams without franchise quarterbacks. This discussion frequently settles into a debate about whether the team should risk the long odds of trying to draft the next starting QB, or take the much more expensive option of trading for an established veteran.

Setting aside the question of which premium veteran would like to make their next move to Ron Rivera’s rebuilding project, the key consideration with signing a vet is the escalating salaries of starting quarterbacks. Several recently signed contracts for top veteran starters have Average Per Year (APY) values in the range of $40 to $45 million, representing 20% or more of the salary cap at signing.

If this trend continues, it may push the cost of keeping a premium veteran starter well above the pain threshold for NFL teams, which is said to fall somewhere between 13 and 14% of cap, forcing GMs to make increasingly difficult decisions about which other star players to let go to save space.

Over the last few weeks, KyleSmithforGM has published a series of articles exploring these issues, including one suggesting that the Baltimore Ravens should consider moving on from 25-year-old, recent league MVP and All-Pro QB Lamar Jackson. A key line of evidence for this train of thought was presented in an earlier article, which showed that over the past decade, teams with relatively inexpensive quarterbacks have dominated the Conference Championships and Super Bowl.

The approach taken in that article, examining QB cap hits in the year that teams made it to Conference Championship game or beyond, is a sensible initial step in asking whether QB cap hits influence team performance. However, a closer examination of the underlying data reveals certain complexities, which might make that simple approach prone to sampling bias.

Most importantly, QB cap hits can vary enormously from year to year within a contract. For example, Aaron Rodgers’ current contract, signed in 2018, has relatively affordable seasons in 2018 and 2020, at 11.3% and 10.6% of cap respectively, and relatively expensive seasons in 2019 and 2021 at 14.9% both years, culminating in an exorbitant 22% of cap in 2022. The Packers finished with identical 13-3 records and made it to the NFC Championship game consecutively in the expensive 2019 season and the affordable 2020 season, then barely missed a return in the expensive 2021 season.

Is that evidence that having an affordable star quarterback is the key to making it to the championship round, or that paying too much for a quarterback limits a team’s ability to make it to the Super Bowl? It’s difficult to say.

More to the point, because QB cap hits vary considerably throughout the duration of a contract, focusing exclusively on cap hit in the year a team made the championship round may lead to overlooking much higher or lower cap hits in previous years. It also throws out a lot of data that might shed more light on this question by ignoring highly competitive seasons by teams that might have lost a divisional playoff by a single play, one bad call or just facing another really good team.

After I was done laughing about the Lamar Jackson article, I got thinking about how to put the concept that QB cap hit has a major influence on team performance to a more rigorous test. Here is what I came up with.

Hypothesis: QB cap hit is a key determinant of team success. The rationale for the hypothesis is that teams with cheap quarterbacks will be able to add and retain top end talent at other positions, whereas teams with expensive ones will struggle to maintain competitive rosters.

Prediction: It follows from the hypothesis that, over time, a team’s success on the field will vary in proportion to the relative affordability of its starting quarterback. Specifically, if we plot some measure of team success against QB salary as a percentage of the salary cap, we should expect to see a downward trend. As QB cap hit increases, team performance should decrease. The steepness of this relationship is an indication of how strongly QB cap hit influences team performance.

Note: for readers who are short of time and/or just want to get to the meat of the results, you can skip right ahead to the section Results - Part 2 without missing too much.

---

First Approach

Sample: In order to test the hypothesis, I initially set about plotting team success as a function of QB salary expressed as a percentage of salary cap. KyleSmithforGM’s sample of teams that made the Championship round over the last decade makes sense, so I stuck with it. To address whether QB salary limits team success, we have to look at teams that have proven capable of making deep playoff runs. I don’t think we would learn much from QB cap hits on teams that haven’t won a playoff game in the last decade.

One minor difference, I extended the sampling period back to 2011 on occasion to get a bit more data.

Team Success: My measure of team success for this analysis is Total Wins in a season, including both regular season and playoff games. This was chosen because the analysis requires a continuous variable. I also took note of playoff finish and will refer to it as needed.

I realize the 2021 season isn’t over, but I didn’t let that stop me. The remaining game only makes a one win difference to one team. That’s not going to throw my conclusions off.

QB cap hit data were sourced from OverTheCap and all other data from Pro Football Reference, including ESPN’s Total QB Rating (QBR).

Results – Part 1

Let’s kick things off by having a look at the Kansas City Chiefs. The first figure shows Total Wins plotted as a function of QB cap hit (% Cap) from 2013 to 2021.

On first look, it appears that we might be on to something here. There is a weak trend for team performance to decline as QB cap hit increases. That might be a little hard to see from the data points, so I plotted a regression line, which illustrates the best fit to central tendency of the data. The coefficient of the line fit, R², provides a measure of how strongly QB cap hit predicts the variations in Total Wins. In this case, QB cap hit explains 0.2137, or 21.37% of the variance in Total Wins. That’s not highly predictive, but it seems to suggest that QB cap hit is having some influence.

When I looked a little closer, though, I realized that there is a very important confounding factor at play here. There are data from two quarterbacks of very different ability levels on that graph. Alex Smith, who started for the Chiefs from 2013 to 2017, was a mediocre performer who posted an average QBR of 56.74 in five seasons and managed to lead the team to just one wild card playoff win. Patrick Mahomes burst onto the scene in 2018 as one of the NFL’s superstars, posting an average QBR of 74.5 across four seasons as primary starter and leading the team to a Superbowl championship, an AFC Championship and a divisional playoff win.

That creates a problem for the analysis. Since Mahomes’s data is mostly from his rookie contract, his cap hits are much lower than Smith’s. Therefore, we can’t really tell if the high level performance associated with low cap hits in the upper left part of the graph is really due to the low cap hit, or to the improved QB play with Mahomes under center. Aside from a few devoted Alex Smith fans, I suspect most of us would say it’s the improved QB play.

To see what I mean, let’s have a look at what happens when the data from the two QBs is plotted separately.

Keeping the data for each QB separate, to control for differences in playing ability changes everything. Now we can see that the apparent trend in the first graph was an illusion, created by combining high performance from a QB with a low cap hit and lower performance from a QB with a higher cap hit.

Team performance with Mahomes at QB doesn’t show any obvious influence of cap hit. That’s unsurprising given that Mahomes’ cap hit has stayed low and hasn’t changed very much in his first five years with the Chiefs.

Smith’s cap varied much more during his time in KC, and there is a decent apparent trend for best team performance to occur in seasons when his cap hit was the highest. That’s opposite to the effect predicted by our hypothesis. However, Smith’s cap hit in KC never reached really painful levels, so he might not provide the best test case.

Updated Approach

The Kansas City example illustrates a few key points. First, in order to separate out the effect of QB cap hit on team performance from confounding effects of differences in QB play, we need look at data from one quarterback at a time.

Second, to be able to see any impact of QB cap hit on team performance, we probably need to focus on QBs with at least a few really expensive contract years.

A third constraint added to the mix is that, in order to have reasonable confidence that any apparent trend is real, we need to have at least five data points per QB. To make that even more challenging, it doesn’t make any sense to include seasons in which a team’s primary starting QB lost significant time to injury. Therefore, I excluded all QB seasons when the primary starter lost more than five games to injury.

Those three constraints limited the pool to the following QBs and Teams:

• Drew Brees/New Orleans
• Aaron Rodgers/Green Bay
• Russell Wilson/Seattle
• Matt Ryan/Atlanta
• Ben Roethlisberger/Pittsburgh
• Cam Newton/Carolina
• Joe Flacco/Baltimore
• Eli Manning/Giants

Tom Brady never exceeded 12.5% of cap during this period and spent all but one year at or below 11%, so he is not a good test case. Hot tip: if you want to get to the championship round regularly, sign Tom Brady.

---

Results – Part 2

For the benefit of readers who skipped here from the introduction, in order to test the hypothesis that QB cap hit is a key determinant of team success, I evaluated the relationship between QB salary as % of Cap vs team success, measured as Total Wins in the regular season and playoffs for teams that made the Conference Championship round at least once from 2011 to 2021. To satisfy the requirements for this analysis the QB must have started at least 11 games for the team in at least 5 seasons, and his salary must have exceeded 13% of cap in at least one season.

The hypothesis predicts a downward sloping relationship, with Total Wins dropping as QB cap hit (% of Cap) increases. Let’s see if that was the case. QBs are listed in order of average QBR in the seasons used for analysis.

Drew Brees, New Orleans Saints, 2011-2020

Average QBR: 71.2

Our hypothesis gets off to a slow start with Brees. The relationship between QB cap hit and team performance is essentially flat. The fit coefficient, R2 = 0.0003, means that QB cap hit explains 0.03% of the variance in team performance.

The Saints won a wild card playoff game in 2013 and a Divisional playoff in 2018 when Brees’ cap hits were relatively expensive, at 14.0% and 13.4% of cap, respectively. They also won wild card games in 2011, 2017, and 2020 when Brees’ salary was more affordable at 12.4%, 11.6%, and 11.9% of cap. They finished out of the playoffs at 7-9 in 2012 when the QB’s salary dipped to 8.5% of cap.

There is no evidence here that team performance is related to QB cap hit.

Result: Fail - no relationship

Aaron Rodgers, Green Bay Packers, 2011-2012, 2014-2016, 2018-2021

Average QBR: 69.5

The results for Aaron Rodgers are essentially the same as Drew Brees. Once again, the relationship between Total Wins and QB cap hit is more or less flat. QB cap hit explains 0.62% of the variance in Total wins, which is a negligible amount.

The Packers made it to the NFC Championship and lost in 2019, when Rodgers’ salary peaked at 14.9% of Cap, and also in 2014 when his salary was approaching the expensive range at 12.4%. They also lost the NFC Championship in 2016 (11.8%) and 2020 (10.6%), when their star quarterback was more affordable.

Result: Fail – no relationship

Russell Wilson, Seattle Seahawks, 2021-2021

Average QBR: 66.8

Wilson is the first QB on the list to transition from his rookie contract to an extension during the period under study. It might not be an accident that he is the first of the QBs to show the predicted relationship between cap hit and team performance, since his cap hit spans a much greater range than the two previous QBs.

The trend line is still flatter than it is steep, meaning that QB cap hit seems to exert a moderate influence on team performance. But the relationship appears to be reliable, as QB cap hit explains 45.7% of the variance of Total Wins from season to season.

Seattle won the Super Bowl and an NFC Championship when Wilson’s salary was less than 1% of cap, as a result of being a phenomenal third-round steal. The best they have done in the playoffs since his salary hit 12% of cap is two wild card playoff wins.

Result: Pass

Matt Ryan, Atlanta Falcons, 2011 to 2021

Average QBR: 65.3

Doh! Just when you thought we were starting to see a pattern, Matt Ryan comes along and wrecks everything. Like Wilson, Ryan’s data spans the transition from his rookie contract which was extended in 2013. The Falcons did not enjoy the same cap savings as the Seahawks, however, because Ryan’s rookie contract predated the rookie wage scale and was renegotiated.

Nevertheless, the key point here is that the trend is for the Falcons to perform better in seasons where Ryan’s contract is more expensive, opposite to the prediction. The trend is not as strong as in Wilson’s case, as it only accounts for 27% of the variance of team performance. But it is still pointing in the wrong direction.

The Falcons’ best playoff performance was an NFC championship in 2015, when Ryan’s cap hit was a whopping 15%.

Result: Fail – trend is opposite to the prediction

Ben Roethlisberger, Pittsburgh Steelers, 2011-2018, 2020-2021

Average QBR: 62.4

Roethlisberger is another QB whose team seems to do ever so slightly better as his cap hit increases. The trend is pretty weak, only accounting for 9% of variance in team success. However, the data, once again, trends in the opposite direction to the prediction.

The Steelers did not have as much playoff success this decade as the previous one. Their best performance was a losing AFC Championship appearance in 2016, when Roethlisberger’s cap hit peaked at 15.3%.

Result: Fail – trend is opposite to the prediction

Cam Newton, Carolina Panthers, 2011-2018

Average QBR: 56.6

Another QB whose data spans the transition from rookie to extension contract. Technically, I shouldn’t have included him, because his most expensive cap hit was just short of 13%, but it’s close enough. This scatter is essentially flat, with salary cap only explaining 0.35% of the variance in team performance; team performance is unrelated to QB cap hit. Cam’s two most expensive years were just as bad as his two cheapest.

Carolina won the NFC championship in 2015, when Cam’s cap hit was 8.7%.

Result: Fail – no relationship

Joe Flacco, Baltimore Ravens 2011 to 2017

Average QBR: 54.5

Flacco’s data show a fairly weak downward trend, in agreement with the hypothesis. It is not very compelling, accounting for a modest 18.6% of the variance in team performance. The Ravens won a Superbowl and lost an AFC Championship game at the end of Flacco’s rookie contract when his cap hit was below 7%. After his extension contract value when above 11%, their best playoff performance was one wild card win.

Result: Pass

Eli Manning, New York Giants, 2011 to 2018

Average QBR: 54.1

Picking up the rear of this exercise is Eli Manning. Like many of the QB’s before him, the trend is essentially flat, indicating no relationship between QB cap hit and team performance.

The Giants won the Super Bowl in 2011, when Manning’s salary was a moderate 11.7% of cap, and didn’t win another playoff game in his remaining time with the team as his cap hit got very expensive and his play declined.

Result: Fail – no relationship

Summary of Results

Out of eight quarterbacks for which there was suitable data, six showed results that were inconsistent with the hypothesis. In four cases (Brees, Rodgers, Newton, Manning), there was no evidence of any relationship between cap hit and team performance. In two cases (Ryan, Roethlisberger), there were trends for the team to play better as the QB’s cap hit became larger, opposite to the prediction.

Results for only two of the eight quarterbacks were consistent with the hypothesis. There was a compelling trend for the Seahawks’ performance to decline as Russell Wilson’s cap hit increases, accounting for 46% of the variance in Total Wins. The Ravens showed a similar downward trend with Joe Flacco behind center, but it was less compelling.

---

Conclusions

On balance, the available data do not support the hypothesis that QB cap hit is a significant determinant of team performance. In the majority of cases where sufficient data was available, there was either no evidence of the predicted trend, or trends in the opposite direction. For each of the QBs that did show the predicted trend, there was another QB who showed the opposite trend.

We might have expected to see that the effect of QB cap hit on team performance would only show up for the poorer performing QBs. The reasoning here is that a weaker QB requires a stronger supporting cast, whereas a superstar QB provides such an advantage to his team that he makes up for the damage to the roster caused by his massive salary. That was not the case either.

One of the QBs who showed the predicted downward trend was Joe Flacco, the second lowest rated QB in this group. But the lowest rated QB in the group, Eli Manning, showed no impact of cap hit on team performance. The other QB whose data agreed with the hypothesis was Russel Wilson, the third highest rated QB in the group. There doesn’t appear to be any pattern here which might rescue the hypothesis.

That is how it stands for now. The highest QB salary relative to cap in this dataset was Russell Wilson’s 2021 salary ($32m), which accounted for 17.5% of the Seahawks’ cap. In the next few years, that might not seem like a particularly expensive contract for a starting quarterback.

In fact, by as early as next season, eight starting QB will have salaries exceeding 16% of cap on their current contracts:

• Matt Ryan 23.4%
• Aaron Rodgers 22.2%
• Kirk Cousins 21.5%
• Deshaun Watson 19%
• Ryan Tannehill 18.3%
• Patrick Mahomes 17.1%, 20.8% in 2023
• Russell Wilson 17%
• Dak Prescott 16.3%, 20.1% in 2023

It will be interesting to see how the quarterback market reacts when a full quarter of the league’s starters have salaries in a range we have not seen much of before. If this trend continues, it would not be surprising if the analysis presented in this article produces different results a few seasons from now.

Acknowledgement: Thanks to James Dorsett for editorial assistance

---