Rating Washington’s drafts in the post-Shanahan era

Every Washington fan seems to have an opinion about how well, or badly, the team has drafted, and which GMs and scouting directors have done a good or bad job. There was strong support for Kyle Smith among certain sectors of the fanbase, while the short-lived tenure of GM Scott McCloughan was more polarizing. Not many will admit to liking Bruce Allen, for good reason, but were his drafts any worse than the other guys?

I have been trying to figure out how to put objective ratings on the team’s draft performance for a while, but could never quite figure it out. KyleSmithForGM’s recent piece on Washington’s drafts rekindled my interest in the topic. He used a ranking approach to compare players. That approach has a lot of merit, but it also has some limitations. In particular, it suffers from a ceiling effect that limits the ability to fairly evaluate players picked near the top of the draft order.

After reflecting on that approach, and some back and forth by email with the man himself, I have come up with a different approach which overcomes some of the limitations of ranking, but has some of its own. My approach uses a statistical metric called a Z-score to compare each draft pick to the next group of players who were likely to be on the board to rate Washington’s draft decision. I don’t have access to Washington’s draft boards, so my best guess at whom those players were likely to have been is the next set of players drafted by other teams.

Photo by Jim McIsaac/Getty Images

Warning: the next four paragraphs contain formulas and may make some readers’ heads hurt. If you don’t care about the statistical ins and outs, simply skip ahead to “Statistical mumbo jumbo ends here.”

The term Z-score sounds like it’s going to be complicated, but the concept is really pretty simple. The Z-scores I calculated compare the player’s career value to date to the average value of the next n players drafted. To put a value on each player, I used the weighted Approximate Value (wAV) metric developed by Pro Football Reference. Because approximate value and its derivatives, including wAV, are cumulative measure, they are not useful for comparing players from different recent draft years. Therefore, I have limited my analysis to comparing players within a single draft.

The raw difference between a draft pick’s wAV value and the average wAV of the next 10 players doesn’t actually mean that much on its own, because there is so much variability from player to player in the draft. For example, Washington’s fourth pick in the 2016 draft, Matt Ioannidis, had a wAV value of 20, which is 11.3 pts higher than the average wAV value of the next 32 players picked after him. That sounds like an amazing draft decision, and a lot of us thought it was.

However, the standard deviation (a common measure of variability) of the next 32 players’ wAV values was 13.4. That means there was so much variability in the career values of the next players who were likely to have been on Washington’s board that there was a reasonable probability they would have picked a player as good as Ioannidis simply by chance.

To get an idea of whether Washington’s draft pick was really any better or worse than would have been expected based simply on the quality and variability of players remaining on the board, we need to adjust for the player to player variability of wAV values. I did this by dividing the raw difference between Washington’s draftee’s wAV and the average wAV of the next group of players by the standard deviation of those players’ wAV values to give a Z-score, as follows:

d = wAV_Washington – mean(wAV_{1 to n})

Z = d/SD(wAV_{1 to n})

In summary, the Z-score measures the difference in career value between Washington’s draft pick and the average of the next n players drafted in units of standard deviation of the next n players’ wAV scores.

Statistical mumbo jumbo ends here

The Z-score indicates how much Washington’s pick stands out from the next group of players that were available on the league’s draft board.

Z-score make a great rating scale, because the values are meaningful. A player whose career value is one standard deviation above the average of his peers (Z = 1) was a very good pick. A player whose career value was two standard deviations above average (Z = 2) is approaching outstanding territory. Conversely, a player whose career value was one standard deviation below average (Z = -1) was a bad choice; and players two standard deviations below average (Z = -2) are major duds.

For this article I decided to rate Washington’s drafts back to 2014, when the current roster first began to take shape with the addition of the longest serving Commander, Tress Way, albeit as an undrafted free agent. Drafts before then are no longer relevant to the current roster, except for a few key players who got away. I rated drafts up to 2020, but could not rate 2021, because my player value metric, wAV, requires a minimum of two accumulated league years to become meaningful.

For readers who skipped the statistical explanation, all you need to know is that I used a statistical metric called a Z-score to derive two ratings for each of Washington’s draft picks as follows:

Draft Decision Rating – As described in detail above, each of the Redskins’/WFT’s draft picks were rated against the next group of players available at their pick whom they might have considered drafting instead. (To close the loop for readers who read the statistical explanation, the Draft Decision Rating is the Z-score of the player’s wAV relative to the wAV values of the next n drafted players. I will explain n in a minute.) This rating indicates how well Washington did at selecting the best player available at their pick from the most likely group of candidates.

Player Rating – I can anticipate that a lot of fans will get upset when I give a C grade to the selection of a good player like Chase Young, because he does not really stand out that much compared to other players selected near the top of the first round. To appease that section of the fanbase, I also gave each draft pick a Player Rating, which indicates how good he is compared to the entire draft class. For statistically minded readers, the Player Rating is the Z-score for the player’s wAV value relative to all of the other players in his draft class. This rating indicates how well Washington did at finding a good player, regardless of where he was picked.

n, a.k.a. Sample Size

Statistical content warning: if you don’t think you need to know, you can probably skip this section, too.

The Z-score calculations I used to derive my player ratings require a minimum sample size in order to calculate a meaningful measure of sample variability. The standard deviation of a sample of wAV values from below about 10 players is a pretty hairy proposition. Things settle down when sample sizes reach around 30.

This poses a problem near the top of the draft order, where there is often a big difference between the first few players drafted and the one drafted 10^th overall. As the draft progresses, it becomes harder and harder to reliably tell the difference between players drafted dozens of picks apart. By around the fourth round, teams hit on draft picks at about the rate expected by chance, based on availability of quality players remaining on the board.

To deal with this challenge, I increased the value of n as the draft progresses, trying my best to keep well within the limits of resolution I have previously calculated. This method might fall down a little for picks in the top five or six, but probably not enough to change any conclusions about Washington’s draft picks.

The values of n were as follows:

Washington Pick 1^st to 10^th overall: n = 10

Washington Pick 11^th to 64^th: n = 16

Washington Pick in 3^rd round: n = 24

Washington Pick in 4^th to 7^th rounds: n = 32

When Washington’s pick was within 32 picks from the end of the draft, the Z-score was calculated relative to the last 32 picks in the draft, excluding Washington’s pick.

Letter Grades

For those readers who don’t care what my numerical ratings mean, I have also translated them into more familiar letter grades. My grading scale might be a little tougher than what you are used to, though. Here is how to interpret the grades and relate them to the ratings:

Please note, that, in this rating scale a draft pick which earns a C Decision Grade meets expectations for his draft position. When that player is picked in the first round, he is a very good player, since that is the expectation for first-round picks. When he is picked in the seventh round, he will struggle to make the roster, since very few seventh round picks ever do.

The Player Ratings and grades indicate how good or bad the player was within his draft class overall. Earning a B Player Rating in the first round requires much less drafting skill than earning one in the sixth round.

Rating Washington’s Drafts, 2014 to 2020

Washington’s drafts have been led by four different combinations of GMs (or de facto GM) and scouting directors since 2013. The draft ratings are grouped accordingly. For each draft pick, I will present the player’s wAV value, as well as the mean (mean wAV) and standard deviation (SD wAV) of the players he is being compared to, along with the ratings and grades.

2014 President/GM Bruce Allen, Scouting Director Scott Campbell

The first draft of the post-Shanahan era was pretty good, resulting in selection of three long-term starters and one Super Bowl champion, despite the lack of a first round pick. Three out of Washington’s eight draft decisions were rated as above average or better: Trent Murphy, Morgan Moses and Bashaud Breeland.

There were equal numbers of better than average and below average draft decisions (three apiece). However, the better draft decisions occurred on Day 2, netting three starters, while the below average-decisions occurred in the sixth and seventh rounds, where they did little damage.

Morgan Moses’ Draft Decision Rating and accompanying statistics are highlighted because his rating benefits from an artificial boost as a result of being selected ahead of a run of 24 unexpectedly poor players for their draft positions. This is an example of streakiness that often occurs in the draft. The giveaway when this happens is that the mean wAV values for the players that he is being compared to is lower than expected in comparison to the mean wAV values after picks #47 (Murphy) and #78 (Spencer Long). While the Moses pick scores an A grade in this analysis, he would probably be a B in most other draft years, which is still very good.

2015 to 2016, GM Scott McCloughan, Scouting Director Scott Campbell

Scott McCloughan’s first draft in Washington was very good, with the selection of four long-term starters in 10 picks, including All-Pro Brandon Scherff and Super Bowl champion Austin Reiter. The fact that Reiter earned his Super Bowl start for another team is not relevant to grading the draft decision. This draft might have been even more special if safety Kyshoen Jarrett had not suffered a career-ending injury.

There were four better-than-average draft decisions and only one below-average decision. Once again, three of the best picks were in the top 105 selections, resulting a good haul of quality players.

McCloughan and company came crashing back to earth with a pedestrian effort in 2016. The better than average selections of Kendall Fuller and Matt Ioannidis in the 3^rd and 5^th rounds were more than offset by whiffing on Josh Doctson in the 1^st and Su’a Cravens in the 2^nd. Fuller was the only long-term starter who stuck from this draft. Ioannidis had three quality years as a rotational piece before missing 2020 to injury and will finally get a chance to start in Carolina. His grade might go up if Carolina uses him in the right scheme.

The Doctson pick was actually worse than the Draft Decision rating makes it seem. Like Morgan Moses in 2014, Doctson was followed by a relatively weak run of 16 players in the late-first and early-second round, including fellow busts Laquon Treadwell, Paxton Lynch, and Robert Nkemdiche. This pick would have earned a D draft decision grade in most other years.

Overall, this draft featured five below-average draft decisions to only two above-average decisions.

2017 President/GM Bruce Allen, Scouting Director Scott Campbell

After sending McCloughan packing, Bruce Allen’s team managed to draft two long-term, quality starters in 2017. The 6^th round selection of Chase Roullier ranks among Washington’s best draft decisions in the post-Shanahan era. The Draft Decision Rating gets an added boost due to the fact that the next 32 players had a lower average wAV value than expected at this position in the draft. Nevertheless, Roullier’s selection this late in the draft would earn an A grade in any draft year.

Aside from Jonathan Allen and Roullier, the rest of this draft was pretty ordinary. There were three below-average draft decisions, including the terrible selection of Ryan Anderson in the 2^nd round.

2018 to 2019 President Bruce Allen, Scouting Director Kyle Smith

Kyle Smith’s first draft in Washington was not good. The team managed to hit on a good starting DT with the 13^th overall pick. They also found special teams mainstay Troy Apke in the 4^th round and rotational DL Tim Settle in the 5^th round. Two above-average draft decisions (Payne, Shaun Dion Hamilton) were matched by two below-average decisions (Derrius Guice, Geron Christian). Four of the eight players selected are no longer playing in the NFL.

Kyle Smith’s second draft as head of scouting was one of Washington’s best in recent memory. The draft started with Washington’s worst draft pick since RG3, when owner Dan Snyder selected Dwayne Haskins with the 15^th overall pick. It has been reported that Smith’s team had a 3^rd round grade on Haskins.

After that, the team went on to select three quality starters, Montez Sweat, Terry McLaurin and Cole Holcomb. The McLaurin and Holcomb picks receive A+ grades and rank amongst Washington’s best draft decisions in the last decade.

The one major dud decision made by Smith’s team was selection of injured RB Bryce Love in the 4^th round. This pick would be rated even worse than it is, if not for the fact that the following 32 players had lower average wAV values than expected in the fourth round.

2020 Head Coach/de facto GM Ron Rivera, Scouting Director Kyle Smith

Ron Rivera’s first draft in Washington appears to have been good to very good two years out. The team drafted two quality starters in Antonio Gibson and Kam Curl. Curl’s selection in the 7^th round is Washington’s highest rated draft decision in the post-Shanahan era.

Second overall pick Chase Young is a big question mark heading into his third season. He earns an A player grade based largely on the strength of his rookie season, in which he earned AP Defensive Rookie of the Year. However, he did very little in nine games before tearing his ACL midway through his second season.

It is still too early to reach final judgement on many of the players in the 2020 draft class. Saahdiq Charles, Antonio Gandy Golden, Keith Ismael, Khaleke Hudson and James Smith-Williams could all still establish themselves as regular contributors and there is a chance that at least one of them could earn a starting role.

Ranking Washington’s Drafts

Now that we have been through each individual draft, the next question is how do they all compare? The following table shows four key metrics for each draft: number of long-term starters drafted (regardless of team they started for), draft picks with A and B Player Grades, draft picks with Decision Grades of C+ or greater, and bad decisions (D Decision Grade).

The clear winner in all four categories is Scott McCloughan’s 2015 draft. His 2016 draft, by comparison is the worst draft since 2014. No wonder he was a polarizing figure.

Second place goes to Ron Rivera and Kyle Smith’s 2020 draft, with the caveat that it is still a little too early to judge many of the players, including second overall pick Chase Young.

Just behind 2020 is Bruce Allen and Kyle Smith’s 2019 draft. Had the owner not interfered with the 15^th overall pick, this one might have rivaled 2015.

Bruce Allen and Scott Campbell’s 2014 draft follows with three long-term, average starters.

Allen and Campbell’s 2017 draft and Allen and Smith’s 2018 draft were fairly weak, but still better than 2016.

Photo by Tony Quinn/Icon Sportswire via Getty Images

Washington’s Best Draft Picks

Washington’s best draft picks in the period, ranked by Draft Decision Rating were:

2020 Kam Curl, 7^th round, rating 7.42, Rivera & Smith

2017 Chase Roullier, 6^th round, rating 5.47, Allen & Campbell

2019 Cole Holcomb, 5^th round, rating 3.99, Allen & Smith

2019 Terry McLaurin, 3^rd round, rating 3.55, Allen & Smith

2014 Morgan Moses, 3^rd round, rating 2.67, Allen & Campbell

2015 Austin Reiter, 7^th round, rating 2.08, McLoughan & Campbell

Conclusions

Overall, it appears that Kyle Smith’s reputation as a good drafter was well deserved. In three seasons as head of scouting, he put together two of the best drafts in the last seven years and was behind three of the team’s best draft picks in the period.

Bruce Allen was involved in one of the better drafts in the period, as well as some average and poor drafts.

Draft guru Scott McLoughan managed to produce the best and worst drafts of the post-Shanahan era in successive years.

Help Me Write My Next Article

Now that I have worked out the analytical method, I can compare Washington’s drafts with other NFL teams. However, this method is pretty labor intensive and I am too lazy to calculate draft ratings for every team in the league. I would like to call on Hogs Haven to tell me which teams you would most like to see compared to Washington. In the comments, please let me know which other teams you think have been the best, worst and most mediocre at drafting in the last five years.

---