Publish date:

They're Really That Bad


You might say there has been a perverse bonus to watching NFL games this season. In addition to the usual compendium of graceful catches, slaloming runs and violent collisions, we have been treated to a festival of officiating gaffes. It began in the preseason, when a replacement ref cited Giants rookie Jayron Hosley for holding on a punt return; odd, given that Hosley was the player returning the punt. That same week another ref repeatedly referred to "Arizona" in his calls during a game between the Falcons and the Ravens.

Three Sundays into the regular season the situation has only gotten worse. We've seen crews lose track of timeouts, forget to reset the play clock after penalties, call punting teams for blocks in the back and trip a receiver in the end zone with a tossed cap. San Francisco linebacker Aldon Smith was called for unsportsmanlike conduct after Packers quarterback Aaron Rodgers inadvertently kicked the helmet off Smith's head during a sack attempt. The Eagles and the Ravens got not one, but two two-minute warnings in the fourth quarter of their Week 2 game. And on Sunday in Minnesota, 49ers coach Jim Harbaugh was awarded two replay challenges that he wasn't entitled to, while in Tennessee a mismarked penalty gave the Titans an extra, crucial 12 yards on their decisive field goal drive in OT.

On Sunday the NFL Players Association posted an open letter to team owners calling on them to end the lockout and charging that the "decision to lock out officials with more than 1,500 years of collective NFL experience has led to a deterioration of order, safety and integrity."

That evening, the Ravens-Patriots showdown in Baltimore produced a fitting capper to the increasingly sorry spectacle when, after the Ravens' bench was assessed a dubious unsportsmanlike conduct penalty (photo, far right), the crowd unleashed what NBC's Al Michaels termed "the loudest manure chant I've ever heard."

It's easy to pick on the scabs. Any casual observer can snicker at the comedy of errors. But are these guys really that bad? Are they that much less competent than the regular officials, a cohort that, until lately, hadn't exactly been known for infallibility among most fans?

In a word: yes. With the help of Cade Massey and Rufus Peabody—the former a visiting assistant professor of operations at Wharton, the latter a recent Yale graduate and Vegas-based sports analyst—SI considered the data from the first three weeks of NFL games. Upon review, it's impossible not to conclude that the replacements are inferior. It's not that this season's unofficial officials are calling more penalties. It's the types of penalties they're calling. It's when they are making—and not making—calls. It's which teams are, to use the term of art, getting hosed. Overall the data suggest that the replacement refs are, if not outright bad, certainly highly vulnerable to the types of behavioral biases and social pressures we expect officials to overcome.

Start with home field advantage. As is the case in virtually all sports, officials, overall, make calls that favor the home team. In the dozen NFL seasons from 2000 to '11, visiting teams were called for .49 more penalties per game on average. This season? The differential has jumped to 1.11. (And you wonder why home teams this year have covered the spread an unheard-of 62% of the time ....)

This is human nature. Most of us want to conform to a group. When, say, a horde of fans suggest how we ought to make a split-second decision, we tend to follow their exhortations—even if we don't believe that the group's position is the correct one. Also human nature: We seek to alleviate pressure. If one decision will make 75,000 rabid people cheer, and another will make those same 75,000 scream horrible things about our relatives, the choice is easy. Problem is, we ask sports officials to insulate themselves from social pressure and simply get it right.

While the actual rate of penalties being called is remarkably consistent with the past—14.7 per game last season versus 14.9 through the first three weeks this season—some calls are being made, and others neglected, at an alarming rate. Last season, there were .14 illegal shift penalties called per game—in other words, a few each Sunday. So far in 47 games this season, there has been one. Maybe this is dumb luck or the admittedly puny sample size. Another explanation: An illegal shift is an esoteric call, and an official whose previous job was working in the Lingerie Football League might not be wholly attuned to this infraction.

On the other hand personal fouls are being called at the highest rate since 2000. Maybe this is the "substitute-teacher effect," i.e., the players are testing the new authority figures. But the replacement refs may also fear losing control and having their authority undermined.

Also, there has been a huge uptick in defensive pass interference calls (from an average of .87 per game last year to 1.36 this year). Is it possible that players in the secondary are simply violating the rules more often this season? Yes, but more likely the rookie officials are susceptible to players' pleas, and when receivers complain immediately after a play, the officials are swayed.

Either way, the compromised officiating has had a real, material impact on the game. According to Massey/Peabody modeling, due solely to the increase in defensive pass interference calls, scoring should increase 0.89 points per game, and overall, scoring should increase by 1.08 points due to the officiating. Not for nothing did Las Vegas oddsmakers set the highest-ever average over/under totals going into Week 3.

In the Republic of Football we've made like the refs and called 'em as we've seen 'em, declaring these officials worse—much worse—than the guys they're filling in for. And upon further review, we were right. Now it's time for the NFL to face the facts, make the necessary concessions and replace the replacements with the real thing.

Follow @jon_wertheim

Freak Flags

While the number of penalties in 2012 is about the same as in previous seasons through Week 3, the types of calls have been very different

[The following text appears within 4 charts. Please see hardcopy or PDF for actual charts.]


15 14.5 14 13.5 13

2009 2010 2011 2012


70 60 50 40 30 20

2009 2010 2011 2012


40 30 20 10

2009 2010 2011 2012


10 9 8 7 6 5 4 3 2 1 0

2009 2010 2011 2012


Illustration by DARROW