Blessed Are the Geeks ...
It began with a question, one that countless thinkers had asked before, but that none had answered—at least not in a way that satisfied him. Brian Burke never intended to shatter decades of conventional football wisdom or to ignite an old-school versus new-school war. He was just a fan who wanted the answer to a simple question: When should a coach go for it on fourth down?
There was a time, not so long ago, when teams were smarter than everyone else. A time when Football Men had all the answers—theirs was a game with as vast a knowledge gap between the insiders and the outsiders as any other sport. But this was before the Moneyball revolution changed baseball and began to seep into other sports; before the rise of fans who began to rethink the conventional wisdom; before those fans began tracking every play with such high levels of precision that teams began asking them for data. This was before Burke sat down in front of his laptop four years ago, in a hotel room in Karachi, Pakistan, and attacked the fourth-down conundrum to create what would become a New Age blueprint for winning games.
On the surface Burke couldn't have been a more unlikely creator of one of the biggest innovations in football statistics. A Navy pilot turned weapons and tactics expert for a military contractor, he'd only recently heard of the godfather of sports analytics, Bill James. But he was an obsessive football fan who knew the power of numbers; in combat in Iraq between the Gulf Wars he'd put his life in the hands of analytical techniques and probabilistic calculations and come out alive. Holed up in a hotel during a three-week work trip to Pakistan in September 2008 ("We could never leave the hotel [out of danger]," he says; "I had a lot of free time"), Burke hit on the idea of building a statistical model that would yield the odds of a team's winning a game in every on-field situation—every down-and-distance from every position on the field, for every point margin. Win Probability would tell a team what it should do based on the numbers, a data set that has since grown to more than 3,000 actual games.
Of course, there remain more skeptics than believers. "I think it's fascinating," says one NFL front-office staffer about Burke's formula, "but it's not tested and proven out over as many situations as you'd like."
Still, to many statheads, Burke's work was evidence of what they had been arguing for years—that the league was stuck in what Burke, 42, calls a "suboptimum equilibrium, with no teams playing the optimum way." Teams didn't pass the ball enough and ran the ball too much, and they were far too passive on fourth downs, settling too often for field goals and punts. "The current coaching crop learned from the trial-and-error conventional wisdom of the 1970s, which was the nadir of passing," says Burke. "The sport has changed dramatically."
In recent years NFL teams started to get smarter. M.B.A. graduates from elite universities began infiltrating football front offices as they had baseball's in the mid-2000s, but the brain power was devoted to salary-cap management and personnel decisions. There was one area in which teams remained stuck in flat-earth thinking: game strategy. Burke believed this would change—that in an information age in which advanced stats had the power to predict a presidential election to the decimal point, new math would be impossible for the Football Men to ignore. He believed the revolution would reach the field, that the game was "reaching a tipping point where one coach would buy into the analytics approach. And if that coach were successful, there would be an avalanche."
Go back, before Brian Burke and his Win Probability model, before bright young front-office minds began managing salary caps like hedge-fund portfolios, before Bill Belichick's Patriots—the closest thing to an NFL Moneyball team—recognized that franchises were overvaluing free agency and undervaluing draft picks.... Analytics have been a part of football since the formative years of the NFL. Back in the early 1960s, the Cowboys' Tex Schramm employed an Indian-born programmer named Salam Qureishi—a cricket fan who knew nothing about football—to develop a computerized analysis of the qualities that make a player draft-worthy. Even then Schramm, who ran the most sophisticated scouting system in the NFL, spoke about using computers in analysis and game planning. He liked to tell the story of how he ran the high school numbers of running back Dan Reeves through his computer as an exercise. Reeves went undrafted in 1965 out of South Carolina before Schramm's Cowboys signed him as a free agent. He went on to play in two Super Bowls for Dallas. The model had Reeves as the 29th-best player in that draft.
"NFL teams have always been ahead of other sports in terms of analytics," says Aaron Schatz, founder of the website Football Outsiders. "But then the movement stopped somewhere along the way." Where football began to lag behind other sports was in its use of advanced statistics. Schatz wanted to change that.
He was a 10-year-old math prodigy growing up in Massachusetts when he experienced enlightenment reading Bill James's Baseball Abstracts. Later, as he worked as a trends analyst for the website Lycos, he became a football fan and wondered why Jamesian principles had never been translated to this sport. Schatz's epiphany came when Boston Globe writer Ron Borges wrote in 2002 that the Patriots would not make the playoffs because they lacked the ability to establish the run. "He argued this even as he picked Oakland to win the Super Bowl, and that didn't make any sense—Oakland ran the ball less than any other team," says Schatz. "I thought, well, when Bill James wanted to know if a certain catcher gave up more stolen bases than others, he opened up the dang Sporting News and started counting. So I was going to do the same thing—go through the play-by-play to see whether teams that run more really win more."
In July 2003 Schatz launched Football Outsiders, which was groundbreaking in its use of empirical evidence and advanced statistics in debunking myths such as the importance of establishing the run. Schatz wrote for "people who followed the highest level of the chess game but were severely underserved in the market place," he says. "I knew the audience was there.
"In 10 years someone who grew up reading Football Outsiders will be running a team," predicts Schatz, 38, who says he turned down a full-time NFL job to continue writing.
Now nearly every team in the NFL has an analytics group, though most are as forthcoming as the CIA regarding the work these employees perform. "There are more teams active in this than most people think," says an analyst for one NFL team. "Some of these guys are Ph.D.'s who clearly know the science—but you need to know football too," he adds. "The vast majority of academic football analysis is, truthfully, out to lunch. One team's analytics guy is studying biorhythms to find out whether players should wake up at five in the morning instead of eight in order to be more alert for a Monday-night game. Some of it is pretty out there."
Among those teams riding the stats wave are the Ravens, who announced in August that they'd hired a former NBA statistical consultant with degrees from Yale and Carnegie Mellon to lead their new analytics department. They're just now catching up to opponents like the Patriots, and the 49ers, who in 2001 lured Stanford M.B.A. Paraag Marathe from his consulting job at Bain & Co. to lead their analytics department.
While acceptance of analytics has grown in the NFL in the 10 years since the start of Football Outsiders—sites like Pro Football Focus and Burke's Advanced NFL Statistics followed in its wake—Schatz believes that when it comes to teams themselves, everything and nothing has changed. "Analytics have had a great impact on personnel decisions," he says. "But watching the way teams play on the field is how you know whether coaches are listening. And despite the work that guys like Brian Burke have done, coaches have become less aggressive on fourth down in recent years, which is mind-blowing."
"For a change to happen," says Schatz, "it's going to take an owner publicly saying, 'This is what I want the coach to do. He is not in danger of losing his job. I understand the process is more important than the outcome.'"
In other words, it will take a team that can afford to take a chance. Perhaps a team like the Jacksonville Jaguars.
I'd be lying if I told you this was totally unexpected," says Tony Khan, on a mid-November afternoon, referring to his reeling Jaguars. The team has lost six straight. Second-year quarterback Blaine Gabbert is having a disastrous season; star running back Maurice Jones-Drew has been hobbled by injuries all year; coach Mike Mularkey and G.M. Gene Smith are under fire. "It's not like we projected to be a 12-win team," says Khan. "Part of what I'm doing is taking an objective look at things through numbers."
This summer Khan—an energetic 30-year-old with a finance degree from Illinois, and the son of Jaguars owner Shahid Khan—left his job at an alternative-energy company to run Jacksonville's new analytics department. One of his first hires was a friend of Schatz's who had started an analytics club at Harvard and who left an M.B.A.-J.D. program at the university to work with the Jaguars, a struggling team on a quest for any edge. "There's so much information out there, it'd be foolish not to use these resources," says Khan, referring to the growing number of websites that do their own play-by-play charting—sites such as Pro Football Focus, the brainchild of Englishman Neil Hornsby, who now has 26 employees analyzing video of every game and supplying data to seven teams.
Throughout professional sports, forward-thinking teams are engaged in an arms race for new technology that can lend them any advantage. The most progressive organizations have quietly begun incorporating video technology into their analysis: Fieldf/x, which tracks players' movements on the baseball field, has been a secret weapon for the world champion San Francisco Giants; and 10 NBA teams use SportVU, which can identify opposing teams' plays based on movements. NFL teams are starting to use similar technology: This year the Falcons, Giants and Jaguars began using GPS tracking technology on their players during practices. (The league prohibits it during games.)
"The things you can learn—route running for receivers, closing speed for defensive backs and linebackers—are huge," says Khan. "The biggest implementation, though, is for rehab and injury prevention: making sure you're putting the optimum stress load on a player." On this the Jaguars are consulting with one top 20 college program that used such technology and saw soft-tissue injuries among its players plummet from one year to the next.
The future of analytics is in the merging of statistics and video. "Look at baseball, with its defensive shifts—outfielders looking at cards on the field much like a quarterback would," says Khan. "It's possible that someday defensive backs will be playing with similar cards based on where receivers are lined up and what those receivers' route-running strengths are. The possibilities are endless."
On this particular day, however, Khan's team is focused on a more immediate issue: optimal overtime strategy under the NFL's new rules, which dictate that if one team kicks a field goal on the first possession of extra time, the opponent gets a rebuttal drive.
This is an area, Khan says, "where being strong analytically can give you an advantage." This season there have been 19 overtime games through Sunday. The Jaguars have played in three and lost each one. Overtime poses a number of questions: Should a team play methodically, as in the course of the game, or run the faster two-minute offense? Should a coach settle for a field goal on fourth-and-short on the first drive? "There are some people, for instance, who think you're better off deferring"—that is, giving the opponent the first possession—to start overtime, says Khan. "But I haven't seen a rock-solid argument yet."
The challenges of overtime shed light on the overall difficulties of statistical analysis in the sport. "The difference between football and baseball is that pretty much every possible situation in baseball has probably happened by now, maybe thousands of times over the years," says Khan. But in football, "there are down-and-distance situations with certain score differentials that haven't happened yet, so we're still trying to piece the data together."
There is a bigger challenge: persuading the coach—and in many cases the G.M.—to buy into the numbers. "I imagine pretty much every analytics guy in football has offered a suggestion based on data, and the coach has probably gone a different way because of momentum or because he just doesn't trust the spreadsheet," says Khan. "The best thing that we can do is to simplify the information and put it on a laminated card. But there's not a lot of time to make these decisions, so the decision maker is usually going with his gut instinct. And that instinct is not necessarily what the spreadsheet says." (On his relationship with Mularkey, Khan says, "Mike thought some of the things we were showing him were a little out there, but as we've gotten further into the season, he's trusting us more and more.")
As the Jaguars' season has progressed, they have begun to resemble a team willing to challenge conventional wisdom, with surprise onside kicks and aggressive fourth-down attempts—some more successful than others. On a play that would have made even Burke cringe, Mularkey chose to go for it on fourth-and-10 on the opposing team's 47 late in overtime against the Texans last month (this after converting earlier on fourth-and-10). Backup Chad Henne threw an incompletion, and the Texans scored two plays later to win.
The Jaguars know it's going to take more than aggressive fourth-down play-calling to turn around a franchise that hasn't had a winning season since 2007, but they also recognize that being ahead of the curve on analytics can accelerate such a turnaround. "There's one team right now that is applying all of this better than everyone else, and that is San Francisco," says Khan. He points to one 49ers play in Week 10, against the Rams: San Francisco faced fourth-and-one at St. Louis's 21-yard line, down 10 in the fourth quarter. The Niners went for it, converted and scored a touchdown two plays later. The game ended in a 24--24 tie. "Most people disagreed with that," says Khan. "But I've seen the chart they use. And certainly guys like Brian Burke agreed with what they did."
Now's the time to strike," says Burke, "to reap the rewards of being an early adopter; to be the Oakland A's until the Yankees start using your stuff and the competitive edge is gone."
Burke and his brethren are still waiting for football's Billy Beane—a G.M. or coach who will embrace the ideas that football's outsiders have been pushing as baseball's sabermetricians did in the 1990s. Someone to spark the revolution. Perhaps someone like Oregon's Chip Kelly, a hot candidate to land a top NFL head coaching job this winter, and an innovator whose aggressive play-calling has been regarded as outside-the-box but is based on mathematics, not momentum.
"The problem is that the sport has changed since the days when a common score was 6--3 and punting was a death blow—if you put a guy deep in his own territory he might not see the other side of the field again," says Burke. "That's not true anymore. That conventional wisdom, passed down from generation to generation, just doesn't work anymore. But everyone is playing this suboptimal way, so there's just no incentive to change."
As he speaks, Burke is sitting in front of his computer in the basement of his home in Reston, Va., where he conducts his analysis. Four years ago he was a single father with nights to himself. He began playing around with football statistics after he and a friend debated whether or not defense wins championships, and a few months later he began posting his work online. He attracted attention after he was one of the few, in November 2009, to defend Patriots coach Bill Belichick's almost universally panned decision to go for it on a fourth-and-two from his own 28-yard line with 2:08 remaining in a game in which New England led Indianapolis 34--28. (The Pats failed to convert, and Peyton Manning drove the Colts in for the winning score.) Around that time Burke was contacted by a representative of an NFL team who told him that his club had been using Burke's models (which are available to the public) to determine when to use timeouts and when to accept and decline penalties. The team was interested in creating models of its own. Today Burke works full time managing for a military contractor, contributes a weekly column to The New York Times and The Washington Post, and consults for a handful of teams. He has no interest in working for any club full time.
Burke opens a chart he's working on, an analysis of overtime strategy, and is puzzled by what his own data shows. "Opening drive in overtime, fourth-and-eight on the other team's 30-yard ine—you punt?" he says questioningly, pointing to the data. "I talked this over with a team, and they got the same thing. Strange, huh?"
When he began meeting with NFL coaches a few years ago, Burke realized "that coaches have thought of things that never have dawned on me," he says. "I'm always humbled after talking to them.
"I'll be honest," he says. "I'll be watching a game, and I'll wish that a team won't do what I had recommended because I'm so frightened they won't convert. It scares me to death when I see teams take my advice. I can understand, on the sidelines, with all that pressure, any one of us would like to go into the safe harbor of the tried and true. I understand that.
"But eventually," he adds, "I believe change will come. The game evolves."
It's All Falling Into Plays
A look at the NFL's shifting run-pass balance over the past 30 years suggests that Burke's ideas may be trickling down—just very slowly
[The following text appears within a chart. Please see hardcopy or PDF for actual chart.]
RUSHING
PASSING
57%
50%
45%
40%
1983
1992
2002
2012
Not-So-Lame Duck
Oregon's Chip Kelly, one of college football's hottest coaching prospects, is expected to be courted by needy NFL franchises this off-season. If he takes a pro job, here's the kind of forward thinking he'd bring to the NFL
114
Fourth-down attempts by Oregon since Kelly took over in 2009, third most in the nation behind traditional option teams Air Force and Georgia Tech
18
Two-point conversions, at least twice as many as all other FBS schools except Boise State
288
Offensive TDs, most in the nation and 29 more than second-place Boise State
Standing Pat
When Bill Belichick's failed fourth-and-two conversion cost the Patriots against the Colts in 2009, pundits picked apart the play. Brian Burke fired back with a defense—but coaches don't appear swayed by the math. After going for it on fourth down 13.99% of the time that year (an 18-year high) NFL teams have gotten more risk averse. The league rate has dropped each year since, to 10.69% this season through Sunday
(0.60 √ó 1) + (0.40 (1 - 0.53)) = 0.79 WP
Likelihood the Patriots would convert fourth-and-two, based on Burke's model
Likelihood the Patriots would fail to convert, based on Burke's model
Likelihood the Colts would score a TD after the Patriots failed to convert
Patriots' Win Probability if they went for it, .10 better than if they punted
SI.COM
To read Kerry J. Byrne's Inside the Numbers column, a weekly take on NFL analytics, visit SI.com/mag
PHOTO
Photograph by BILL FRAKES
THE MATH OF KHAN The Jaguars' new analytics guru, Tony Khan, is tasked with finding a mathematical solution to 2--10.
CHART
CHART BY ANTHONY MANNINO
PHOTO
JOHN BIEVER (MATTHEW STAFFORD)
PHOTO
ANDY KING/GETTY IMAGES (ADRIAN PETERSON)
PHOTO
JED JACOBSOHN
PHOTO
JAMIE SQUIRE/GETTY IMAGES