In light of my recent post on the difficulty of changing our decision-making habits – even when we’re aware that our habits are biased and flawed – I thought it might be interesting to look at two examples from professional football. Why sports? Given the intense competitive pressure in the NFL – there’s a thin line between victory and ignominy – you’d expect head coaches to have corrected many of their decision-making mistakes, especially once those mistakes have been empirically demonstrated. But you’d be wrong.
Consider some research done by David Romer, an economist at UC Berkeley, who published a 2001 paper entitled “Do Firms Maximize? Evidence From Professional Football”. The question Romer was trying to answer is familiar to every NFL fan: what to do on 4th down? Is it better to bring on the kicking team for a punt or field-goal attempt? Under what conditions should coaches risk going for it?
To answer this immortal mystery, Romer analyzed every fourth down during the first quarter in every NFL game between 1998 and 2000. (He had help from a computer program.) The first thing Romer did was figure out the fluctuating value of a first down at each point on the football field. After all, a first down was more valuable for a team if it occurred on an opponents two yard line than on their own twenty yard line.
Then Romer calculated the statistical likelihood of going for it on fourth down under various circumstances and actually getting a first down. He also calculated the probability of kicking a successful field goal from various spots on the field. So let’s say you are NFL coach, and you have a fourth and three on your opponent’s 30 yard line. Romer could tell you that 1) you have a 60 percent chance of getting a first down, and that teams with 1st downs inside the thirty yard line score a touchdown 40 percent of the time, for an expected point value of 1.7 and 2) that field goal attempts from the 32 yard line failed almost 65 percent of the time, which meant that going for a field goal only had an expected point value of 1.05. In other words, it’s almost twice as effective to go for it than to attempt a field goal.
So what do most coaches do? Well, NFL coaches consistently make the wrong decision. According to Romer’s analysis, teams would have been better off going for it on fourth down during the 1st quarter on 1100 different drives. Instead, coaches decided to kick the ball 992 times. This meant that NFL coaches made the wrong decision over 90 percent of the time. Romer summarized his counterintuitive results: “This analysis implies that teams should be quite aggressive. A team facing fourth and goal is better off on average trying for a touchdown as long as it is within 5 yards of the endzone. At midfield, being within 5 yards of a first down makes going for it on average desirable. Even on its own 10 yard line – 90 yards from a score – a team within three yards of a first down is better off on average going for it.” Romer conservatively estimates that a more aggressive approach on fourth downs would make a team 5 percent more likely to win the game. This is a significant advantage: a coach willing to endure the risks would win one more game in three seasons out of every four.
But if kicking a field goal or punting on fourth down is such a bad idea, then why do coaches always do it? To explain the consistently bad decisions of NFL coaches, Romer offered two different answers. The first is risk aversion. If coaches followed Romer’s strategy, they would fail about half the time they were within ten yards of the endzone. This means that instead of kicking an easy field goal and settling for three points, they would come away empty handed. Although that’s a winning strategy in the long-run, it’s awfully hard to stomach. (As Daniel Kahneman notes, “Worst case scenarios overwhelm our probabilistic assessment, as the mere prospect of the worst case has so much more emotional oomph behind it.”) After a long drive down the field, fans expect some points. A coach that routinely disappointed the crowd would quickly get fired.
The second reason coaches stink at making decisions on fourth down is that they stink at statistics. As Romer politely writes, “Many skills are more important to running a successful football team than a command of mathematical and statistical tools…It may be that individuals involved want to make the decisions to maximize their teams’ chance of winning, but that they rely on experience and intuition rather than formal analysis.”
So how have coaches reacted to this data? In 2001, before Romer published his findings, the average team went for it on fourth down 15.1 times per season. During the 2005 season, the average NFL team went for it on fourth down 14.5 times. Learning about our mistaken decisions led to even worse decisions.
