Win Probability Graphs

Check out the Win Probability graphs and play-by-play for the current season or for previous seasons. Or check out your favorite team's biggest comebacks and most exciting games. Browse the archive for previous years or the current season by week.

Advanced Stats for Players and Teams

Browse innovative advanced stats for offensive skill players, defenders and teams, including Win Probability Added, Expected Points Added, and Success Rate.

Community Research

Interested in publishing your own football research, analysis, or stat-based commentary? Advanced NFL Stats Community is the site to share your thoughts and ideas. There's plenty of data available to get started.

Sep 15, 2009

The 4th Down Study - Part 1

If there's one topic where quantitative analysis can change the way football is played, it's 4th down decision-making. Many articles here have chronicled the conservative nature play-calling on 4th down in the modern NFL. In this post I'll explain, as clearly and simply as possible, why the evidence points to a more aggressive attack on 4th down.

Previous studies on 4th down decision-making include Carroll, Palmer, and Thorn's book Hidden Game of Football (1988, 1998) and Professor David Romer's Do Firms Maximize? (2005). The first serious study of the concepts used in these studies was by former NFL quarterback Virgil Carter, who co-authored an operations research paper examining the value of field position using data from the first 56 games of the 1969 season.

My own analysis published in this post largely repeats the methods used in previous studies. But I think I can add a good deal to the topic. First, this analysis is based on a much larger data set compared to previous research. Second, this analysis offers possible confirmation of previous results. Third, I think I can explain a complex, abstract subject such as this in a straightforward manner, which is essential if the 4th down revolution is going to make any headway. Frankly, it doesn't matter how strong the analysis is if it can't be communicated clearly and convincingly.

This is how the study goes: At each yard line, I'll calculate and compare the expected point value, based on recent historical averages, of each of the three 4th down options--punt, field goal, or go for it. The option with the highest value is the recommended choice.

I've divided the article into four parts. In this the first part, I review the concept of Expected Points for those who aren't familiar. I also review the concept of expected utility. Both ideas are critical for understanding the thrust of the study. The second part of the article will detail the kicking game. The third part will explore the value of 4th down conversion attempts. The final part of the article will put all the concepts together to ultimately produce a chart of recommended decisions for 4th downs at every combination of field position and distance to go.

'Expected Points'

Speaking of abstract subjects, this entire analysis rests upon the foundation of the Expected Points (EP) concept. Most readers here may already be familiar with EP, but I'm going to summarize it now before I move on to the meat of the study.

EP is the average potential points a team can expect given a certain situation. The most common example is the potential point value of a 1st down at each yard line on the field. EP is the average of all 'next' score values at any given yard line. It's not necessary the average points scored on the current possession because possession could be exchanged several times before the 'next' score. EP is positive when the offense will usually score next, and negative if the defense will usually score next.

Here is the EP graph for a 1st down at each field position. These EP values are based on data from 2,400 NFL games from the 2000-2008 seasons. I used only data from the 1st and 3rd quarters to exclude situations hurried by an expiring clock and by desperate teams or teams with large leads playing differently late in games.


I'll be referring to the graph several times, so be sure you understand how to read it. A 1st down on an opponent's 20 is worth 3.7 EP. But a 1st down on an offense's own 5 yd line (95 yards to the end zone) is worth -0.5 EP. The team on defense is actually more likely to eventually score next.

Note that a 1st down at an offense's own 27 yd line is worth 0.7 EP. This is critical to explain an important twist in the EP concept. Every score requires a subsequent kickoff, and this has value to the receiving team. So to understand the real value of a the score, we need to subtract the value of the kickoff. For example, field goals aren't really worth 3 points. In the long run, they're worth 3 - 0.7 = 2.3 EP. And touchdowns are really worth 6.3 EP.

This concept is especially apparent when considering safeties. After a safety, the scoring team gets the ball back, on average at its own 40, which is equivalent to 1.3 EP. A safety is therefore really worth 2 + 1.3 = 3.3 EP. These score values and the resulting EP values are used throughout the rest of this analysis.

Expected Utility

Before I go any further, I'm going to take a step back and explain the concept of 'expected utility' in general. Say I can choose one of two routes home from work each day. One route is through the side roads, and the other is via the highway. If I take the side roads my commute always takes 20 minutes.

But the highway is more dicey. Sometimes it's clogged with traffic and can take 25 minutes to get home, but most other times traffic is clear and it only takes 15 minutes to get home. The highway is backed up 40% of the time and clear 60% of the time. Which route should I normally choose?

My 'expected commute time' can be calculated using simple proportions. The side roads take 20 minutes 100% of the time, so that's easy--the expected time for that route is 20 minutes. But assuming each minute of my day is equally valuable to me, the highway's expected time would be:

(25 min * 0.40) + (15 min * 0.60) = 19 min

So over the long run, the freeway is the better option. Unfortunately, life isn't that simple. We don't have such clear-cut options and we don't know the probabilities and payoff values of our decisions. But in football, we do.

Continue reading part 2 of this article.

14 comments:

Jason said...

Unfortunately, even if a coach knows the odds, going for it on 4th down is high-risk/low-reward from a social standpoint -- i.e., what will fans/owners/players/media think if we make it (or punt) vs. what will they think if we miss it? If you make it or punt, nobody says anything. If you miss it, why did you go for it there, you idiot? While that shouldn't be the sole deciding factor for a coach, it can have an effect on his job security and will therefore almost certainly figure into his decision-making process.

I tend to think of editing the same way. Nobody notices and rewards good editing, but if I let a few sutble errors slip by in whats supposed to be a profesional text, I'm labeled as a poor editor.

(You probably looked at my first paragraph and didn't think there was anything special about it, but you looked at my second and probably figured I was an idiot. Obviously, there's no "risk" in editing properly, but it's the same general issue of perception. Nobody notices if you do the right thing but they all notice if you fail.)

James said...

I'm excited to see where this is going!

Kulko said...

Jason, of course you are probably nailing the reason why coaches behave as they do. But I think you are missing a few things here.

IF a more aggressive 4th Down Approach has a higher expected utility, it will lead to more wins, which help your job security a lot more then a nice perception.

Also if you read media articles of the last fw years, you will notice that the climate has definitely changed. Coaches get more ofen critized for being to timid, and aggressive failures have ample defenders in the press by now. SO I think even from a social risk/reward standpoint the times are changingand th caches should too.

Happy said...

Ultimately you can't argue with success. The coach may take some grief in the short run for an unconventional decision, but if their decisions have a firm basis in expectation value and increase the team's likliehood of winning the team will win more games.

That's when the coach gets to say "that's why I'm the coach and you're the fans." Not every coach has this level of confidence, but some do.

I'm curious if there are any teams with greater than 50% on 2-point conversions. Those teams should never kick an extra point in the first three quarters of a game.

Anonymous said...

Happy - there's tons of info on the net about the TPC situation

Part of the reason it can be successful is that teams can try in unexpected scenarios

But there is another school of thought that suggests TPC should be reserved (+ or -) for the last six minutes. A team trailing by 14 on scoring a TD should go for two because if successful the game can be won on the tying TD with the PAT. If it fails, the game can still be tied with a TPC. This route gives the best chance of winning without the dreaded OT

Eugene Carpet said...

Great trin of thought. I have never seen such a complete analysis of points. What really struck me is the expected points from a safety. Really cool.

Anonymous said...

Yes. Yes. Yes.

-Belichick fan

Anonymous said...

How does this breakdown in the case of last nights game? The model should include:

1. Home vs away team
2. Lead or trailing at the time and by how many points
3. Current field position
4. Time left in the game
What does the model look like and tell us then? You should also weight the QB rating for the offense, and in the case you don't make the 1st down, the other teams QB coming back at you to score. That likelihood would go up the deeper you are in your own territory of course.

Anonymous said...

Here is the problem with this argument. The odds of making the first down on fourth and one or fourth or two are statistically good. That is great and seems to tell us that "going for it" is a good decision. However it is not a good decision when you are in your own territory at all. Even if you team has better fourth down efficiency than the average. Here is why. Let's say your team makes 75% of their 4th and 2s. A very high percentage. The other side of that is that there is a 25% chance of not making it or a TURNOVER. What are the odds on a turnover on any other play? A run with a decent running back? I believe that the average is somewhere around 1 fumble for every 40 carries. How about intercepts per pass attempt? For a good QB it is around 1 in 30. For a bad QB it is lower. Now I am a coach in the NFL and I have the ball on the 30 yard line and I am going to run a play that has a 25 to 45% chance of turning the ball over? Which is the bad decision? Punting the ball and giving the ball to the other team in their territory when the best team in the NFL has around a 30% chance of scoring a Touchdown in that situation or risking an extremely high turnover play and giving the same team a 40 to 50% chance of scoring a TD and a much higher chance of kicking a field goal. I think we can see why 90% of the time or higher NFL coaches make the desicion they make.

All that being said. I am not questioning Bill Belichek's call. He has a better feel for what his team is doing on the field than I have. He would win the game with a first down. How his line is blocking. How beat up the other D line is. How his backs are running. However in most cases coaches should and do opt to punt in this situation. The odds of a TURNOVER are too high.

Chris said...

Anonymous...here is the problem w/ your argument. A punt is a guaranteed turnover. Its throwing the ball 40 yards downfield and a 100% chance of interception. If you go for it on 4th down you are limiting your chances to a turnover to 25% (according to your math) so what's your preference on any given play, a 100% chance of a turnover or a 25% chance of a turnover.

Bill said...

An issue of some interest might be play selection in go-for-it mode. It appears that most coaches want to dominate the other team, thus multiple straight-ahead runs is a common choice (and commonly leads to failure). The Broncos used to have good results from variations off an Elway bootleg. At the very least, it adds some deception for the next crack at the center of the line!

Anonymous said...

This is an interesting discussion no doubt, but it seems a little too analytical and not so much practical. The previous anonymous poster hit it right that the Pats coach has a feel for whats going on on the field better than anyone else. Analyzing data can be a useful tool when calculating percent success over all teams over decades of football. It does not, however necessarily relate to the given game being played as all teams are of different skill level from talent to play calling and certain teams are just going to flat out be better at converting than others. Some may be successful some of the time while others may never squeeze a win or points after a decision to go for it. So many more variables would affect a given decision to go or punt that it would be impossible for one to accurately calculate the chances of winning a game based on this generic quantitative analysis.

Ben Rushing said...

What most people don't know, but should is that football is like a coin that has two sides. Everyone knows the universal truth of the first side of the coin that says that a team that scores more points than the opponent always wins. What people don't know is how a team lost a game without factoring the score. The other side of the coin would say that the team with the most real turnovers always losses the game. Statisticians do not count all events that turn the ball over without first scoring atleast 6 points. If they did they would see that the team with the higher number always losses. Try this with any game and you will see that it works:

Complete Turnovers: Interceptions, fumbles lost, missed fg's, punts, 4th down fails, kickoffs lost, safeties, and turnovers due to time (given if the team in possesion at the end of the half failed to score and has to kick to start the second half. Not counted if another turnover half or whole occurs at the same time. Also when the game ends and the team trailing fails to score before the time expiresn not counted against the team with the lead. also nullified if another turnover takes place.

Half Turnovers: Failed extra point or 2pat, Successful field goal, 2point conversion allowed.

Anonymous said...

Nomination for worst 4th down decision of the year so far goes to Sean Payton for punting on 4th and 2 with 5 mins left in the game down by 10 lol

Post a Comment