Week 4 Efficiency Rankings

NFL team efficiency rankings are back for 2008. The ratings are listed below in terms of generic win probability. The GWP is the probability a team would beat the league average team at a neutral site. Each team's opponent's average GWP is also listed, which can be considered to-date strength of schedule, and all ratings include adjustments for opponent strength.

Offensive generic win probability (OGWP) is based on each team's offensive efficiency stats only. In other words, it's the team's GWP assuming it had a league-average defense. DGWP is a team's generic win probability assuming it had a league-average offense.

GWP is based on a logistic regression model applied to current team stats. The model includes offensive and defensive passing and running efficiency, offensive turnover rates, and team penalty rates. A full explanation of the methodology can be found here. This year, however, I've made one important change based on research that strongly indicates that defensive interception rates are highly random and not consistent throughout the year. Accordingly, I've removed them from the model and updated the weights of the remaining stats.

RANK	TEAM	GWP	Opp GWP	OGWP	DGWP
1	WAS	0.78	0.62	0.79	0.73
2	SD	0.72	0.53	0.73	0.58
3	NYG	0.69	0.40	0.73	0.69
4	CAR	0.66	0.58	0.65	0.72
5	PHI	0.66	0.50	0.63	0.66
6	DAL	0.65	0.49	0.77	0.53
7	NO	0.61	0.56	0.70	0.53
8	ARI	0.58	0.56	0.54	0.56
9	BUF	0.56	0.40	0.48	0.61
10	MIN	0.56	0.54	0.56	0.60
11	TEN	0.55	0.39	0.46	0.64
12	CHI	0.55	0.55	0.49	0.69
13	PIT	0.55	0.45	0.43	0.68
14	NYJ	0.54	0.54	0.39	0.60
15	DEN	0.53	0.50	0.66	0.26
16	OAK	0.52	0.54	0.41	0.59
17	SF	0.49	0.43	0.41	0.54
18	MIA	0.48	0.49	0.54	0.37
19	TB	0.48	0.52	0.50	0.61
20	ATL	0.47	0.38	0.58	0.41
21	IND	0.46	0.48	0.53	0.44
22	JAX	0.45	0.54	0.57	0.32
23	BAL	0.44	0.37	0.41	0.64
24	SEA	0.44	0.49	0.42	0.49
25	HOU	0.40	0.56	0.44	0.31
26	GB	0.39	0.45	0.41	0.53
27	NE	0.37	0.43	0.37	0.29
28	STL	0.35	0.63	0.51	0.27
29	CIN	0.34	0.55	0.31	0.52
30	KC	0.30	0.53	0.20	0.37
31	CLE	0.23	0.53	0.32	0.32
32	DET	0.20	0.53	0.33	0.15

To give everyone an insight into why the rankings are what they are, here are the team efficiency stats.

TEAM	OPASS	ORUN	OINTRATE	OFUMRATE	DPASS	DRUN	DINTRATE	PENRATE
ARI	7.5	3.2	0.028	0.032	6.4	4.0	0.017	0.40
ATL	5.6	5.6	0.019	0.005	5.9	4.6	0.031	0.40
BAL	4.6	3.9	0.025	0.029	3.7	3.1	0.059	0.48
BUF	6.6	3.5	0.017	0.019	5.1	4.2	0.025	0.24
CAR	6.2	3.6	0.008	0.021	5.1	4.1	0.008	0.51
CHI	5.4	4.1	0.033	0.023	5.3	3.7	0.028	0.40
CIN	4.4	3.5	0.048	0.039	5.6	4.3	0.009	0.37
CLE	4.0	3.5	0.051	0.024	6.7	4.1	0.057	0.52
DAL	8.2	4.9	0.029	0.031	5.9	4.2	0.000	0.53
DEN	7.9	4.7	0.025	0.029	7.8	5.0	0.007	0.24
DET	5.2	4.4	0.055	0.024	8.7	5.6	0.000	0.36
GB	6.5	3.9	0.023	0.043	5.4	5.2	0.057	0.64
HOU	5.4	4.2	0.045	0.033	7.0	4.8	0.013	0.18
IND	6.0	3.6	0.033	0.008	6.0	4.9	0.015	0.33
JAX	5.5	4.3	0.034	0.005	7.3	4.1	0.034	0.32
KC	4.1	4.9	0.045	0.026	7.0	5.3	0.018	0.22
MIA	6.2	4.4	0.010	0.028	7.4	3.3	0.012	0.39
MIN	5.2	4.7	0.023	0.025	6.1	2.9	0.016	0.40
NE	5.2	3.9	0.011	0.020	6.9	5.0	0.026	0.14
NO	8.6	3.4	0.027	0.023	6.4	5.2	0.027	0.55
NYG	6.6	5.4	0.009	0.006	4.5	3.8	0.010	0.47
NYJ	6.4	3.8	0.039	0.021	6.6	3.1	0.034	0.34
OAK	5.6	4.8	0.010	0.041	6.1	4.0	0.039	0.50
PHI	6.7	3.5	0.019	0.023	5.5	2.6	0.033	0.35
PIT	5.5	3.4	0.021	0.037	4.4	2.9	0.045	0.35
SD	8.5	3.9	0.036	0.016	5.9	4.5	0.029	0.25
SF	6.6	4.7	0.027	0.037	6.0	4.0	0.038	0.35
SEA	4.9	4.9	0.031	0.013	6.5	3.5	0.011	0.42
STL	4.8	4.0	0.025	0.024	8.2	4.7	0.009	0.43
TB	5.3	5.0	0.041	0.010	6.1	3.6	0.061	0.55
TEN	6.3	3.8	0.029	0.024	4.5	3.7	0.056	0.31
WAS	6.3	4.3	0.000	0.000	5.9	4.3	0.034	0.32
AVG	6.0	4.2	0.027	0.023	6.1	4.1	0.027	0.38

Home Field Advantage by Quarter

In his 2007 paper Home Advantage in the NBA as a Game-Long Process, Marshall Jones found that home court advantage in the NBA is not consistent throughout a game. Instead, it's disproportionately realized in the 1st quarter. The home advantage diminishes in the 2nd and 3rd quarters, then is smallest in the 4th quarter. He also found that when home teams enter a quarter behind, they tend to substantially outscore their visiting opposition. So how about in the NFL?

In the NFL, home teams also enjoy an advantage, although not as large as in the NBA. HFA is commonly thought of as about 3 points or so, and home teams win 57% of regular season games.

Below is the breakdown by quarter of the home team's share of points.



Overall, home teams score 52.5% of all points through the entire game. By far the largest advantage is seen in the 1st quarter, when home teams score 54.7% of the points scored during that period. The 2nd and 3rd quarters see a significant drop off, followed by the 4th quarter which shows the smallest advantage.

So just like the NBA, HFA in the NFL is primarily realized in the 1st quarter. Now let's compare how home teams fare when behind or ahead.



When trailing, home teams show the same pattern as the overall share in the 1st graph. But when ahead, home teams show no pattern, scoring about 59% of the points in each quarter. So there is a difference in the HFA effect not only by quarter, but by score difference as well.

Note that although trailing home teams score fewer than half of all points, this is what we'd expect. A team that's behind is likely the weaker team, so we would not necessarily expect them to outscore opponents, whether at home or not. This would not mean there is a home field disadvantage.

With that in mind, we can just look at points scored when the game is tied to (partially, at least) account for bias due to team strength.


Again, we see HFA diminish through the game. In fact, home teams score only 49% of points in the 4th quarter. My guess is that home teams that are tied in the fourth quarter have enjoyed 3 quarters of HFA, so they would tend to be the slightly weaker team without HFA. And since HFA becomes weakest in the 4th quarter, we're still seeing some bias due to team strength. In other words, it's better to be good than to be at home, especially in the 4th quarter.

This is more than just random trivia. Understanding how strong HFA is throughout the game helps us understand where HFA comes from. For example, as Marshall points out in his paper, it was commonly thought that HFA comes mostly from the crowd. It was expected therefore, that HFA would be strongest in the 4th quarter when crowds are typically loudest and most involved in the game, especially when the score is close. Also, if travel fatigue is the cause, then we'd expect HFA to be strongest toward the end of the game when fatigue becomes most important. But we see that the opposite is the case, so HFA may come from somewhere else.

I think there are probably multiple processes involved. There could be game-long effects, such as the crowd, but then there could be another process that diminishes as the game progresses. But whatever the underlying causes, HFA is strongest at the beginning of the game and diminishes as the clock winds down.

In-Game Win Probabilities Beta

Today I'm testing a real-time win probability site. The status of each ongoing game will be reported along with the probability of winning for each team. The probabilities are based on over 2000 games from the past 8 regular seasons. For now, the model is fairly basic and considers score, time, and possession. Modifications for field position, down and distance, and other factors are in work. It's an extremely challenging project, to say the least. But even now, the model is very revealing.

But for now, readers of this site are invited to check out the beta version during the games today. Be warned there will certainly be hiccups, bugs, and other problems throughout the day. So if it's not working one minute, it may be back up the next. It goes live shortly after the 1 o'clock kickoffs today. Comments and suggestions are more than welcome!

The link is wp.advancednflstats.com.

(Be patient. It's a very slow homemade server.)

Game Probabilties Week 4

Win probabilities for week 4 NFL games are listed below. The probabilities are based on an efficiency win model explained here and here with some modifications. The model considers offensive and defensive efficiency stats including running, passing, sacks, turnover rates, and penalty rates. Team stats are adjusted for previous opponent strength.

Pwin	Visitor	Home	Pwin
0.47	SF	NO	0.53
0.35	HOU	JAX	0.65
0.21	CLE	CIN	0.79
0.38	ATL	CAR	0.62
0.61	ARI	NYJ	0.39
0.44	MIN	TEN	0.56
0.72	DEN	KC	0.28
0.36	GB	TB	0.64
0.64	SD	OAK	0.36
0.72	BUF	STL	0.28
0.56	WAS	DAL	0.44
0.46	PHI	CHI	0.54
0.36	BAL	PIT	0.64

Week 3 Efficiency Rankings

NFL team efficiency rankings are back for 2008. The ratings are listed below in terms of generic win probability. The GWP is the probability a team would beat the league average team at a neutral site. Each team's opponent's average GWP is also listed, which can be considered to-date strength of schedule, and the ratings include adjustments for opponent strength.

GWP is based on a logistic regression model applied to data through week 3. The model is based on offensive and defensive passing and running efficiency, offensive turnover rates, and team penalty rates. A full explanation of the methodology can be found here. This year, however, I've made one important change based on research that strongly indicates that defensive interception rates are highly random and not consistent throughout the year. Accordingly, I've removed them from the model. Removing defensive interception rates from last year's prediction model did not harm its predictive ability by a single game.

There will be some surprises in the rankings, so take them with a heavy grain of salt. The results are so nutty, I considered not posting them. But the whole point of this site is to let the numbers speak for themselves, so here they are. This is the time of year that makes the least intuitive sense. Most fans, including myself, are wrapped up in how good various teams "should be" or "were last year." But those notions are based on old information mixed with media hype. With only 3 games of data, there's not a lot to go on. But at this point last year, 10 of the top 11 teams in GWP rankings went on to make the playoffs.

After week 3, the top team is 2-1 Washington, which stands out by virtue of its zero turnovers--no interceptions and no fumbles, lost or otherwise. Last year's AFC South powerhouses Indianapolis and Jacksonville are ranked 23rd and 24th , below even hapless Cincinnati. The explanation is in their respective strengths of schedule. The AFC North might be the strangest division, with 0-3 CIN leading in efficiency, and 2-0 Baltimore 3rd out of 4 teams. Lots more surprises below.

Click on the table headers to sort.

RANK	TEAM	GWP	Opp GWP
1	WAS	0.72	0.61
2	ARI	0.71	0.60
3	SD	0.70	0.50
4	DEN	0.63	0.57
5	SF	0.62	0.45
6	NYG	0.61	0.40
7	BUF	0.60	0.45
8	DAL	0.58	0.44
9	NO	0.57	0.55
10	PHI	0.56	0.47
11	TB	0.55	0.57
12	CAR	0.54	0.59
13	NYJ	0.53	0.55
14	MIA	0.51	0.52
15	ATL	0.51	0.34
16	CHI	0.51	0.47
17	CIN	0.51	0.64
18	MIN	0.51	0.49
19	GB	0.49	0.48
20	OAK	0.49	0.50
21	SEA	0.48	0.51
22	TEN	0.47	0.36
23	IND	0.47	0.48
24	JAX	0.47	0.59
25	PIT	0.42	0.37
26	HOU	0.40	0.58
27	NE	0.38	0.42
28	BAL	0.38	0.29
29	DET	0.37	0.62
30	KC	0.32	0.53
31	STL	0.29	0.58
32	CLE	0.28	0.60

Payton's Gamble

In last Sunday's game between New Orleans and Denver, down 24-17 with 30 seconds left in the 2nd quarter, the Saints faced a 4th down and goal from the 1 yard line. Normally, all the numbers say 'go for the touchdown,' and this was indeed what the Saints' decided to do. I love aggressive decisions on 4th down, but we'll see why going for the field goal would have been the slightly better decision.

According to the Romer paper, going for it on 4th goal makes sense anytime the offense is inside the 6 yd line. The key to the advantage in going for the TD is that a failed attempt leaves the ball deep in an opponent's own territory. It's likely that the opponent will end up punting and giving the ball back in excellent field position, or even allowing a safety.

And that's exactly what happened Sunday. The Saints failed in their attempt, leaving the ball for the Broncos at the 1. On the very next play, the Saints stuffed a run and scored a safety. And even though they got 2 points out of the situation, it was a highly improbable outcome. They should have kicked the field goal.

The fundamental difference between the normal Romer-type expected points analysis and this situation is that there was under 30 seconds left in the half. Neither team had any time outs remaining, so neither team had time to mount a follow-on drive. The lack of time means that the benefits of the follow-on field position are very limited. It also means that the value of the ensuing kick after a score doesn't have to be factored in. With only a few seconds remaining in the half, a kick off would undoubtedly be of the 'squib kick to the 3rd string TE' variety. There's no chance it would be returned.

We can evaluate the decision in a simple expected utility calculation. The chance of scoring from the 1 yd line is 39%, and field goals are 99% successful from that range. The chance of forcing a safety from the 1, given an unsuccessful TD attempt, is 4%. (In the past 8 years, there have been 267 runs from the 1, featuring 12 safeties and zero fumbles. The Broncos would have been crazy to pass, so I'll ignore the possibility of an interception.)

Value(TD attempt)= 7 * P(successful TD attempt) + 2 * P(unsuccessful TD) * P(Safety)
= 7 * 0.39 + 2 * (1-0.39)(0.04)
= 2.73 + 0.05
=2.8 points

Value(FG attempt)= 3 * P(successful FG attempt)
= 3 * 0.99
= 3.0 points

The percentage play for the Saints would have been the field goal in this situation, but not by much. If for some reason Sean Payton thought his offense had a much better than the league-average chance of scoring from the 1, it would have made sense. But that would be a stretch given New Orleans' lack of a power running game.

To prevent a safety, Shanahan might have called for the QB sneak. The Broncos only needed to take a single snap, and the sneak is probably the most safety-proof type of play. I suspect that he feared a fumble more, and didn't want the ball in Cutler's hands in that situation.

This situation is a good example of how the end of the half alters the equation for decision making. Early in the half, we can treat the flow of the game as effectively infinite. But as the clock winds down, we have to account for the effect of an approaching time horizon.

What's a Safety Really Worth?

Safeties are so cool. Nothing fires up a defense and demoralizes an offense like a safety. They also throw off the 7 and 3-point arithmetic that football scores almost always follow. I always enjoy watching the score ticker at the bottom of the TV and thinking, “PIT 7 CLE 5?...How'd they get...oh yeah.”

But safeties are rare, with only 109 of them over the past 8 years, or about 1 in every 20 games. They’re also unique because the scoring team gets the ball. A free kick from the 20 yd-line usually means pretty good field position, and this is what makes safeties worth more than you might think.

Say you won $7 in a lottery 'scratcher.' But to go claim your prize, you’d have to use about $1 of gas. You could get stuck in traffic and it could cost $2, but you might get a ride from a friend and it would be free. But on average it would cost a buck. How much is that lottery ticket worth now? Now apply the same concept to football.

After a touchdown or field goal, the scoring team has to give possession of the ball to its opponents through a kick off. The resulting average field position is the 27 yd-line. In contrast, after a safety, the scoring team gets the ball back with average field position at its own 40.

In abstract terms, a touchdown really isn’t worth 7 points. Given enough time for the opponent to score, it’s really worth 7 points minus the expected point value of having the ball at the 27. The same principle applies to field goals.

Similarly, safeties aren’t really worth 2 points. Their ultimate value is 2 points plus the expected point value of getting the ball at the 40. Teams with 1st downs at their own 40 can expect to score 1.6 points on average, (assuming there is time to mount a drive). This makes the net value of a safety 3.6 points.

The table below lists the scoreboard point value of each type of score, the associated expected value of the ensuing kick, and the resulting net value.

Score Type	Point Value	Kick Off Value	Net Value
Touchdown	7	-0.7	6.3
Field Goal	3	-0.7	2.3
Safety	2	+1.6	3.6

Two-point safeties are actually (or abstractly, if you prefer) worth more than three-point field goals. And more importantly, field goals aren't almost half the value of a touchdown. They're worth closer to a third.

Predictability on 2nd and 10

For any down and distance situation, a defensive coordinator wants to know how likely a run or pass play will be. He needs to select the right personnel and the right defensive scheme for the situation. Take the 2nd and 10 situation, the second most common down and distance combo in the NFL (1 in 5 of all 1st downs results in a 2nd and 10). In general, offenses tend to pass 55% of the time and run 45%. So defensive coordinators need to be equally prepared for any kind of play type. Or do they?

For offenses to be most effective, they need to be unpredictable. In the 2nd and 10 situation, this means defenses would have to prepare for the nearly equal chance of a run or pass. Many analysts refer to 'balance' as the key to unpredictability. But balance itself doesn’t matter if the offense is predictable in achieving its balance. Running and passing on every other down would provide perfect balance but would be completely predictable. That’s why randomness is at least as important as balance to keeping the defense on its heels. Anything other than random play selection provides a pattern, however subtle, that an opponent can detect and exploit.

In a recent article, I discussed an interesting pattern in NFL 2nd down plays illustrated below. Note how runs are far more common on 2nd and 10 than either 9 or 11 yards to go. The graph is basically continuous and smooth except for a notable spike in run plays on 2nd and 10.


This struck me as odd because 2nd and 10 is not tactically different than 2nd and 9 or 11. The situations are basically the same. My theory was that offenses were running more frequently on 2nd and 10 because that situation arises most often due to an incomplete pass, and offenses tend to predictably alternate between passes and runs. This would result in the unexpectedly high percentage of run plays on 2nd and 10.

I’ve dug into the data now, and I’ve confirmed my suspicion. The graph below illustrates the relative share of pass and run plays based on what kind of play the preceding 1st down was. On 2nd and 10, teams indeed run more often after a pass than after a run, and vice versa.

(The data consist of 14,384 2nd down plays in the 1st through 3rd quarters of all regular season games from 2000-2007. Fourth quarter plays were excluded to remove the possible biases from 'trash time' and running out the clock.) The graphs should be read as follows: The left column are 2nd down plays following a pass, and the right column are plays following a run. The blue portion of each column is the % of pass plays, and the yellow portion is the % of run plays.


This is significant because armed with this information, defensive coordinators can select personnel and plays tilted toward the anticipated play type. They no longer have to be on their heels without an idea of what to expect on 2nd and 10. If the previous play was a run, a coordinator can now be 72% confident the next play will be a pass.

But not all teams have the same tendencies. Compare Brian Billick’s Ravens with Bill Cowher’s Steelers over the 2000-2006 period. The Ravens were far more predictable compared to the Steelers. Cowher’s teams selected 2nd down plays without regard to what kind of play was called on 1st down, but Billick’s teams tended to follow a run with a pass.




(Statistically, the Ravens’ difference in proportions is significant at p<0.001. Typically, a single team’s proportions would need to be within about 8% to be considered non-significant. But that still would not indicate good non-predictable play calling independent of the previous play. The proportions would typically need to be within 3% to be more likely due to randomness than not.)

One counter-argument to this analysis is that teams are wisely choosing to run after an incomplete pass. If a pass falls incomplete, that would be fresh evidence about an offense’s ability to complete passes against this particular opponent. A run stuffed at the line is similarly an indication of each team’s relative strength. Shouldn’t offenses shy away from unsuccessful tactics? Doesn’t it make sense to try the alternate strategy next?

I would say no for two reasons. First, this would be a classic example of the small-sample fallacy, otherwise known as the hasty generalization. Over 40% of all passes are incomplete in the NFL. The outcome of a single pass should not be the basis of a change in strategy, however slight. The sample size of an entire game of passes would still not be enough to make conclusions about its relative merits as a strategy against a particular opponent. Second, even if the evidence of the single trial were so overwhelming, tending heavily toward the alternate play type on successive plays makes the offense too predictable, as we’ve seen here.

Coaches and coordinators are apparently not immune to the small sample fallacy. In addition to the inability to simulate true randomness, I think this helps explain the tendency to alternate. I also think this why the tendency is so easy to spot on the 2nd and 10 situation. It’s the situation that nearly always follows a failure. The impulse to try the alternative, even knowing that a single recent bad outcome is not necessarily representative of overall performance, is very strong.

So recency bias may be playing a role. More recent outcomes loom disproportionately large in our minds than past outcomes. When coaches are weighing how successful various play types have been, they might be subconsciously over-weighting the most recent information—the last play. But regardless of the reasons, coaches are predictable, at least to some degree. Fortunately for offensive coordinators, it seems that most defensive coordinators are not aware of this tendency. If they were, you’d think they would tip off their own offensive counterparts, and we’d see this effect disappear.

In case anyone's interested, here are some other team’s tendencies on 2nd and 10. I picked these teams because they’ve had the same head coaches over the entire period of the data set.

Predictability

Over the past few months I've been writing about how game theory can help us understand play-calling in football. Not only can it help us understand why coaches call the plays they do, but it can instruct us on what the optimum balance of play types should truly be. Offenses always need a mix of strategies to maximize their gains, no matter how much better they might be at running over passing or vice versa. But just important as the ratio of the strategies is the unpredictability of each call.

The mix of plays needs to be random to be effective. That's not to say play calling should be picked willy-nilly out of a hat. For every situation there will be an optimum ratio of play types. For example, on 3rd and 1 situations, teams should generally run at least about 85% of the time. But within that 85/15 run-pass mix, the decision needs to be unpredictable, which means it must be random and independent of the previous play call. The problem for play callers, and the opportunity for defensive coordinators, is that people are terrible at randomizing.

There's a story about a statistics professor who challenges his class to a contest. He divides the class in half and tells one group to flip a coin 100 times and write the sequence on the board-- THHTTH... The other group is told to invent and write their own sequence of heads and tails on the board as randomly as they can, without looking at the other group's sequence. The professor says that if he can't tell the true random sequence from the fake one, he'll give everyone an A (or something). He leaves the room until both groups are done, then returns and instantly spots the fake sequence.

The professor can identify the fake random sequence so easily because it has too many alternations between heads and tails, and too few long streaks. The fake sequence looks like HTTHTHHTHT, while true randomness often looks like HHHHHTHTTH. True randomness can be quite streaky (which is partly why people fall for fallacies like "being in the zone" or "the hot hand").

If I'm a defensive coordinator, I'd like to know what kind of play the offense is going to run. I don't need absolute certainty--any idea is better than no idea. For example, for all 2nd and 10 plays in the NFL, offenses run the ball 46% of the time. But what if defensive coordinators could know that based on other circumstances, this particular 2nd and 1o will be a pass 80% of the time?

Take a look at run-pass balance on 2nd down situations. The graph below shows the percentage of run plays on 2nd downs according to the yards-to-go situation. There is one noticeable aberration at 2nd and 10: runs are far more frequent.


Why would runs be far more frequent on 2nd and 10 yards to go than on 9 or 11 yards to go? The task facing the offense is not meaningfully different. What makes 10 yards to go so special?

The key is that 2nd and 10 situations are several times more likely to occur due to an incomplete pass than due to a run for no gain. This suggests that, effectively, teams are significantly more likely to run following a pass than pass following a pass. Therefore, offenses are not randomizing as much as they are alternating.

Play calls are not independent of the previous plays, even for similar down and distance situations. NFL offenses are therefore substantially, although not completely, predictable.

Instead of HTHHTHTH in a statistics class, we have RPRRPRPR in football. The patterns remind me of a language with consonants and vowels. But play calls are not simple either/or run or pass decisions. There are several variations to each, just like there are a's and e's and b's and c's.

Linear B was an ancient written language found in Crete and named for its straight lines. It was a precursor to the Hellenic Greek language dating back to the times of the Homeric epics. Several tablets with etchings in Linear B were excavated in Knossos, thought to be the capital of King Minos. The script was completely unlike any other, and baffled archeologists for decades. Researchers had nothing to go on except the patterns found in the writing.

In the mid-1940s Alice Kolber, an American professor, theorized that the characters represented syllables and that the language was highly inflective (having lots of different conjugations). Then in the 1950s, an amateur archeologist named Michael Ventris cracked the code. He found that each character represented a consonant-vowel combination. Each sound a person can make either goes well with others or it doesn't. This was all that was needed to eventually decode Linear B and unlock all its secrets.

Just like vowels and consonants, runs and passes tend to alternate. And certain types of plays tend to work well before and after others, just like "th" or "rn" or other consonant combinations.

I'm not claiming that we can crack the code on play-calling any more than you can predict my next word. My point is that a serious cryptographic analysis of play-calling could reveal tendencies not previously thought possible. For example, try to predict the next letter of the word "th..." Chances are very good it's a vowel, and if it's not, it's got to be an 'r.'

I'm sure coaches pore over hours of film trying to discern opponent tendencies, and are looking for things I couldn't even fathom. But it seems that they are focusing on situations in isolation. They're zeroing in on observations like, "they run on 2nd and long 35% of the time in the red zone." Apparently, coaches are not picking up on the fact that the same team might run 65% of the time in the same situation following a pass. If they were detecting these patterns, they wouldn't let their own offenses be so predictable.

I realize this is a wondering essay. My main points are that:

1. Offenses need to be unpredictable to be effective.
2. Plays need to be random both with respect to previous instances of the same down/distance situation and with respect to previous plays.
3. NFL offenses show evidence of patterns, even when holding for situational effects.
4. Coaches don't seem to be aware of the patterns, and they can be exploited.
5. And lastly, the only true countermeasure is to somehow inject genuine randomness into play calls.

I was going to finish this article by retelling an Edgar Allen Poe story in The Purloined Letter. But as I researched the details of the story, I realized I was beaten to the punch by the Smart Football blog. Check out this article on Poe, rock-paper-scissors, and play-calling.

Blindsided?

Michael Lewis, author of the best-selling baseball book Moneyball, recently followed up with a book on innovation in football. The Blind Side follows the story of the left tackle, the player whose job of protecting the more vulnerable side of right-handed quarterbacks has become increasing important in the NFL ‘arms race’ of the pass rush vs. passing offense.

The entirety of Lewis’ premise is based on the relative pay of LTs compared to other positions. Lewis cites the fact that LT has become the second highest paid position, behind only the all-important QB. Unfortunately, the comparison of LT salaries with those of other positions is a false comparison, and a fairer comparison reveals a different story.

I was intrigued by Phil Birnbaum’s response to a write-up of Blind Side at the Freakonomics blog. Phil questioned the justifications for the extremely high salaries for LTs. And although I believe there are sound economic reasons based on the scarcity of qualified players and the contribution of the position, my main concern questioned the premise that LT salaries are truly any higher than other positions.

Like many other positions, offensive tackles are largely ’swappable’ in that they can go from left to right pretty easily. Most backups don’t even have a defined side and are available to fill in on either side to spell a starter or replace him in case of injury.

Due to the 'blind side' consideration, the LT is almost always the better of the two starting tackles on each NFL team. And he’s very likely to make a lot more money than the lesser player who is assigned RT. Starting LTs are basically a group of the #1 offensive tackles from each of the 32 teams.

So when we compare average salaries of LTs to those of say, left corner back or all starting wide receivers, the comparison is not fair. Those positions do not place the better player on a certain side, or they are not defined as left/right positions to begin with. And if a player does always line up on one side, it’s not always the same side for every team.

If we compared the average salaries of LTs to the average salaries of all the best WRs from each team, we might expect to see drastically different results.

A much fairer comparison of position salaries is to compare the average salary of the 32 top paid offensive tackles, whether left or right, with the top 32 salaries of players at WR, CB, or various other positions. So that’s what I did.

I looked at the average of the top 32 salaries of 2007 at OT, QB, WR, CB, and RB. Because a player’s salary is a convoluted mix of regular salary, signing bonuses and other bonuses, I favor salary cap charges as the best measure of salary. A cap charge is basically a player’s base salary plus an amortized amount of bonus salary. I think it’s the best measure because it most realistically reflects the value of the player to the team. Total salary and base salary, the only other plausible measures, can be highly irregular based on the particular timing of bonuses. However, I’ll include all three types of salary below the graph, and you can judge for yourself.

The graph and table below list the salaries in $millions for the 32 highest paid players at various positions.

Average Salary of Top 32 Players by Position ($ million)

	QB	OT	WR	CB	RB
Base Salary	2.9	2.3	3.5	3.3	1.8
Total Salary	5.8	4.9	5.7	5.7	4.9
Cap Charge	5.6	4.5	5.2	5.4	3.8

The 32 highest paid offensive tackles, whether left or right, rank only 4th out of 5 in all three measures of salary. I haven’t looked at other positions yet, so there may be others that are higher paid than OT. Further, only 33 of the 100 top paid offensive linemen were tackles, left or right.

While I agree LT is a critically important position and should be highly paid, the comparison of salaries against other left/right positions, or non-“sided” positions is severely biased. A fairer comparison reveals that the top players at other positions are paid even higher salaries.

Are Coaches Aggressive Enough on 2nd and 1?

In a recent post we saw that having 2nd down and 1 is actually preferable to a 10-yard gain for a 1st down. So if 2nd and 1 is really that valuable, are NFL offenses taking advantage when they get one? Are they actually taking shots down the field? The advantage is only as big as offenses make it. This post will look at how coaches exploit the 2nd and 1 situation.

Coaches are calling run plays on 78% of 2nd and 1 situations, which is even more than the 76% share for 3rd and 1s. For all 2nd down situations, runs are called 50% of the time. This indicates that no, coaches are not capitalizing on the opportunity and are treating the 2nd and 1 simply like an extra 3rd down.

Only 4% of 2nd and 1 plays are long pass attempts, defined as passes at least 15 yards down through the air. This is fewer than nearly all other 2nd down situations (only 2nd and 4 has fewer deep attempts). The remaining 18% are "short" pass plays, but those could be as long as 14 yards. It's these short passes that have generated most of the tactical advantage that makes 2nd and 1 preferable to a fresh 1st down. The chart below lists each play type and their associated expected points, the number of each type and the percentage.

Play Types on 2nd Down and 1

	Run	Short Pass	Deep Pass
Exp Pts	1.9	2.2	2.2?
Count	231	54	13*
Share	78%	18%	4%

Because there were only 13 deep pass attempts in the entire data set, I had to estimate their expected points from a larger set of plays. I grouped the deep passes on 2nd down and 1 through 4 yards to go. The resulting average expected points was 1.9, however, the expected points for 2nd and 1 would be higher. If unsuccessful, having a 3rd and 1 is far preferable to a 3rd and 2, 3, or 4 by a weighted average of 0.5 expected points. Deep passes are successful an average of 35% of the time, so this difference would be realized 65% of the time. This equates to a bonus of at least 0.3 expected points.

This suggests that the advantage in scoring from having a 2nd and 1 exists almost purely from the inherent nature of the 2nd and 1, and not due to coaches' deliberate efforts to capitalize on the opportunity. Imagine how large the advantage could be if coaches exploited it with more passes--short or long.

Although it would be hard (or impossible) to convince players to refrain from that second effort plunge at the 1st down marker, it may be easier selling coaches on taking greater advantage of the 2nd and 1 when they do come along.

The Vikings' 2-Point Attempt

With just over 14 minutes remaining in their game against the Packers, the Vikings scored a touchdown to make the score 17-12. Head coach Brad Childress elected to go for the 2-point conversion attempt. Down by 5 at that point, a 2-point conversion would put the Vikings within a field goal. At first glance, it makes sense. But was this a good decision? And how would we know anyway?

I think the best way to judge a decision like this is to use win probability. Essentially, WP is the probability a team will win a game given the current state of the game. We can compare the WPs for a team down by 5, 4 and 3 according to the likelihood of the outcomes of each extra point conversion option.

First, let's look at the conversion probabilities. Extra point kicks are good 99% of the time, and 2-point conversions are successful 44% of the time.

Second, lets look at the WP for the potential score differences at about 14 minutes left in the game. A team down by 5 has a 17% chance of winning, a team down by 4 has a 19% chance, and a team down by 3 has a 20% chance.

Combining the probabilities gives us the following probabilities of winning:

Kicking the extra point:

WP (kicking xp) = P(good) * WP(-4) + P(miss) * WP(-5)
= (0.99 * 0.19) + (0.01 * 0.17)
= 0.19

Going for the 2-point attempt:

WP(2-pt att) = P(success) * WP(-3) + P(no good) * WP(-5)
= (0.44 * 0.20) + (0.56 * 0.17)
=0.18

So it's approximately a wash. The Vikings would have a 19% chance of winning by kicking the extra point and an 18% chance of winning by going for the 2-point conversion.

Childress' decision seemed unusual. It's not that I'd expect him to do the math, but NFL coaches typically will only make the less-conventional decision when the odds are painfully obvious. Why would he make that decision without much to gain?

My theory is that he was tempted by the lure of the 3-point deficit. Field goals are obviously easier to come by than touchdowns, so we could say he was playing for the tie. I think he overestimated the chance of winning when being down by 3. Even if the Vikings were able to tie up the game with a field goal and go into overtime, they'd still only have a 50% chance of winning.

NFL Team Finally Gets It

Long-time reader JTapp sent me this today. Maybe Fred Taylor and Jack Del Rio have been reading too much Taleb.


Pre-Game Coin Toss Makes Jacksonville Jaguars Realize Randomness Of Life

2nd Down and 1

Al Michaels: “First and 10 from the 30. Campbell back to pass…it’s a screen to Portis. Right sideline…a 9-yard, no make that a 10 yard gain with the spot. It’ll be 1st and 10 for the Redskins at the 40.”

John Madden: “Yeah, Al. That’s just a totally Portis thing. He just knows where the first down marker is by instinct. See, right here [during the replay], he just --BAM!—reaches across the line for the first down. It’s like he’s got radar, Al.”

Actually, in a strange way, 'Sherriff Gonna Getcha' may have just unwittingly cost his team almost a point.

Portis may have cost his team almost a point (on average) because he passed up the juiciest down-and-distance situation of all: the 2nd and 1.

It must give defensive coordinators across the league nightmares. An offense can do anything on 2nd and 1. It can run, and probably pick up the first down, but they could just as easily take a shot down the field without much risk. The QB has the luxury of a no-pressure down. He doesn't have any need to force the pass and can throw it away if needed. An incompletion still leaves a very manageable 3rd and 1. And failing that, an attempt on 4th and 1 may not be out of the question (especially if it's a short 1).

A look at the expected points for 1st and 10 situations shows there is statistical evidence that a 9 yard gain is actually preferable than a 10-yard gain. The graph below shows the average expected points for most 1st and 10 plays in the 1st half of all regular season games from 2000 to 2007. (I excluded plays inside field goal range (the 35) and plays within 2 minutes of halftime. This limits the data to normal football situations, when teams are neither desperate nor nursing big leads, and when time is not a consideration. It also removes any bias in the data due to having the option to play very conservatively inside FG range.)


We can see a fairly clear drop in expected points from a 9-yard gain to a 10-yard gain from 2.3 points to 1.6. That's about a 0.7-point drop in the average number of points scored between having a 2nd and 1, and actually getting the first down. It may not sound like a lot of points, but it's a relatively large difference for a single yard on a single play.

There is, however, some noise in the data. So how can we be sure that the sudden discontinuity between 9- and 10-yard gains isn't just a very large random blip? First, the blip is fairly large. In fact, it's the largest jump between any two yardage gains. Second, it goes in the opposite direction we'd expect it to go. Further, the graph indicates that on 1st down, a 9-yard gain is not only better than a 10-yard gain, but it's better than anything up to a 16-yard gain. It also indicates that a 2nd and 3 is notionally just as good as converting 1st and 10. Lastly, we have a good theoretical basis as to why we'd see such a result.

So am I actually suggesting that ball carriers should intentionally try for a 9-yard gain instead of try for the extra 1 or 2 yards? It might be a hard sell, but yes, the evidence is there. On the other hand, the first time anyone actually did it intentionally, and his team failed to convert the 1st down, the criticism would be merciless and it would never be done again.

However, it appears that it may already be happening, at least unintentionally. The graph below plots the frequency of each gain (or loss) from a 1st and 10. Notice the the divot at exactly 10 yards. There is an unnaturally low number of 10-yard gains compared to 9- and 11-yard gains. This could be due to how refs spot the ball or how defenses guard the 10-yard marker, but it's intriguing.


So if 2nd and 1 is really that valuable, are NFL offenses taking advantage when they get one? Are they actually taking shots down the field? After all, the advantage is only as big as offenses make it. Perhaps it could be even larger if coaches properly exploited the situation. I'll take a look at that in the next post.

Edit: I hope no one thinks I'm suggesting untouched ball carriers should spontaneously drop after 9 yards. I'm only suggesting that the outstretched arm/second effort thing can be strangely counterproductive. But mostly I'm just illustrating how the rules of football sometimes create counter-intuitive effects.

Play Calling on 3rd and Short Part 3

In this third and final installment looking at play calling on third down, I'll analyze the statistical relationship between individual team's tendencies and their conversion rate. So far, in part 1 and part 2, we've seen that running on 3rd and short tends to get more first downs and leads to scoring more points. So, are individual teams that tend to run on third and short more successful than teams that tend to pass?

A Case Study

One of the notable differences between Patriots’ coach Bill Belichick and most other coaches is his tendency to run on 3rd and short. I cannot recall where I read that, and I’ve wanted to examine 1—if its true, and 2—how it affects his team’s success on third down.

First, yes, it is true. In his tenure with the Patriots, Belichick has run 80% of the time on 3rd and 1 compared to a league-wide average of 70%. This rate makes the Pats tied for 3rd in the NFL for run tendency in that time frame. Their overall conversion rate for all 3rd down and 1 situations was 5th in the league at 72%. So Belichick’s reputation is at least partially true. His teams are near the top of the league in both categories, but not at the very top.

But what about the rest of the teams? If there is a statistical link between run tendency and conversion rate across the NFL, this would be confirmation that teams should run more often on 3rd and short.

The correlation between percentage of run attempts and conversion rate on 3rd and 1 situations is 0.19 (which is statistically significant at p<0.01, n=254.) This seems pretty big considering all the factors that go into conversion rate—team strength, opponent strength, luck. Simply calling more run plays would significantly improve most teams’ conversion rates on 3rd and short. The effect may be somewhat overstated, however. Not all 3rd and 1s are equal. Both 3rd and 1 inch, and 3rd and 1½ yards are considered ‘3rd and 1’ according to the NFL. So teams that have a disproportionate number of 3rd and 1 inch situations would logically run more often, and more successfully. But any bias in the correlation would likely be very small as there are over 28,000 3rd and 1 plays in the data set, and any effect should even out among teams. Additionally, 3rd and 2 and 3rd and 3 situations do not exhibit that complication, but the evidence that teams pass too often is just as strong if not stronger.

The total of all the evidence makes the conclusion clear. Offenses should run more often on 3rd and short. Based on conversion rate, expected points, and team-by-team correlation, running more often on third and short leads to more 1st downs, more points, and consequently more wins.

Play Calling on 3rd and Short Part 2

Are NFL coaches calling the right plays on 3rd down and short? In part 1, we saw that 3rd down conversion rates suggested teams should run more often in such situations, as long as getting a first down is the only goal. In this installment, I'll continue the analysis by considering two other factors--average yardage gained and expected points.

Adjusted Gain

Although conversion rates indicate teams pass too often, they're not the only measure of success on 3rd down. It’s always better to have a 1st down and more yards (except on 3rd and goal). So on one hand, passing seems to offer an advantage over running in that its average gain is longer. On the other hand, there is the chance of an interception when passing. To include both considerations in the analysis, I’ve calculated the “adjusted gain” of each play. Adjusted gain subtracts 45 yards for each interception. This method accounts for interception risk well because a 45 yard loss results in roughly the same change in expected scoring as a turnover. The graph below plots the distribution of adjusted gain for 3rd and 1 situations.


A simpler way of looking at each option is the average adjusted gain for each type of play. Here is a graph of average adjusted gain for 3rd down situations.


Until 3rd and 4 offenses are getting more adjusted yards by passing than by running. By 4 yards to go, adjusted gain equalizes. (Again, except for 3rd and short, it’s striking just how equal the adjusted gains are.) On 3rd and 1 the pass outgains the run by 3.3 yards per play, but the run gets a first down more often. The question now becomes, what would an offense rather have: an extra 12% probability of conversion by running, or an extra 3.3 yards on average by passing?

Expected Points

By comparing the expected points of both options, we can get an answer. A typical punt results in a loss of about 1.8 expected points. This is 12% more likely when passing than when running, which results in a 0.12 * 1.8 = 0.22 loss in expected points. An extra 3.3 yards is worth 0.10 additional expected points. The net difference is 0.12 points in favor of the run. So for 3rd and 1, the extra gain achieved by passing is not worth the added risk of the failing to convert.

The 0.10 expected points from passing’s added gain would be worth it only when running is successful 5.5% more often than passing. This would be the true Nash equilibrium for 3rd and 1.

x * 1.8 = 0.10
x = 0.10/1.8
x = 0.055 = 5.5%

But after I did that little algrebra exercise, I realized it was unnecessary because I already have the historical data for actual expected points for each situation. The graph below shows how many points, on average, a team scored following a 3rd down. Expected points after a run or pass shows that offenses pass too often on 3rd and short and run too often on 3rd and long.


On 3rd and 1, offenses scored an average of 2.38 points if they ran, and an average of 2.24 points if they passed. The difference of 0.14 points is remarkably close to the 0.12-point theoretical estimate calculated above.

The differences are even bigger for 3rd and 2 and 3rd and 3 situations. The advantage of running is 0.45 and 0.31 expected points respectively—convincing evidence that offenses should be running more often on 3rd and short. The graph also indicates passing is ultimately more fruitful on 3rd and long, despite equal conversion rates.

In the third and final part of this article, I'll look at whether teams that run more often on third and short really are more successful. I'll also quantify just how important optimum play calling can be.