Week 4 Results

The efficiency model predicted upsets by Cleveland over Baltimore, Detroit over Chicago, Buffalo over the Jets, Tampa Bay over Carolina, and Kansas City over San Diego.

Overall, the efficiency model correctly predicted winners in 9 out of 12 games. The consensus favorite, as defined by betting lines, was only 3 out of 11 as of Sunday evening. (The difference in total games is due to the GB at MIN game being a pick 'em, i.e. point spread = 0).

Game Predictions Week 4

Game probabilities for week 4 are listed below. The probabilities are based on an efficiency win model explained here. The model considers offensive and defensive efficiency stats including running, passing, sacks, turnover rates, and penalty rates. Team stats are adjusted for previous opponent strength.

Visitor	Home	Vprob	Hprob
BAL	CLE	0.48	0.52
CHI	DET	0.17	0.83
GB	MIN	0.61	0.39
HOU	ATL	0.76	0.24
NYJ	BUF	0.16	0.84
OAK	MIA	0.19	0.81
STL	DAL	0.05	0.95
SEA	SF	0.70	0.30
TB	CAR	0.78	0.22
DEN	IND	0.44	0.56
KC	SD	0.53	0.47
PIT	ARI	0.68	0.32
PHI	NYG	0.70	0.30
NE	CIN	0.86	0.14

Season Win Projections Week 3

Season win totals and division standing projections are listed below. Projections are based on each team's opponent-adjusted generic win probability (Adj GWP). Total wins account for current and projected wins. Methodology can be found here.

TEAM	GWP	Opp GWP	Adj GWP	Tot. Wins
AFC E
NE	0.97	0.17	0.80	14
MIA	0.37	0.58	0.41	5
BUF	0.08	0.94	0.30	4
NYJ	0.23	0.56	0.26	4
AFC N
PIT	0.90	0.22	0.77	13
CIN	0.52	0.38	0.46	7
BAL	0.34	0.43	0.30	6
CLE	0.19	0.55	0.22	4
AFC S
IND	0.91	0.39	0.85	15
JAX	0.63	0.60	0.68	11
TEN	0.61	0.52	0.62	10
HOU	0.54	0.61	0.59	10
AFC W
DEN	0.94	0.31	0.85	13
KC	0.46	0.38	0.40	6
SD	0.20	0.56	0.23	4
OAK	0.23	0.49	0.22	4
NFC E
DAL	0.91	0.28	0.80	14
PHI	0.80	0.51	0.80	12
WAS	0.60	0.50	0.60	10
NYG	0.34	0.70	0.44	6
NFC N
GB	0.59	0.44	0.57	10
MIN	0.49	0.35	0.41	6
DET	0.34	0.50	0.35	6
CHI	0.13	0.52	0.14	2
NFC S
TB	0.85	0.28	0.74	12
CAR	0.47	0.33	0.38	7
ATL	0.26	0.53	0.28	3
NO	0.03	0.79	0.18	2
NFC W
SEA	0.61	0.63	0.68	11
ARI	0.53	0.45	0.51	8
SF	0.40	0.54	0.42	7
STL	0.20	0.57	0.23	3

For now, the projections don't account for future strength of schedule. For example, notice how strong the AFC S appears. Obviously, they have to play each other frequently, so it would be very rare for a division to win so many games. At this point so early in the season, you should pay most attention to the relative ranking of each team rather than the projected win total.

Week 3 Efficiency Rankings

Team efficiency rankings are listed below in terms of generic winning probability. The GWP is the probability a team would beat the league average team at a neutral site. Each team's opponent's GWP is also listed, which can be considered strength of schedule. The adjusted GWP (ADJGWP) modifies the generic win probability to reflect the strength of to-date opponents. A full explanation of the methodology can be found here.

TEAM	GWP	Opp GWP	ADJGWP
IND	0.91	0.39	0.85
DEN	0.94	0.31	0.85
NE	0.97	0.17	0.80
PHI	0.80	0.51	0.80
DAL	0.91	0.28	0.80
PIT	0.90	0.22	0.77
TB	0.85	0.28	0.74
JAX	0.63	0.60	0.68
SEA	0.61	0.63	0.68
TEN	0.61	0.52	0.62
WAS	0.60	0.50	0.60
HOU	0.54	0.61	0.59
GB	0.59	0.44	0.57
ARI	0.53	0.45	0.51
CIN	0.52	0.38	0.46
NYG	0.34	0.70	0.44
SF	0.40	0.54	0.42
MIA	0.37	0.58	0.41
MIN	0.49	0.35	0.41
KC	0.46	0.38	0.40
CAR	0.47	0.33	0.38
DET	0.34	0.50	0.35
BAL	0.34	0.43	0.30
BUF	0.08	0.94	0.30
ATL	0.26	0.53	0.28
NYJ	0.23	0.56	0.26
STL	0.20	0.57	0.23
SD	0.20	0.56	0.23
OAK	0.23	0.49	0.22
CLE	0.19	0.55	0.22
NO	0.03	0.79	0.18
CHI	0.13	0.52	0.14

NFL Win Prediction Methodology

Throughout the rest of the 2007 season I intend to publish win probabilities for each game, and season win projections for each team. This post explains the methodology used to calculate these.

Based on a logit regression of every game played for the past 5 seasons, a mathematical model was established to determine the probability each opponent would win a game. The model is based on team efficiency stats which include:

• Offensive pass efficiency, including sack yardage
• Defensive pass efficency, including sack yardage
• Offensive run efficiency
• Defensive run efficiency
• Offensive interception rate
• Defensive interception rate
• Offensive fumble rate
• Penalty rate (penalty yards per play)

Home field is also included in the model. These factors were selected because they are most predictive of future performance, and not necessarily because they explain past performance. Over the past five seasons, the model predicts winners correctly in 69.8% of games (retrospectively). In 2006, the model was correct in 65% of games, well ahead of consensus favorites as determined by betting lines. Last year was particularly difficult year for prognosticators, as consensus favorites only won 57% of the games.

Touchdowns, or red zone performance, or third down success rates are not used in the model because I believe those things are the results of passing and running ability etc. To include them in a model intended for prediction would guarantee it is severely "overfit." In other words, it would capture and explain the unique qualities of past events at the expense of predictive power.

Once the model is established, each game's outcome probability can be calculated. But there are other applications. By calculating the probability a team will win against a notional league-average team at a neutral site, a generic win probability can be determined for each team.

This year the model includes an adjustment for opponent strength. This is especially important earlier in the season when there are fewer data points to establish each team's baseline performance levels. Each opponent's generic win probability is averaged for each team. It is then included back into the win model to refine each prediction. For example, a team with impressive stats against weak teams would not be favored as strongly as a team with similar stats against strong teams.

Another application of the opponent-adjusted generic win percentage is a ranking of each team. Such a ranking is similar to the now ubiquitous "power rankings." A better term for the rankings on this site would be "efficiency rankings."

Lastly, final win totals can be estimated by calculating the probabilities of a team's future games. By using the law of total probability, the probility that each possible final record will occur can be determined. For example, if a team has two games left, one with a 0.7 chance of winning and one with a 0.5 chance of winning, the probability of winning 0, 1, or 2 games can be calculated.

2 wins = 0.5 * 0.7
1 win = 0.5 * (1-0.7) + (1-0.5) * 0.7
0 wins = (1-0.7) * (1-0.5)

The same math can be applied with many more games to go but becomes far more complex. Then once we determine the most likely winning percentage for each team, we can compare those expected values to actual outcomes to determine which teams have been lucky or unlucky.

Lastly, as playoff time approaches, we can go one step further. By applying the same principal of total probability, the outcomes of playoff races can be estimated.

Note: The actual game prediction model and coefficients can be found here.

RB Wins Added 2006

Similar to my efforts to devise a better QB rating, I've applied the same method to estimating the wins contributed by running backs. Although not perfect, it provides a sense of who is helping his team and who is hurting his team, and by how much.

The components of the RB rating are weighted according to how important they are in terms of team wins. The formula is based on a multivariate regression model of team wins. Using data from the past five NFL regular seasons, the regression model estimates team wins based on the efficiency stats of each team including passing, running, turnovers, and penalties.

The rating includes Yards Per Carry (YPC), fumble rate, and an adjusted Yards After Catch per reception (YAC/Att). YAC/Rec is adjusted to reflect the fact that RB receptions constitute 14% of all team pass attempts. Fumble rate is defined as fumbles per carry plus receptions.

Some may ask why I don't include touchdowns in the rating. Touchdowns are the result of yards per carry, reception yds, etc. Including TDs would also skew the rating towards the Alstott-esque "vulture-backs." Also, rushing TDs are often the result of an excellent passing game that frequently gets the ball close to the goal line, and not necessarily the result of good rushing.

The RB rating is computed in terms of how many wins a RB contributes to his team through his level of performance over the course of a full 16-game season, all other things being equal. The equation is:

+WP16 = [(YPC * 0.92) + (YAC/Att * 1.57) - (Fum Rate * 63.2)] - 4.0

(I subtract 4.0 as the linear constant because it's the average score for RBs. An average RB on an average team would produce exactly 8 wins, not 12.)

The resulting ranking of 2006 RBs is below. Keep in mind the list assumes a full 16-game season for each RB.

Player	Team	Rush	YPC	YAC/Att	Fum	+WP16
Norwood	ATL	99	6.4	0.5	0	2.6
Jones-Drew	JAC	166	5.7	0.6	1	1.8
Tomlinson	SDG	348	5.2	0.6	2	1.3
Jones	DAL	267	4.1	1.0	1	1.1
Barber III	DAL	135	4.8	0.4	0	1.1
Westbrook	PHI	240	5.1	0.5	2	1.0
Barber	NYG	327	5.1	0.5	3	1.0
Taylor	JAC	231	5	0.7	3	0.9
Portis	WAS	127	4.1	0.7	0	0.8
Johnson	KAN	416	4.3	0.7	2	0.7
Gore	SFO	312	5.4	0.5	6	0.6
Addai	IND	226	4.8	0.5	2	0.6
Benson	CHI	157	4.1	0.6	0	0.6
Maroney	NWE	175	4.3	0.6	1	0.6
Bell	DEN	157	4.3	0.6	1	0.5
Jacobs	NYG	96	4.4	0.9	2	0.3
Williams	CAR	121	4.1	0.6	1	0.3
Dillon	NWE	199	4.1	0.7	2	0.3
Jackson	STL	346	4.4	0.5	4	0.3
Jones	CHI	296	4.1	0.4	1	0.2
Washington	NYJ	151	4.3	0.6	2	0.2
Dunn	ATL	286	4	0.5	1	0.2
Henry	TEN	270	4.5	0.4	3	0.0
Dayne	HOU	151	4.1	0.4	1	0.0
Jordan	OAK	114	3.8	0.6	1	-0.1
McAllister	NOR	244	4.3	0.4	3	-0.1
Lundy	HOU	124	3.8	0.5	1	-0.1
Betts	WAS	245	4.7	0.5	6	-0.1
Houston	NYJ	113	3.3	0.5	0	-0.3
Morency	GNB	96	4.5	0.5	2	-0.3
Green	GNB	266	4	0.6	4	-0.3
Fargas	OAK	178	3.7	0.4	1	-0.3
Brown	MIA	241	4.2	0.5	4	-0.3
McGahee	BUF	259	3.8	0.7	4	-0.3
Parker	PIT	337	4.4	0.5	7	-0.4
Bush	NOR	155	3.6	0.5	2	-0.5
Morris	SEA	161	3.8	0.3	1	-0.5
Taylor	MIN	303	4	0.4	5	-0.6
Bell	DEN	233	4.4	0.4	5	-0.6
Thomas	BUF	107	3.5	0.4	1	-0.7
Lewis	BAL	314	3.6	0.5	4	-0.8
Foster	CAR	227	4	0.3	4	-0.8
Williams	TAM	225	3.5	0.4	3	-0.9
Jones	DET	181	3.8	0.6	5	-0.9
James	ARI	337	3.4	0.3	3	-0.9
Rhodes	IND	187	3.4	0.4	3	-1.1
Johnson	CIN	341	3.8	0.3	6	-1.1
Droughns	CLE	220	3.4	0.5	5	-1.4
Barlow	NYJ	131	2.8	0.3	1	-1.5
Alexander	SEA	252	3.6	0.3	6	-1.7

QB Wins Added Week 3

The QB Wins Added Per 16 Games stat (+WP16) estimates how many wins a quarterback adds to his team's record over the course of a 16-game season. +WP16 is explained here and here, as well as in the last post. Last year's ratings can be found in the second link.

Here is the list of 2007 QBs and their vital stats through week 3.

Name	Att	PassYds	YAC	Int	Rush	RushYds	SkYds	Fum	+WP16
Brady	88	887	208	1	3	8	19	2	5.57
Romo	88	860	224	2	7	47	38	0	4.60
Manning P	101	873	251	1	1	-2	15	0	4.05
Garcia	65	595	208	0	6	12	22	0	3.38
McNabb	105	805	212	1	9	31	23	1	2.78
Palmer	125	937	208	4	6	9	34	0	2.18
Anderson	98	760	118	4	4	10	36	3	2.12
Garrard	75	630	211	0	21	99	42	2	2.06
Kitna	115	980	188	4	8	31	74	4	1.95
Hasselbeck	97	751	220	2	7	4	31	1	1.59
Harrington	96	760	193	2	4	14	74	0	1.59
Campbell	84	621	148	3	14	80	27	3	1.30
Pennington	43	291	66	0	5	2	39	0	1.06
Green	114	824	185	5	5	9	32	1	1.05
Roethlisberger	77	563	225	1	3	30	23	1	1.04
Favre	125	861	275	2	5	-3	44	1	0.75
Young	67	426	121	2	21	96	18	2	0.38
Manning E	106	755	238	4	2	1	26	2	0.13
Delhomme	86	626	280	1	6	26	46	1	-0.01
McNair	61	401	90	1	4	14	37	4	-0.15
Bulger	109	651	188	3	3	18	49	1	-0.33
Cutler	95	795	367	4	8	15	18	2	-0.63
Boller	51	287	117	1	4	9	7	0	-0.66
Smith	83	461	158	1	8	67	64	2	-0.71
Schaub	83	688	306	3	4	7	41	2	-0.77
Leinart	85	454	173	3	8	33	10	0	-1.08
Rivers	97	675	186	4	7	4	66	4	-1.23
Brees	130	677	241	7	5	23	28	2	-2.55
Huard	90	549	282	3	4	5	42	1	-2.68
Grossman	89	500	144	6	4	9	82	1	-3.37
Clemens	47	295	150	2	3	2	30	1	-3.46
Losman	47	255	128	1	6	51	67	3	-3.70
Jackson	56	329	188	5	7	26	0	0	-4.36
McCown J	68	494	205	5	9	46	55	7	-4.39

The +WP16 stat basically tells us how much a QB is hurting or helping his team. For example, Tom Brady's performance to date would take a team that is completely average in every other way to between a 13 and 14 win season (8 + 5.57 = 13.57).

How Bad is Rex Grossman?

We keep hearing how Rex Grossman costs the Bears wins. Is that true, and if so, how many wins?

Sure, Grossman's NFL passer rating is the lowest of all starting QBs. He has 1 TD and 6 INTs in 3 games. Clearly that's not good. In this post, I'll estimate how many wins he would cost his team if he were allowed to continue playing at his present level of performance.

The estimate is based on an improved passer rating formula. The components of the new rating are weighted according to how important they are in terms of team wins. The formula is based on a multivariate regression model of team wins. Using data from the past five NFL regular seasons, the regression model estimates team wins based on the efficiency stats of each team including passing, running, turnovers, and penalties. Regression models can hold all other factors equal, so by only adjusting the factors of interest (QB passing, rushing and turnover performance) we can calculate the effect of QB performance on the estimate of season wins.

Further, the rating excludes receiver Yards After Catch (YAC) and relies only on QB Air Yards. It also includes rushing yards, sack yards, interceptions, and fumbles. All factors are computed on a "per attempt" basis. The resulting formula computes the wins added by a QB's performance per a 16-game season, "+WP16" (apologies to David Berri). The equation is:

+WP16 = [(Total Yds/Total Plays * 1.57) - (Total Turnovers/Total Plays *50.5)] - 4.5

Total yards includes air yards, rushing yards, and sack yards. Total plays includes pass attempts, rushes, and sacks. Turnovers include interceptions and half of all fumbles (lost or not). The equation subtracts 4.5 because that is the average +WP16 score for QBs in 2007 so far. An average QB on an average team would win 8 games, not 12.5.

So for Rex Grossman:

+WP16 = [(283 Yds/102 plays * 1.57) - (6.5 TOs/102 plays * 50.5) - 4.5
= -3.4 wins

So holding all other factors equal, Rex Grossman would cost his team 3.4 losses over the course of a full season. Last year Grossman was in the middle of the pack with +0.3 wins added. He did what the Bears asked of him, which was to break even and allow the defense and special teams to make the difference. This year, not so much.

Improved FG Kicker Ranking

I previously looked at FG kicker performance. My evaluation of each kicker was based on how much better the kicker did than would be expected given how distant his attempts were. However, there were several flaws in the original analyisis, which I've hopefully corrected here.

1. Vandergajt's perfect 2003 season was an outlier that distorts the data greatly. Technically, the regression treated his average miss distance as zero, which made his expected accuracy very low. I decided to exclude this one season in the regression.

2. I realized that including average miss and made distances in calculating expected FG% severely overfits the data. Average miss and made distances are affected directly by missed and made field goals, which is essentially what FG% is. Although those variables help describe the distribution of FG attempt distances, they hurt more than help the model. I excluded them.

3. It was pointed out that many of the top kicker seasons belonged to warm weather and indoor kickers, so the model needs to account for home field environment. I included dummy variables for indoor stadia and warm weather teams.

4. It was also suggested that I should use a non-linear relationship between average attempt distance and FG%. For example, a 50-yd FG attempt is more than twice as difficult as a 25-yd attempt. The best fit turned out to be logarithmic, so I used ln(FG%) as the dependent variable in the regression. In case anyone is curious, the graph below illustrates the average FG% for kicks from various distances.


Below are the results of the regression (n=122). About 8% of the variance in FG kicker seasonal performance can be accounted for by situational variables: average attempt distance, warm weather, and a domed home stadium.

VARIABLE	COEFFICIENT	P-VALUE
Indoor	0.008	0.726
Warm Wx	0.017	0.442
Avg Att Dist	-0.014	0.004
r-squared	0.077

Indoor home stadiums and warm weather kickers do not appear to enjoy a significant advantage throughout a season. Neither variable was close to significance. I got the same results for Indoor and Warm Wx regardless of various combinations of independent and dependent variables. But I left both variables in the final regression model. It's a general rule to keep insignificant variables in a model if their coefficients have the expected sign and they would intuitively help account for factors that are not the primary focus of the model. Although the coefficients may be less than perfectly accurate, they help get you closer to the truth than farther from it.

The table below lists each FG kicker's performance from the 2003-06 seasons. The list is sorted from best to worst in terms of exceeding the expected FG accuracy given each kicker's average attempt distance and home stadium environment.

Click on the table headers to sort.

Year	Player	Team	Avg Yds Att	Act FG%	Exp FG%	Act-Exp
2003	Hanson	Det.	38.1	0.96	0.78	0.18
2005	Rackers	Ariz.	38.1	0.95	0.78	0.17
2004	Hanson	Det.	38.1	0.96	0.79	0.16
2003	Vanderjagt	Ind.	33.8	1.00	0.84	0.16
2005	Nedney	S.F.	38.5	0.93	0.77	0.15
2003	Wilkins	St.L.	35.9	0.93	0.80	0.13
2004	Wilkins	St.L.	35.9	0.93	0.80	0.13
2004	Vinatieri	N.E.	34.9	0.94	0.81	0.13
2006	Kasay	Car.	38.9	0.89	0.77	0.12
2006	Lindell	Buff.	35.9	0.92	0.80	0.12
2003	Janikowski	Oak.	39.0	0.88	0.78	0.10
2005	Wilkins	St.L.	39.1	0.87	0.77	0.10
2006	Stover	Balt.	33.2	0.93	0.83	0.10
2006	Gould	Chi.	36.8	0.89	0.79	0.10
2004	Janikowski	Oak.	36.3	0.89	0.80	0.09
2004	Stover	Balt.	34.8	0.91	0.81	0.09
2006	Elam	Den.	33.3	0.93	0.85	0.09
2003	Graham	Cin.	37.2	0.88	0.79	0.09
2006	Vinatieri	Ind.	36.6	0.89	0.81	0.08
2006	Hanson	Det.	36.3	0.88	0.80	0.08
2006	Kaeding	S.D.	35.8	0.90	0.82	0.08
2003	Longwell	G.B.	36.2	0.89	0.81	0.08
2004	Longwell	G.B.	36.2	0.89	0.81	0.08
2005	Kaeding	S.D.	36.4	0.88	0.80	0.08
2005	Vanderjagt	Ind.	32.8	0.92	0.84	0.08
2003	Anderson	Ten.	36.5	0.87	0.80	0.08
2005	Dawson	Clev.	31.6	0.93	0.86	0.07
2005	Stover	Balt.	35.9	0.88	0.81	0.07
2003	Elam	Den.	36.2	0.87	0.80	0.07
2005	Peterson	Atl.	31.6	0.92	0.85	0.07
2004	Graham	Cin.	37.2	0.87	0.80	0.07
2006	Wilkins	St.L.	36.8	0.87	0.80	0.07
2006	Carney	N.O.	32.4	0.92	0.86	0.06
2006	Nugent	N.Y.J.	33.8	0.89	0.83	0.06
2005	Graham	Cin.	34.0	0.88	0.82	0.05
2005	M. Bryant	T.B.	38.4	0.84	0.79	0.05
2003	Kasay	Car.	36.9	0.84	0.79	0.05
2006	Andersen	Atl.	34.3	0.87	0.82	0.05
2004	Elam	Den.	36.1	0.85	0.81	0.05
2004	Kasay	Car.	36.9	0.84	0.80	0.04
2003	Brien	N.Y.J.	37.3	0.84	0.80	0.04
2003	Stover	Balt.	34.7	0.87	0.83	0.04
2003	K. Brown	Hou.	38.2	0.82	0.78	0.03
2006	Feely	N.Y.G.	34.4	0.85	0.82	0.03
2003	Andersen	K.C.	39.1	0.80	0.77	0.03
2003	Akers	Phil.	36.4	0.83	0.80	0.03
2004	Akers	Phil.	36.4	0.83	0.80	0.03
2006	Scobee	Jax.	39.0	0.81	0.78	0.03
2006	Graham	Cin.	36.9	0.83	0.81	0.03
2004	Reed	Pitt.	34.0	0.85	0.82	0.03
2005	Feely	N.Y.G.	36.4	0.83	0.81	0.02
2005	Lindell	Buff.	35.3	0.83	0.81	0.02
2004	Brien	N.Y.J.	35.9	0.83	0.81	0.02
2005	Mare	Mia.	34.8	0.83	0.81	0.02
2003	P. Dawson	Clev.	33.6	0.86	0.84	0.01
2005	Tynes	K.C.	35.4	0.82	0.81	0.01
2006	Nedney	S.F.	34.3	0.83	0.82	0.01
2006	Brown	Sea.	36.2	0.81	0.80	0.01
2005	Bironas	Ten.	37.4	0.79	0.79	0.01
2005	Reed	Pitt.	35.4	0.83	0.82	0.01
2004	Dawson	Clev.	34.7	0.83	0.82	0.01
2006	Longwell	Minn.	33.6	0.84	0.83	0.01
2005	Vinatieri	N.E.	36.4	0.80	0.80	0.00
2005	Kasay	Car.	38.9	0.77	0.77	-0.01
2003	Cundiff	Dall.	36.0	0.79	0.80	-0.01
2004	Cundiff	Dall.	36.0	0.79	0.80	-0.01
2003	Hall	Wash.	38.8	0.76	0.77	-0.01
2006	Tynes	K.C.	36.9	0.77	0.79	-0.02
2006	K. Brown	Hou.	39.4	0.76	0.78	-0.02
2005	Nugent	N.Y.J.	35.7	0.79	0.80	-0.02
2005	Hanson	Det.	35.7	0.79	0.81	-0.02
2004	Mare	Mia.	39.0	0.75	0.77	-0.02
2004	Hall	Wash.	38.8	0.76	0.78	-0.02
2004	Vanderjagt	Ind.	35.5	0.80	0.82	-0.02
2004	Kaeding	S.D.	35.3	0.80	0.82	-0.02
2004	Anderson	Ten.	37.6	0.77	0.80	-0.02
2004	Lindell	Buff.	29.5	0.86	0.88	-0.03
2005	J. Brown	Sea.	41.3	0.72	0.75	-0.03
2004	J. Brown	Sea.	39.7	0.73	0.76	-0.03
2006	Bironas	Ten.	34.4	0.79	0.82	-0.03
2006	Rackers	Ariz.	37.6	0.76	0.79	-0.03
2005	Gould	Chi.	35.0	0.78	0.81	-0.03
2006	Bryant	T.B.	36.7	0.77	0.81	-0.03
2004	Scobee	Jax.	35.2	0.77	0.81	-0.04
2005	Akers	Phil.	39.6	0.73	0.76	-0.04
2006	Akers	Phil.	34.3	0.78	0.82	-0.04
2003	Mare	Mia.	37.4	0.76	0.80	-0.04
2004	Tynes	K.C.	37.9	0.74	0.78	-0.04
2003	J. Brown	Sea.	39.7	0.73	0.78	-0.04
2003	Carney	N.O.	38.4	0.73	0.78	-0.04
2004	Carney	N.O.	38.4	0.73	0.78	-0.04
2005	Elam	Den.	38.0	0.75	0.79	-0.04
2005	Scobee	Jax.	36.4	0.77	0.81	-0.04
2003	Christie	S.D.	35.9	0.75	0.80	-0.05
2003	Edinger	Chi.	38.4	0.72	0.78	-0.05
2004	Edinger	Chi.	38.4	0.72	0.78	-0.05
2005	Carney	N.O.	33.6	0.78	0.83	-0.05
2005	K. Brown	Hou.	35.6	0.77	0.82	-0.05
2005	Longwell	G.B.	37.7	0.74	0.80	-0.06
2006	Janikowski	Oak.	38.9	0.72	0.78	-0.06
2005	Edinger	Minn.	36.5	0.74	0.80	-0.06
2006	Rayner	G.B.	35.7	0.74	0.80	-0.06
2003	Conway	Clev.	36.8	0.74	0.80	-0.06
2006	Reed	Pitt.	35.6	0.74	0.81	-0.06
2006	Mare	Mia.	37.1	0.72	0.79	-0.07
2006	Dawson	Clev.	36.7	0.72	0.79	-0.07
2003	Elling	Minn.	36.8	0.72	0.79	-0.07
2004	Elling	Minn.	36.8	0.72	0.79	-0.07
2003	Feely	Atl.	38.1	0.70	0.78	-0.07
2004	Feely	Atl.	38.1	0.70	0.78	-0.07
2004	K. Brown	Hou.	37.5	0.71	0.79	-0.08
2006	Gostkowski	N.E.	32.7	0.77	0.85	-0.08
2005	Cortez	Ind.	35.3	0.71	0.81	-0.10
2003	Vinatieri	N.E.	33.0	0.74	0.84	-0.11
2003	Reed	Pitt.	34.4	0.72	0.83	-0.11
2006	Vanderjagt	Dall.	34.0	0.72	0.84	-0.12
2005	Janikowski	Oak.	37.6	0.67	0.78	-0.12
2003	Lindell	Buff.	33.9	0.71	0.83	-0.12
2003	Gramatica	T.B.	37.2	0.62	0.79	-0.17
2004	Gramatica	T.B.	37.2	0.62	0.79	-0.17
2003	Marler	Jax.	35.9	0.61	0.81	-0.20
2004	Gramatica	Ind.	34.6	0.58	0.82	-0.24

One thing that immediately stands out to me is how consistent some kickers are. Notice how many times some kickers' seasons are stacked together in order--Wilkins, Longwell, Akers (twice), Cundiff, Carney, J. Brown, and Edinger. Some kickers are incredibly inconsistent--Janikowski and Vanderjagt in particular.

The bottom line in this analysis is that, accounting for attempt distance and home stadium environment, the standard deviation of adjusted accuracy (Actual - Expected FG%) is 7.7%. A FG kicker one standard deviation above the mean kicks 7.7% more accurately than expected given his attempt distance. The average number of FG attempts for a team is 29, so an extra SD of accuracy would yield an additional 2.33 field goals worth 6.7 points in a season.

A rough estimate of the spread between best and worst FG kicker each year is about 34% of accuracy. That would equate to a difference of 9.9 field goals worth 29.6 points. This makes sense because it represents roughly a 4-standard deviation spread between best and worst.

The kickers who had enough qualifying attempts over the 2003-2006 seasons are ranked below in terms of how far they exceeded their expected FG% given their average attempt distance. Vanderjagt was manually assigned the top score for 2003 for his cumulative ranking. The number of years each kicker qualified is also listed. Take the guys with 1 or 2 years of kicks with a grain of salt.

Rank	Player	Years	Avg Att	Exp FG%	Act FG%	Act-Exp
1	Wilkins	4	36.9	0.79	0.90	0.105
2	Hanson	4	37.1	0.80	0.90	0.101
3	Nedney	2	36.4	0.80	0.88	0.082
4	Stover	4	34.7	0.82	0.90	0.076
5	Rackers	2	37.9	0.78	0.85	0.070
6	Peterson	1	31.6	0.85	0.92	0.069
7	Graham	4	36.3	0.81	0.86	0.058
8	Kasay	4	37.9	0.78	0.83	0.052
9	M. Bryant	1	38.4	0.79	0.84	0.051
10	Kaeding	3	35.8	0.81	0.86	0.045
11	Andersen	2	36.7	0.79	0.84	0.041
12	Elam	4	35.9	0.81	0.85	0.040
13	Gould	2	35.9	0.80	0.83	0.031
14	Brien	2	36.6	0.81	0.84	0.031
15	Vanderjagt*	4	34.1	0.83	0.81	0.030
16	Longwell	4	35.9	0.81	0.84	0.027
17	Anderson	2	37.1	0.80	0.82	0.025
18	Vinatieri	4	35.2	0.82	0.84	0.025
19	Nugent	2	34.8	0.82	0.84	0.022
20	P. Dawson	1	33.6	0.84	0.86	0.015
21	Brown	1	36.2	0.80	0.81	0.008
22	Janikowski	4	38.0	0.78	0.79	0.006
23	Dawson	3	34.3	0.82	0.83	0.003
24	Akers	4	36.7	0.79	0.79	-0.003
25	Lindell	4	33.7	0.83	0.83	-0.003
26	Cundiff	2	36.0	0.80	0.79	-0.008
27	Bironas	2	35.9	0.80	0.79	-0.013
28	Tynes	3	36.7	0.79	0.78	-0.016
29	Scobee	3	36.9	0.80	0.78	-0.017
30	Hall	2	38.8	0.77	0.76	-0.017
31	Carney	4	35.7	0.81	0.79	-0.019
32	Feely	4	36.8	0.80	0.77	-0.024
33	Mare	4	37.1	0.79	0.77	-0.027
34	K. Brown	4	37.7	0.79	0.76	-0.029
35	J. Brown	3	40.2	0.76	0.73	-0.032
36	Bryant	1	36.7	0.81	0.77	-0.035
37	Reed	4	34.9	0.82	0.78	-0.037
38	Christie	1	35.9	0.80	0.75	-0.052
39	Edinger	3	37.8	0.78	0.73	-0.056
40	Rayner	1	35.7	0.80	0.74	-0.062
41	Conway	1	36.8	0.80	0.74	-0.062
42	Elling	2	36.8	0.79	0.72	-0.073
43	Gostkowski	1	32.7	0.85	0.77	-0.084
44	Cortez	1	35.3	0.81	0.71	-0.103
45	Gramatica	3	36.3	0.80	0.60	-0.195
46	Marler	1	35.9	0.81	0.61	-0.203

FG Kickers Follow-up

I just redid the regression. I probably don't have time to post everything tonight, but here are a few observations.

1. The perfect Vanderjagt year really threw off the data. His zero miss distance skewed the entire model. I excluded his '03 season for the redo of the regression.

2. By using a natural log of the accuracy as the dependent variable, the relationship between kick accuracy and attempt distance is more linear.

3. But by excluding Vanderjagt's perfect season and by using the ln(accuracy) model, the results are very different. The difference between the best and worst kickers may be several times larger than I previously estimated. The relative rankings among the kickers are not drastically different however.

4. Surprisingly, neither warm weather or indoor kickers were significant factors.

More to follow.

The Best FG Kickers

Note: This analysis has been updated. New post can be found here.

Judging who the best field goal kickers are in the NFL is difficult. The kickers with the best range are often sent out to attempt impossibly long field goal tries. Some kickers benefit from circumstance in some years because their team's drives stall closer to the endzone than others.

Using data from the 2003-2006 regular seasons, obtained at profootballweekly.com, I may have a stumbled on a novel approach to solve the problem. My intent was to follow up on an earlier article that suggested that field goal kickers were severely underpaid relative to their impact on winning. A couple of readers (correctly) pointed out that some kickers' performances appear inflated due to luck, so the "best" kicker one year may really only be an average kicker who got lucky. In attempting to isolate the true talent level from other circumstances, including luck, involved in kicking performance, I developed the following approach.

For each kicker from '03-'06, I computed an expected percentage of FGs made based on three kicker stats:

1. Average FG attempt distance
2. Average FG made distance
3. Average FG missed distance

To compute the expected FG%, I ran a regression of actual FG percentages based on those three variables. The fitted values of the regression model became expected FG%. In simple terms, it's the average FG% that an NFL kicker would be expected to have given his average attempt, made, and miss distances. Essentially, this establishes a level-of-difficulty score, much like in diving, for each kicker's season.

The difference between each kicker's actual FG% and his expected FG% can therefore be considered a true measure of a kicker's performance in a season, accounting for attempt distances. This method does not yet isolate how much of a kicker's performance was due to luck, but it is the first necessary step to do so.

Here are the best kicking performances of the 2003-2006 period, accounting for attempt distances:

Year	Player	Team	Avg Att	Avg Made	Avg Miss	Actual %	Expected %	Act-Exp %
2003	Vanderjagt*	Ind.	33.8	33.8	NA	1.00	0.71	0.287
2004	Hanson	Det.	38.1	37.9	43.0	0.96	0.88	0.077
2003	Hanson	Det.	38.1	37.9	43.0	0.96	0.88	0.077
2006	Kasay	Car.	38.9	36.8	56.0	0.89	0.83	0.063
2005	Kasay	Car.	38.9	35.2	50.8	0.77	0.70	0.062
2005	Rackers	Ariz.	38.1	37.6	48.5	0.95	0.89	0.061
2005	Nedney	S.F.	38.5	38.0	45.5	0.93	0.87	0.058
2003	Janikowski	Oak.	39.0	37.3	51.3	0.88	0.82	0.056
2006	Longwell	Minn.	33.6	30.4	50.0	0.84	0.78	0.055
2003	Graham	Cin.	37.2	35.2	51.3	0.88	0.83	0.055
2003	K. Brown	Hou.	38.2	35.6	49.8	0.82	0.77	0.047
2003	Brien	N.Y.J.	37.3	35.0	49.6	0.84	0.80	0.047
2006	Hanson	Det.	36.3	34.4	50.0	0.88	0.83	0.045
2005	Hanson	Det.	35.7	32.4	48.2	0.79	0.75	0.045
2006	Rackers	Ariz.	37.6	34.0	48.9	0.76	0.71	0.044
2004	Wilkins	St.L.	35.9	35.5	41.7	0.93	0.88	0.044
2003	Wilkins	St.L.	35.9	35.5	41.7	0.93	0.88	0.044
2006	Lindell	Buff.	35.9	35.3	42.5	0.92	0.88	0.043
2006	Stover	Balt.	33.2	33.0	35.5	0.93	0.89	0.040
2006	Vinatieri	Ind.	36.6	35.5	45.3	0.89	0.85	0.039
2004	Vinatieri	N.E.	34.9	34.0	48.5	0.94	0.90	0.038
2006	Graham	Cin.	36.9	34.6	48.4	0.83	0.80	0.038
2003	Longwell	G.B.	36.2	35.0	44.7	0.89	0.85	0.036
2004	Longwell	G.B.	36.2	35.0	44.7	0.89	0.85	0.036
2005	Bironas	Ten.	37.4	34.6	48.3	0.79	0.76	0.033
2004	Janikowski	Oak.	36.3	35.5	43.0	0.89	0.86	0.030
2003	Stover	Balt.	34.7	32.8	47.6	0.87	0.84	0.029
2003	Lindell	Buff.	33.9	29.5	44.4	0.71	0.68	0.027
2004	Stover	Balt.	34.8	34.1	41.7	0.91	0.88	0.027
2006	Elam	Den.	33.3	32.6	43.5	0.93	0.90	0.027
2006	Kaeding	S.D.	35.8	35.2	41.0	0.90	0.87	0.027
2003	Andersen	K.C.	39.1	36.9	47.5	0.80	0.77	0.027
2005	Wilkins	St.L.	39.1	38.4	44.0	0.87	0.84	0.026
2005	Vanderjagt	Ind.	32.8	32.2	39.5	0.92	0.89	0.026
2003	Akers	Phil.	36.4	34.3	46.4	0.83	0.80	0.026
2004	Akers	Phil.	36.4	34.3	46.4	0.83	0.80	0.026
2006	Wilkins	St.L.	36.8	35.5	45.2	0.87	0.84	0.025
2005	Reed	Pitt.	35.4	33.1	46.8	0.83	0.80	0.025
2005	Stover	Balt.	35.9	34.9	43.3	0.88	0.86	0.025
2003	Anderson	Ten.	36.5	35.3	44.8	0.87	0.85	0.024
2006	Gould	Chi.	36.8	36.2	42.0	0.89	0.86	0.024
2004	Graham	Cin.	37.2	36.3	43.0	0.87	0.85	0.023
2005	Dawson	Clev.	31.6	31.3	36.5	0.93	0.91	0.022
2005	Vinatieri	N.E.	36.4	33.9	46.4	0.80	0.78	0.022
2004	Kasay	Car.	36.9	35.3	45.3	0.84	0.82	0.021
2003	Kasay	Car.	36.9	35.3	45.3	0.84	0.82	0.021
2005	Kaeding	S.D.	36.4	35.5	42.7	0.88	0.85	0.020
2006	Carney	N.O.	32.4	32.0	37.5	0.92	0.90	0.020
2005	Peterson	Atl.	31.6	31.0	38.5	0.92	0.90	0.018
2004	Elam	Den.	36.1	34.7	44.0	0.85	0.83	0.018
2003	Feely	Atl.	38.1	34.3	47.1	0.70	0.69	0.018
2004	Feely	Atl.	38.1	34.3	47.1	0.70	0.69	0.018
2004	Gramatica	T.B.	37.2	31.9	45.6	0.62	0.60	0.017
2003	Gramatica	T.B.	37.2	31.9	45.6	0.62	0.60	0.017
2003	Elam	Den.	36.2	35.4	41.5	0.87	0.86	0.014
2003	Christie	S.D.	35.9	32.7	45.4	0.75	0.74	0.014
2005	Graham	Cin.	34.0	32.8	43.0	0.88	0.86	0.011
2006	Feely	N.Y.G.	34.4	32.9	43.0	0.85	0.84	0.010
2005	Akers	Phil.	39.6	36.5	48.0	0.73	0.72	0.010
2006	Nugent	N.Y.J.	33.8	33.2	38.7	0.89	0.88	0.010
2003	Carney	N.O.	38.4	35.3	47.0	0.73	0.72	0.009
2004	Carney	N.O.	38.4	35.3	47.0	0.73	0.72	0.009
2006	Andersen	Atl.	34.3	33.4	40.0	0.87	0.86	0.007
2003	P. Dawson	Clev.	33.6	32.1	43.0	0.86	0.85	0.007
2003	Cundiff	Dall.	36.0	33.7	45.0	0.79	0.79	0.006
2004	Cundiff	Dall.	36.0	33.7	45.0	0.79	0.79	0.006
2005	Gould	Chi.	35.0	32.3	44.7	0.78	0.77	0.006
2006	Bryant	T.B.	36.7	34.2	45.2	0.77	0.77	0.004
2005	Elam	Den.	38.0	35.3	46.0	0.75	0.75	0.003
2004	Vanderjagt	Ind.	35.5	33.4	44.2	0.80	0.80	0.000
2005	Tynes	K.C.	35.4	33.7	43.0	0.82	0.82	-0.001
2005	Nugent	N.Y.J.	35.7	33.5	43.8	0.79	0.79	-0.004
2006	Scobee	Jax.	39.0	37.8	44.2	0.81	0.82	-0.004
2005	Janikowski	Oak.	37.6	33.6	45.6	0.67	0.67	-0.005
2005	Scobee	Jax.	36.4	34.0	44.0	0.77	0.77	-0.005
2003	J. Brown	Sea.	39.7	37.1	46.8	0.73	0.74	-0.006
2004	J. Brown	Sea.	39.7	37.1	46.8	0.73	0.74	-0.006
2004	Tynes	K.C.	37.9	35.2	45.5	0.74	0.75	-0.007
2004	Hall	Wash.	38.8	36.6	45.5	0.76	0.77	-0.008
2003	Hall	Wash.	38.8	36.6	45.5	0.76	0.77	-0.008
2004	Reed	Pitt.	34.0	33.1	38.8	0.85	0.86	-0.011
2004	Scobee	Jax.	35.2	32.9	43.1	0.77	0.79	-0.012
2006	Mare	Mia.	37.1	34.1	45.1	0.72	0.73	-0.012
2003	Elling	Minn.	36.8	33.8	44.4	0.72	0.73	-0.014
2004	Elling	Minn.	36.8	33.8	44.4	0.72	0.73	-0.014
2004	Lindell	Buff.	29.5	28.1	37.8	0.86	0.87	-0.015
2004	Dawson	Clev.	34.7	33.5	40.6	0.83	0.84	-0.016
2006	Nedney	S.F.	34.3	33.2	39.5	0.83	0.85	-0.019
2005	Feely	N.Y.G.	36.4	35.7	40.0	0.83	0.85	-0.020
2006	Brown	Sea.	36.2	34.9	41.8	0.81	0.83	-0.021
2006	Tynes	K.C.	36.9	35.1	43.1	0.77	0.80	-0.024
2006	Bironas	Ten.	34.4	32.5	41.7	0.79	0.81	-0.025
2005	M. Bryant	T.B.	38.4	38.4	38.5	0.84	0.87	-0.026
2005	J. Brown	Sea.	41.3	39.1	46.9	0.72	0.75	-0.027
2003	Mare	Mia.	37.4	35.5	43.1	0.76	0.79	-0.027
2004	Kaeding	S.D.	35.3	34.0	40.6	0.80	0.83	-0.032
2005	Lindell	Buff.	35.3	34.7	38.3	0.83	0.86	-0.032
2006	K. Brown	Hou.	39.4	37.9	44.0	0.76	0.79	-0.034
2004	K. Brown	Hou.	37.5	34.9	43.7	0.71	0.75	-0.039
2004	Brien	N.Y.J.	35.9	35.5	38.2	0.83	0.87	-0.039
2003	Vinatieri	N.E.	33.0	30.4	40.2	0.74	0.78	-0.041
2005	Mare	Mia.	34.8	34.6	35.8	0.83	0.88	-0.045
2006	Janikowski	Oak.	38.9	36.8	44.4	0.72	0.77	-0.045
2003	Conway	Clev.	36.8	34.9	42.0	0.74	0.79	-0.050
2006	Gostkowski	N.E.	32.7	30.9	38.7	0.77	0.82	-0.050
2006	Akers	Phil.	34.3	33.0	39.0	0.78	0.83	-0.051
2005	Carney	N.O.	33.6	32.2	38.4	0.78	0.83	-0.051
2005	K. Brown	Hou.	35.6	34.1	40.6	0.77	0.82	-0.051
2006	Rayner	G.B.	35.7	33.9	40.9	0.74	0.80	-0.056
2004	Anderson	Ten.	37.6	36.7	40.6	0.77	0.83	-0.058
2006	Vanderjagt	Dall.	34.0	31.8	39.6	0.72	0.79	-0.064
2006	Dawson	Clev.	36.7	34.9	41.5	0.72	0.79	-0.067
2005	Cortez	Ind.	35.3	33.3	40.2	0.71	0.79	-0.082
2005	Longwell	G.B.	37.7	36.8	40.4	0.74	0.83	-0.088
2005	Edinger	Minn.	36.5	35.5	39.4	0.74	0.83	-0.096
2004	Edinger	Chi.	38.4	37.7	40.3	0.72	0.83	-0.111
2003	Edinger	Chi.	38.4	37.7	40.3	0.72	0.83	-0.111
2006	Reed	Pitt.	35.6	35.0	37.4	0.74	0.85	-0.113
2004	Mare	Mia.	39.0	39.1	38.8	0.75	0.87	-0.117
2004	Gramatica	Ind.	34.6	31.1	39.5	0.58	0.70	-0.122
2003	Reed	Pitt.	34.4	33.8	36.1	0.72	0.86	-0.141
2003	Marler	Jax.	35.9	33.5	39.5	0.61	0.75	-0.148

Here is a list ranking the kickers from best to worst based on their multi-year performance during the same period. The number of seasons in which each kicker qualified is also listed. (* Vanderjagt's perfect year skews his results strongly. His average miss distance in 2003 was theoretically infinite! Giving him a realistic yet excellent score (+0.10) for '03 would place him between Stover and Nedney.)

Kicker	% Act-Exp	Years
Vanderjagt*	0.062	4
Hanson	0.061	4
Rackers	0.053	2
Kasay	0.042	4
Wilkins	0.035	4
Graham	0.032	4
Stover	0.030	4
Nedney	0.019	2
Peterson	0.018	1
Andersen	0.017	2
Elam	0.016	4
Gould	0.015	2
Vinatieri	0.014	4
Christie	0.014	1
Longwell	0.010	4
Janikowski	0.009	4
P. Dawson	0.007	1
Feely	0.007	4
Cundiff	0.006	2
Lindell	0.006	4
Kaeding	0.005	3
Bironas	0.004	2
Bryant	0.004	1
Brien	0.004	2
Akers	0.003	4
Nugent	0.003	2
Carney	-0.003	4
Scobee	-0.007	3
Hall	-0.008	2
Tynes	-0.010	3
J. Brown	-0.013	3
Elling	-0.014	2
Anderson	-0.017	2
K. Brown	-0.019	4
Dawson	-0.021	3
Brown	-0.021	1
M. Bryant	-0.026	1
Gramatica	-0.029	3
Conway	-0.050	1
Gostkowski	-0.050	1
Mare	-0.050	4
Rayner	-0.056	1
Reed	-0.060	4
Cortez	-0.082	1
Edinger	-0.106	3
Marler	-0.148	1

Acounting for Vanderjagt's perfect year, Jason Hanson comes out on top. But to put things in perspective, a +6% accuracy rate above average equates to about 1.75 extra FGs made per season.

Belichick Cheating Evidence?

One of the more apparent signs that a spy or corrupt official is cheating is that he is living a lifestyle beyond his means.

If Belichick's Patriots exploited unfair advantages in stealing signs from opposing sidelines we would expect to see some sort of evidence that they won games "beyond their means." By means I am referring to the Patriots' passing and running performance on offense and defense.

By successfully exploiting stolen signs, we might expect the Patriots to choose to use that advantage on critical plays--3rd downs in the 4th quarter for example. These critical plays would heavily "leverage" performance on the field to be converted into wins. In other words, the Patriots would win more games than their on field stats would indicate.

This is exactly what we see in the data. Year-in and year-out, Belichick's Patriots have won about 2 more games than expected given their offensive and defensive efficiencies, including turnovers and penalties. No other modern team has even come close to the Patriots in consistently winning more games than their stats indicate. Could those extra wins be due to cheating?


For a comparison of other teams' actual/expected wins charts, see this article. When I first discovered this pattern, I believed it was evidence of Belichick's in-game "genius." He was known for some unconventional tactics, such as going for it on 4th downs more often than his counterparts. (I'd go for it on 4th down too if I knew the play the defense would run.)

In a previous article, I ranked every head coach since 1983 in terms of how many excess wins they had above their expected wins based on team efficiency. Belichick's career, as a whole, was rather average due to his poor results in Cleveland. But by isolating his tenure with New England, his excess wins per season would rank him as the best tactical coach ever, with an extra +2.33 wins per season. The rest of the pack is far behind with the 2nd best coach at +1.83 wins per season. Belichick is a true outlier, at almost 3 standard deviations above the mean.

This is only circumstantial evidence of cheating, but it is evidence. And although hardly damning, we can be sure of one thing about Belichick--he is willing to cheat. If someone has crossed the line by breaking one rule, what makes you think he's not willing to break others?

On some other sites I've read some emotional comments rationalizing the Patriots' cheating. Whether other teams have attempted this or not is not relevant. Patriots fans and Belichick fans must accept the possibility that much of their success has been due to cheating. What I've done here has added weight to that possibility and quantified the potential scope of this scandal. At the very least, we would be justified in continuing to investigate the Patriots' methods.

Note: Some other examples of teams' expected vs. actual wins can be found here. Also, since this post is appearing on many other sites recently, I've posted a response to many of the excellent comments and criticisms over the past several months.

Patriots Sign Stealing

Is there evidence that the New England Patriots benefitted on the field from videotaping their opposition's defensive signals? One indication would be their success in rematch games. If the Patriots have been exploiting signal stealing regularly in past years, we would expect them to have an advantage against teams they play more than once in a season. The Patriots would be expected to score more points in the second game against a given opponent.

Here are the games in which the Patriots played the same opponent twice in a season for the past four years. Each year they play their three division rivals twice, plus they happened to play one of their playoff opponents during the regular season. The points scored by the Patriots in the first meeting and the rematch are listed.

2006	Points Scored
Opponent	1st Game	2nd Game
NYJ	24	14
MIA	20	0
IND	20	34
BUF	19	28
Total	83	76

2005	Points Scored
Opponent	1st Game	2nd Game
BUF	21	35
DEN	20	23
MIA	23	26
NYJ	16	31
Total	80	115

2004	Points Scored
Opponent	1st Game	2nd Game
IND	27	20
BUF	31	29
MIA	24	28
NYJ	13	23
PIT	20	27
Total	115	127

2003	Points Scored
Opponent	1st Game	2nd Game
BUF	0	31
NYJ	23	21
IND	38	17
MIA	19	12
Total	80	81

Only 2005 shows any real difference in points scored between the first and second games against the same opponent. Even so, we'd have to show that the 8.3 point/game difference in 2005 is significantly outside the normal variation between first and second games between all teams during the same period.

This result does not indicate the Patriots were not cheating by videotaping signals, but it does show they did not benefit signifcantly in points scored during rematch games in recent years.

Are Kickers Underpaid?

Yes, they are. By a lot.

The place kicker is a unique player because his impact on the game is solitary and direct. A kicker's performance, accounting for field goal distance and attempts, is independent from the performance of the rest of his team. No other position in football is like that, even the punter.

Recently I calculated the weight of each phase of the game in terms of team wins. The results are listed below.

Variable	Importance
O PASS	19%
D PASS	12%
O RUN	7%
D RUN	7%
O INT	9%
D INT	9%
O FUM	5%
D FFUM	5%
PEN	6%
FG/XP	5%
KICK	7%
K RET	1%
PUNT	3%
P RET	4%

The place kicker's performance accounts for at least 5% of the variance in regular season team wins. I say 'at least' because usually the kicker performs kick-offs in addition to field goals and extra points. Accordingly, he should account for 5% of the salary cap as well.

The table below lists all kickers' salary cap charges by year from 2002 to 2006. Also listed in the team salary cap and the total league salary cap for all teams. The final column lists the percentage of the league's salary cap allocated to place kickers.

Year	K Salary	Team Cap	League Cap	K Share
2002	25,062,723	71,100,000	2,275,200,000	1.1%
2003	29,478,799	75,000,000	2,400,000,000	1.2%
2004	36,922,489	85,500,000	2,736,000,000	1.3%
2005	38,873,799	102,000,000	3,264,000,000	1.2%
2006	65,056,804	109,000,000	3,488,000,000	1.9%
Total	$195,394,614	$442,600,000	$14,163,200,000	1.4%

Since 2002, kickers have earned 1.4% of the total salary available in the NFL despite accounting for at least 5% of the variance in team wins. But things are looking up for them, as their share of the total NFL salary in 2006 rose to 1.9%.

The Importance of Field Position

I'm currently reading a copy of KC Joyner's Scientific Football 2007. It's an excellent annual prospectus full of useful stats and analysis, which I highly recommend. KC regularly writes for ESPN Insider. I just read a part of Scientific Football entitled Straight from the Dept of Meaningless Statistics (p.210). KC suggests that average drive starting position is "not meaningful at all." He cites an example comparing the Steelers and Browns in 2006:

"PIT's starting field position was their 26-yd line, which was the worst in the NFL. Cleveland's average starting field position was at their 30-yd line, which was the 2nd best...If the average team has 12-13 drives per game, that would mean each team would have [approximately] 100 drives after 8 games. This means CLE would have a 400-yd advantage in this category.

"Four hundred yards sounds like a lot but let's also put it into prespective. If the Browns had 70 potential field position yards on each of their drives (i.e. they had 70 yds to go for a TD), that would mean they have 7,000 yds to go versus PIT's 7,400. Four hundred yards sounds like a lot until you consider that is in the context of needing to gain 7,000 yards."

At first I began to think about the question this way: 400 out of 7000 yds is about 5%. That's not that much, but as a Ravens fan, I'd gladly accept an extra 5% performance boost to my team's offense. But then I realized that field position is not linear, and percentage would not the best way to conceptualize it.

Think of an offensive drive not in terms of a series of passes and runs, but in terms of a chain of first downs, regardless of how they are achieved. To arrive in scoring position a team needs not just yards, and not just 1st downs, but consecutive first downs. The success rate for achieving a 1st down on each series has been 65% over the past 5 years. So a team's probability of sustaining a scoring drive is 0.65^x, where x is the number of 1st downs needed (which would include the final scoring series as well).

On average, an NFL offense needs 3.7 first downs (including the score itself) to score a touchdown. Therefore, the estimated TD rate would be 0.65^3.7 = 0.20 TDs per drive. (Note: The actual share of drives that resulted in touchdowns over the past five years is very close--19%.)

One way to think of those 4 extra yards is that they would typically require 0.4 more first downs to score. The resulting effect on the probability of scoring is 0.65^4.1 = 0.17. The difference is 0.20-0.17 = 0.03.

A difference of only 3% in the chance of scoring a TD on a typical offensive drive may seem very small, but it has a large impact on points. Given a league average of 12.4 drives per game (according to KC Joyner), the effect on two teams with a 4-yd difference in starting field position would be:

0.17 * 12.4 = 2.1 TDs per game (14.7 points)
0.20 * 12.4 = 2.5 TDs per game (17.4 points)

The result is a 0.4 TD per game advantage to a team with a 4-yd field position edge, the equivalent of 2.8 points per game. But it wouldn't work out exactly that way, because there is obviously no such thing as 0.4 touchdowns. So sometimes a team would end up with an additional TD, sometimes not, but perhaps sometimes 2 additional TDs. In my view, this effect is very meaningful.

Here is perhaps a simpler way to conceptualize it. Instead of saying the team with lesser starting field position needs 0.4 more 1st downs per drive to score, we could say that they need a full additional 1st down in 40% of its drives.

The resulting probabilities of successful TD drives are somewhat simpler to understand. This time I'll say the average # of drives per game is 10, which I believe is closer to the actual number than KC's 12.4 number.

0.65^3.7 = 0.20 probability of TD drive
0.20 * 10 drives/game = 2.1 TD drives/game

0.65^3.7 = 0.20 probability of TD drive
0.65^4.7 = 0.13 probability of TD drive
0.20 * 6 drives/game + 0.13 * 4 drives/game = 1.7 TD drives/game

Again, the difference is 0.4 TDs/game.

But we still need to consider field goals. Drives that stall just shy of the end zone are typically converted into field goals. So the estimated difference in expected points due to touchdowns would be mitigated by the expected consolation of 3 points for the team with the worse starting field position. That is, until we consider that the team with better starting field position would also get into field goal range easier themselves. The effect of field goals is essentially a wash.

This is a league-wide general analysis. I've used a lot of words such as typically, on average, and expected. For individual teams, there are a lot of other variables, the most significant of which is 1st down success rate--65% is only the league average. For example, the 2006 Colts' 1st down success rate was 79%. In contrast, the Buccaneers' success rate last year was only 59%. That's going to have a stronger effect on the probability of scoring than starting field position. But those 4 yds still matter a good deal.

Post Script--I asked KC Joyner about this topic. He pointed out that by "meaningless," he was referring to the statistic of starting field position due to its lack of context, and not field position itself.

How to Beat the Over-Under

Note: A follow-up to the results of the 2007 season can be found here.

Just in time for the beginning of the season, I think we've cracked the code on beating the Vegas over-under lines. Prompted by a recent comment from Tarr, I did some additional analysis. Although this is not a gambling site, I realize a lot of people are interested in it, and many people would at least be interested in how to beat conventional wisdom.

I only had the last two seasons of data available, and although I'd prefer to have more, the results are convincing.

In my past several months of researching NFL win-loss records, I've noted two overwhelming phenomena:

1. The NFL is impossible to predict before the season starts. And,
2. Regression to the mean rules the day.

In practical terms, expert predictions, including the consensus Las Vegas over-under predictions, are bad primarily because they underestimate the annual tendency for bad teams to improve their records and good teams to worsen their records. Expert predictions stink--that's the good news.

The bad news is our own predictions usually stink worse. Sometimes people get lucky and outguess Vegas or the experts, but over time, luck will catch up to you. So the trick is to take advantage of Vegas' flaws by applying the two lessons above, all the while ignoring our own predictions. Here's how:

1. Take the under on teams predicted to finish with 9.5 wins or more.
2. Take the over on teams predicted to finish with 6.5 wins or less.

And here's why:

1. Of the 18 teams predicted to have 9.5 wins or more, 13 finished the season under, and 5 finished over.
2. Of the 15 teams predicted to have 6.5 wins or fewer, 11 finished the season over, and 5 finished under.

Using these two rules, the '9.5 or more' teams would have yielded a net of 13-5=8 winning bets, and the '6.5 wins or fewer' teams would have yielded 11-4=7 winning bets.

I am not a gambler at all, so forgive me if I mess this up. In the last 2 years, if you bet $110 on each game according to these rules, you'd have placed 18+15=33 bets for a total of $3300. For each loss, you'd receive nothing, but for each of the 24 wins, you'd get back $210 ($110 + $100) for a total of $5040. Your net winnings would be $1740. That's a 152% return on your "investment" over two years.

This analysis is based on only two years of data. However, of the 33 observations included, the chance of being correct by chance 24 or more times is p=0.0068. To statisticians, that means it's significant. To gamblers, it means it's a safe bet. But if anyone has over-under data from years prior to 2005, I can include it in the analysis to increase the confidence level.

The lesson here is if you're going to gamble, don't bet on your own dumb guesses. Bet against the dumb guesses of everyone else. Take the over on stupidity.

The table below lists each team's actual wins, pre-season over-under lines, and the error of the over-under predictions. The system's correct predictions are in green and its incorrect guesses are in red.

Year	Team	Wins	Over-Under	O-U Error
2005	IND	14	11.5	-3
2005	PHI	6	11.5	6
2006	IND	12	11.5	-1
2005	NE	10	10.5	1
2006	NE	12	10.5	-2
2006	SS	9	10.5	2
2006	DEN	9	10	1
2005	ATL	8	9.5	2
2005	BAL	6	9.5	4
2005	CAR	11	9.5	-2
2005	MIN	9	9.5	1
2005	NYJ	4	9.5	6
2005	PIT	11	9.5	-2
2006	CAR	8	9.5	2
2006	DAL	9	9.5	1
2006	JAX	8	9.5	2
2006	KC	9	9.5	1
2006	NYG	8	9.5	2
2006	PIT	8	9.5	2
2006	CHI	13	9	-4
2006	CIN	8	9	1
2006	MIA	6	9	3
2006	SD	14	9	-5
2006	WAS	5	9	4
2005	BUF	5	8.5	4
2005	DAL	9	8.5	-1
2005	DEN	13	8.5	-5
2005	DET	5	8.5	4
2005	JAX	12	8.5	-4
2005	KC	10	8.5	-2
2005	SS	13	8.5	-5
2005	STL	6	8.5	3
2006	ARI	5	8.5	4
2006	MIN	6	8.5	3
2006	PHI	10	8.5	-2
2006	TB	4	8.5	5
2005	SD	9	8	-1
2006	ATL	7	8	1
2005	ARI	5	7.5	3
2005	CIN	11	7.5	-4
2005	GB	4	7.5	4
2005	HOU	2	7.5	6
2005	NO	3	7.5	5
2005	OAK	4	7.5	4
2005	WAS	10	7.5	-3
2006	BAL	13	7.5	-6
2006	NO	10	7	-3
2006	STL	8	7	-1
2005	CHI	11	6.5	-5
2005	NYG	11	6.5	-5
2005	TB	11	6.5	-5
2005	TEN	4	6.5	3
2006	BUF	7	6.5	-1
2006	CLE	4	6.5	3
2006	DET	3	6.5	4
2006	GB	8	6	-2
2006	OAK	2	6	4
2005	MIA	9	5.5	-4
2006	HOU	6	5.5	-1
2006	NYJ	10	5.5	-5
2006	TEN	8	5.5	-3
2006	SF	7	5	-2
2005	CLE	6	4.5	-2
2005	SF	4	4.5	1

Pre-Season Predictions Are Worthless

I recently came across online discussions regarding pre-season predictions of win totals for each team in the NFL. In this post I'll show why you shouldn't put much faith in any of the predictions you read before the first snap of the 2007 regular season.

For comparison purposes, I'll use Football Outsiders' predictions and the Vegas over-under lines for the last two NFL seasons as representative statistical and consensus predictions. As it turns out, neither fare very well in their predictions.

I'll judge the accuracy of predictions by mean absolute error (MAE). This is the average of the absolute value of the error. If the MAE is 3.5, it would mean the predictions were off by an average of 3.5 games in either direction. The smaller the MAE, the better the prediction.

For the 2005 and 2006 NFL seasons the accuracies for Football Outsiders and Vegas betting lines are listed in the table below.

Year	FO	Vegas
2005	3.0	3.0
2006	2.3	2.3
Avg	2.6	2.6

Over the past two years, the Football Outsiders predictions are no better than the consensus Vegas lines. They are wrong by an average 2.6 wins for each team. Although the NFL is difficult to predict, 2.6 games is quite a bit in a 16 game season.

How do we judge whether 2.6 games is a good or bad prediction? To judge if predictions are worth anything, we should compare them to obvious knowledge. I compared the FO and Vegas predictions to two sets of obvious predictions. The first is if I mindlessly predicted 8 wins for every team. The second is just using a regression of last year's wins. The resulting comparison of average errors is listed below.

Year	FO	Vegas	8 Wins	Last Year
2005	3.0	3.0	3.0	2.9
2006	2.3	2.3	2.1	2.2
Avg	2.6	2.6	2.5	2.5

The average errors for '05 and '06 was 2.5 games for both the mindless 8-game predictions and last year's records. Both "obvious" methods actually do slightly better than the expert predictions.

Pre-season predictions are completely worthless, at least those of Football Outsiders and Las Vegas. In fact, they're worse than worthless.

One final note--If each division is ranked by their current Vegas over-under lines, the results are virtually identical to last year's final standings. The only exceptions are two ties. If you want to know how to take advantage of bad expert predictions, read this.

Hat Tip: Some data for this analysis came from Football Prediction Network. Original idea stemmed from the Sports Economist via Sabermetric Research.

End note: The regression of season win totals based on previous year wins yielded the following result (n=96): Next Yr Wins = 5.7 + 0.29 * Last Yr Wins.
r-squared=0.12. Significance for Last Yr Wins was p=0.00.

Median Salary and Wins

The last post examined total team salary and estimated its effect on regular season win totals. The data showed there was a connection, and that for every standard deviation above average in team salary ($13 million), a team could expect to win an extra 0.33 wins.

In recent years, I've heard some analysts explain the success of some teams, particularly the Patriots, by noting they are a team of few stars but of great depth. Beyond a couple notable exceptions, their best teams were filled with many above average players and very few big stars. They had very few holes in their starting line-up.

It seemed plausible to me. With less salary cap room taken up by big name stars, there is more to spread around at each position and for reserve players. A team composed like that would have very few weak links and would be less vulnerable to injuries.

To see if there is a connection between team composition and winning, I compared team median salary and regular season wins for 2001-2006 (n=191). A team with a lot of highly paid stars would have a low median salary, and a team with few highly paid stars would have a high median salary. However, I need to account for team total salary, because as a team increases its total salary its median salary would naturally increase without any additional "spreading of the wealth." Also, as I did previously, I normalized the salary variables by year because the salary cap grows annually.

I ran a regression model of regular season wins based on median salary and total team salary. The results are:

VARIABLE	COEFFICIENT	STDERROR	T STAT	P-VALUE
Z Median Salary	0.34	0.23	1.47	0.14
Z Total Salary	0.28	0.18	1.58	0.12
r-squared	0.03

This result supports the notion that an even team composition is advantageous, but it is not quite conclusive. Median salary is marginally significant, as is total salary. For every standard deviation above average in median salary, a team can expect an additional 0.34 wins, holding total salary constant. The team that is most "fair" in spreading the wealth would be about 2 standard deviations above average, so they could expect to win about an extra 0.68 games per year on average.

When I see results like this, that confirm what we'd expect, but with significance levels around 0.1, I suspect that there is almost certainly a connection but because the effect is small we need a larger data set to see higher significance levels.

Also note that the coefficient of total team salary is revised to 0.28 wins per standard deviation (compared to the previous post). This confirms the effects of total and median salary are not independent of one another.

Ultimately, the effects of total salary and median salary are most likely real and measurable, but small. Even the highest spending team can't even guarantee themselves one full additional win for their spending spree. I believe this underscores the importance of revenue sharing and the salary cap. Without those mechanisms to counter-balance free agency, the NFL would be very predictable, and we'd likely be watching the richest teams in the playoffs every year.

Data was obtained from the USA Today NFL salary database.