Fan Fielding Survey versus range measures

This is the seventh part of my series on fielding in 2007. The table of contents for the entire series is listed below:

Basic fielding stats
Converting Zone Rating to something useful
Revised Zone Rating
Probabilistic Model of Range
Fielding Bible
Ultimate Zone Rating
Fan Fielding Survey versus range measures
Outfield arms
Ranking the second basemen
Ranking the shortstops
Ranking the third basemen
Ranking the first baseman
Ranking the center fielders
Ranking the right fielders
Ranking the left fielders
What about catchers?

Today, I'm going to revisit Tom Tango's Fan Fielding Survey. Tango anually conducts a survey on defensive skills which many of you completed last year. For those of you who are not familiar with the survey, he asked fans to rate the fielding skills of players on their favorite teams just based on observation and instructed them not to use any stats at all. As you can see on the ballot, fans were asked to scout players on reaction/instincts, acceleration/first few steps, speed, hands, release/footwork, throwing strength and throwing accuracy.

He then tabulated the results which can be seen in detail on his site. Each player ends up with a score between 0 and 100 on each of the 7 skills. Tango explains that the league average rating for each of the 7 categories is 50 and that a player with a rating of 70 or better is in top 16% in the league.

Tango also came up with an idea on how to turn these ratings into runs scored above average. First, weighted averages are created for each player based on the importance of the skills at his position. The weights are shown on Table 1 below. I'm not exactly sure how he decided upon these weights but they make intuitive sense. For example, a third baseman's arm strength carries more weight than his speed. On the other hand, speed and acceleration are very important for outfielders.

Table 1: Weights for skills on fan fielding survey

Pos	Instincts	First Steps	Speed	Hands	Release	Arm Strength	Accuracy
C	1.3	0.3	0.3	1.3	1.3	1.3	1.3
1B	1.6	1.6	0.4	1.6	0.8	0.4	0.4
2B	1.6	1.6	0.8	1.6	0.8	0.4	0.4
SS	1.5	1.5	0.7	0.7	1.5	0.7	0.4
3B	0.9	0.9	0.5	0.9	0.9	1.9	0.9
LF	1.0	2.0	2.0	1.0	0.5	0.3	0.3
CF	1.0	1.9	1.9	1.0	0.5	0.5	0.2
RF	0.9	1.9	1.9	0.9	0.5	0.5	0.5

Once the weighted average is computed for a player, you then subtract from it the average score for all players at his position. I did this for all players and compared the distribution of scores to the distribution of runs scored per 150 games (RSAA/150) for PMR and RZR. It turned out that the distributions were very similar with standard deviations of 14 for PMR, 15 for RZR and 15 for fan fielding. Thus, I used these scores as RSAA/150 estimates for fan fielding.

Tango recommended multiplying the result by .7 to get run estimates because he was comparing to the UZR distribution which had a smaller standard Deviation (at least in 2003). I don't have all the UZR scores so I compared to distributions which I did have.

The results for the Tigers are in Table 2. If you are skeptical that you can turn fan observations into run estimates, then just think of the results as above or below average skill for the position.Table 3 compares the range measures discussed previously with the fan fielding estimate. The Fan Fielding results for the Tiger players correlate reasonably well with the range measures. Players that did well on the range measures - Curtis Granderson , Brandon Inge and Placido Polanco - were also given high scores by the fans. The fans also agreed that Carlos Guillen was a poor fielder last year. Magglio Ordonez and Sean Casey who went up or down according to the measure, were ranked below average by the fans.

One player who was ranked radically lower by the fans was Jacque Jones. There are a couple possible reasons for this. First, the Cubs had only 25 voters and there was a low agreement among them in (.55) in rating Jones. This makes me think that a small number of fans may have brought down his average. Another reason was that the range measures do not consider an outfielder's ability to prevent base runners from advancing with his arm and, according to the fans, Jones has poor throwing skills. I'll talk more about throwing arms shortly.

Table 2: Fan Fielding Scores for Tigers - 2007

POS	Player	n	instinct	first step	speed	hands	release	arm strength	accuracy	RSAA/150
1B	Casey	44	57	20	11	71	43	37	45	-4
2B	Polanco	52	81	67	54	89	86	56	89	17
3B	Inge	53	83	83	68	73	75	91	62	17
3B	Cabrera	9	55	29	28	42	51	73	33	-12
CF	Granderson	52	82	84	81	81	63	47	45	9
CF	Jones	25	47	62	66	53	32	25	6	-15
LF	Monroe	38	37	46	47	38	49	58	48	3
RF	Ordonez	50	52	39	36	52	45	48	51	-11
SS	Renteria	22	64	52	52	67	70	61	72	-1
SS	Guillen	52	52	41	42	29	38	47	31	-22

Table 3: Comparison of runs of RSAA/150 across methods

PLAYER	RZR	ZR	PMR	+/-	UZR	FFS
Granderson	34	10	19	18	18	9
Jones	24	9	13	42	N/A	-15
Inge	13	12	16	18	12	17
Polanco	12	4	9	8	N/A	17
Monroe	9	7	-12	N/A	N/A	3
Renteria	5	-9	-4	N/A	N/A	-1
Ordonez	-5	10	-3	N/A	14	-10
Casey	-8	8	-15	N/A	8	-4
Cabrera	-13	-16	-26	-20	-28	-12
Guillen	-18	-7	-18	N/A	-24	-22