No Agreement on Johnny Peralta as a Defender at Shortstop

Today, I'm going to revisit Tom Tango's Fan Scouting Report. Tango annually conducts a survey on defensive skills which many of you completed last year. For those of you who are not familiar with the survey, he asks fans to rate the fielding skills of players on their favorite teams just based on observation and instructed them not to use any stats at all. As you can see on the ballot, fans are asked to scout players on reaction/instincts, acceleration/first few steps, speed, hands, release/footwork, throwing strength and throwing accuracy.

Tango then tabulates the results which can be seen in detail on his site. Each player ends up with a score between 0 and 100 on each of the 7 skills. The league average rating for each of the 7 categories is 50 and a player with a rating of 70 or better is in top 16% in the league.

Tango also has a method for turning these ratings into runs scored above average. First, weighted averages are created for each player based on the importance of the skills at his position. The weights are shown on Table 1 below. I'm not exactly sure how he decided upon these weights but they make intuitive sense. For example, a third baseman's arm strength carries more weight than his speed. On the other hand, speed and acceleration are very important for outfielders.

Table 1: Weights for skills on fan fielding survey

Pos	Instincts	First Steps	Speed	Hands	Release	Arm Strength	Accuracy
C	1.3	0.3	0.3	1.3	1.3	1.3	1.3
1B	1.6	1.6	0.4	1.6	0.8	0.4	0.4
2B	1.6	1.6	0.8	1.6	0.8	0.4	0.4
SS	1.5	1.5	0.7	0.7	1.5	0.7	0.4
3B	0.9	0.9	0.5	0.9	0.9	1.9	0.9
LF	1.0	2.0	2.0	1.0	0.5	0.3	0.3
CF	1.0	1.9	1.9	1.0	0.5	0.5	0.2
RF	0.9	1.9	1.9	0.9	0.5	0.5	0.5

Once the weighted average is computed for a player, it is subtracted from the average score for all players at his position.  Adjustments are then made to make it comparable to the distribution of the Ultimate Zone Rating (UZR) statistic which has a smaller standard deviation.  For those who are interested in the details, the adjustments are explained in the comments here. I tried following that algorithm and got slightly different results (although very close) from what is listed on FanGraphs under FSR. The FanGraphs results are used in this post.

The results for past and present Tigers are shown in Table 2.  The table  tells us that there where 61 fans who rated center fielder Austin Jackson.  They gave him top 16% in MLB scores for instincts (84), first steps quickness (88), speed (86) and hands (79), above average scores on release/footwork (65) and accuracy of arm (55) and a below average score on arm strength (45).  This resulted in an average rating of 72 which translates to 8 runs above average.

Remember that these observations should be based on skills rather than performance. So, Jackson's score says that, in the eyes of fans, he has the skills to be a center fielder who is +8 runs above average in 1,184 innings (his total for 2012).  On the other hand, his +5 runs above on Defensive Runs Saved (DRS) is an estimate of his actual performance last year.

Table 2:  Fan Scouting Report for Tigers, 2012

pos	player	n	instincts	first step	speed	hands	release	arm strength	accuracy	FSR
C	Avila, Alex	47	52	40	33	56	67	63	63	2
C	Pena, Brayan	18	33	23	23	31	29	36	29	-4
1B	Fielder, Prince	55	32	20	25	25	25	36	31	-6
2B	Infante, Omar	47	70	65	61	63	65	49	63	+5
3B	Cabrera, Miguel	62	49	30	27	48	47	79	58	-3
SS	Peralta, Jhonny	56	46	32	36	62	59	51	61	-11
CF	Jackson, Austin	61	84	88	86	79	65	45	55	+8
RF	Hunter, Torii	19	75	70	61	75	77	70	77	+7
RF	Boesch, Brennan	52	10	23	38	11	22	45	35	-5

  
Table 3 compares the fan fielding estimate with other defensive measures.  For the most part, the Fan Fielding results for the Tiger players correlate reasonably well with the advanced measures. Players that did well on the advanced statistics such as second baseman Omar Infante, center fielder Austin Jackson and right fielder Torii Hunter - were also given high scores by the fans.  It should be noted though that Tigers fans rated Infante as average while Marlins fans had him well above average.

The fans and defensive statistics agreed that Miguel Cabrera was a a below average fielder, but his -3 on FSR was better than any of his other statistics.  There was universal agreement that right fielder Brennan Boesch was a poor defender.

There was more disagreement on shortstop Jhonny Peralta than any other player.  Peralta was rated worse by the fans (-11) than any of the advanced fielding measures.  Even the defensive metrics did not come to a consensus with Peralta ranging from -6 on Baseball Prospectus' FRAA to +10 on UZR.  There is some thought that DRS and UZR could be biased for some players due to human error in recording of location of batted balls.  So, perhaps the lower scores on FRAA and RZR might be more reliable for Peralta.

Why might the fans score be lower than all the metrics?  It could be that Peralta's skills were poor, but was able to perform above his skills for the year.  In that case, we might be wary about his ability to keep it up in 2013.  On the other hand, it could be that the fans are biased because he does not look athletic and that he's actually better than he seems.  We may learn more this year. 

Table 3: Comparison of Runs Across Different Fielding Measures

Pos	Player	FSR	DRS	UZR	FRAA	RZR
C	Avila, Alex	+2	+6	---	-3	---
C	Pena, Brayan	-4	+1	---	-1	---
1B	Fielder, Prince	-6	-4	-2	+1	+4
2B	Infante, Omar	+5	+6	+10	+4	+12
3B	Cabrera, Miguel	-3	-4	-4	-10	-18
SS	Peralta, Jhonny	-11	-1	+10	-6	-3
CF	Jackson, Austin	+8	+5	+4	+9	+11
RF	Hunter, Torii	+7	+15	+10	+1	+11
RF	Boesch, Brennan	-9	-8	-12	-5	-11