Wednesday, January 09, 2008

Fan Fielding Survey versus range measures

This is the seventh part of my series on fielding in 2007. The table of contents for the entire series is listed below:

Basic fielding stats
Converting Zone Rating to something useful
Revised Zone Rating
Probabilistic Model of Range
Fielding Bible
Ultimate Zone Rating
Fan Fielding Survey versus range measures
Outfield arms
Ranking the second basemen
Ranking the shortstops
Ranking the third basemen
Ranking the first baseman
Ranking the center fielders
Ranking the right fielders
Ranking the left fielders
What about catchers?

Today, I'm going to revisit Tom Tango's Fan Fielding Survey. Tango anually conducts a survey on defensive skills which many of you completed last year. For those of you who are not familiar with the survey, he asked fans to rate the fielding skills of players on their favorite teams just based on observation and instructed them not to use any stats at all. As you can see on the ballot, fans were asked to scout players on reaction/instincts, acceleration/first few steps, speed, hands, release/footwork, throwing strength and throwing accuracy.

He then tabulated the results which can be seen in detail on his site. Each player ends up with a score between 0 and 100 on each of the 7 skills. Tango explains that the league average rating for each of the 7 categories is 50 and that a player with a rating of 70 or better is in top 16% in the league.


Tango also came up with an idea on how to turn these ratings into runs scored above average. First, weighted averages are created for each player based on the importance of the skills at his position. The weights are shown on Table 1 below. I'm not exactly sure how he decided upon these weights but they make intuitive sense. For example, a third baseman's arm strength carries more weight than his speed. On the other hand, speed and acceleration are very important for outfielders.


Table 1: Weights for skills on fan fielding survey


Pos

Instincts

First Steps

Speed

Hands

Release

Arm Strength

Accuracy

C

1.3

0.3

0.3

1.3

1.3

1.3

1.3

1B

1.6

1.6

0.4

1.6

0.8

0.4

0.4

2B

1.6

1.6

0.8

1.6

0.8

0.4

0.4

SS

1.5

1.5

0.7

0.7

1.5

0.7

0.4

3B

0.9

0.9

0.5

0.9

0.9

1.9

0.9

LF

1.0

2.0

2.0

1.0

0.5

0.3

0.3

CF

1.0

1.9

1.9

1.0

0.5

0.5

0.2

RF

0.9

1.9

1.9

0.9

0.5

0.5

0.5


Once the weighted average is computed for a player, you then subtract from it the average score for all players at his position. I did this for all players and compared the distribution of scores to the distribution of runs scored per 150 games (RSAA/150) for PMR and RZR. It turned out that the distributions were very similar with standard deviations of 14 for PMR, 15 for RZR and 15 for fan fielding. Thus, I used these scores as RSAA/150 estimates for fan fielding.


Tango recommended multiplying the result by .7 to get run estimates because he was comparing to the UZR distribution which had a smaller standard Deviation (at least in 2003). I don't have all the UZR scores so I compared to distributions which I did have.


The results for the Tigers are in Table 2. If you are skeptical that you can turn fan observations into run estimates, then just think of the results as above or below average skill for the position.Table 3 compares the range measures discussed previously with the fan fielding estimate. The Fan Fielding results for the Tiger players correlate reasonably well with the range measures. Players that did well on the range measures - Curtis Granderson , Brandon Inge and Placido Polanco - were also given high scores by the fans. The fans also agreed that Carlos Guillen was a poor fielder last year. Magglio Ordonez and Sean Casey who went up or down according to the measure, were ranked below average by the fans.


One player who was ranked radically lower by the fans was Jacque Jones. There are a couple possible reasons for this. First, the Cubs had only 25 voters and there was a low agreement among them in (.55) in rating Jones. This makes me think that a small number of fans may have brought down his average. Another reason was that the range measures do not consider an outfielder's ability to prevent base runners from advancing with his arm and, according to the fans, Jones has poor throwing skills. I'll talk more about throwing arms shortly.



Table 2: Fan Fielding Scores for Tigers - 2007


POS

Player

n

instinct

first step

speed

hands

release

arm strength

accuracy

RSAA/150

1B

Casey

44

57

20

11

71

43

37

45

-4

2B

Polanco

52

81

67

54

89

86

56

89

17

3B

Inge

53

83

83

68

73

75

91

62

17

3B

Cabrera

9

55

29

28

42

51

73

33

-12

CF

Granderson

52

82

84

81

81

63

47

45

9

CF

Jones

25

47

62

66

53

32

25

6

-15

LF

Monroe

38

37

46

47

38

49

58

48

3

RF

Ordonez

50

52

39

36

52

45

48

51

-11

SS

Renteria

22

64

52

52

67

70

61

72

-1

SS

Guillen

52

52

41

42

29

38

47

31

-22


Table 3: Comparison of runs of RSAA/150 across methods


PLAYER

RZR

ZR

PMR

+/-

UZR

FFS

Granderson

34

10

19

18

18

9

Jones

24

9

13

42

N/A

-15

Inge

13

12

16

18

12

17

Polanco

12

4

9

8

N/A

17

Monroe

9

7

-12

N/A

N/A

3

Renteria

5

-9

-4

N/A

N/A

-1

Ordonez

-5

10

-3

N/A

14

-10

Casey

-8

8

-15

N/A

8

-4

Cabrera

-13

-16

-26

-20

-28

-12

Guillen

-18

-7

-18

N/A

-24

-22

Saturday, January 05, 2008

Ultimate Zone Rating - 2007

This is the sixth part of my series on fielding in 2007. The table of contents for the entire series is listed below:

Basic fielding stats
Converting Zone Rating to something useful
Revised Zone Rating
Probabilistic Model of Range
Fielding Bible
Ultimate Zone Rating
Fan Fielding Survey versus range measures
Outfield arms
Ranking the second basemen
Ranking the shortstops
Ranking the third basemen
Ranking the first baseman
Ranking the center fielders
Ranking the right fielders
Ranking the left fielders
What about catchers?

Today, I'm going to talk about Mitchel Litchtman's Ultimate Zone Rating (UZR). Developed in the 1990s, UZR revolutioned the evalution of fielding. It was the first system to use advanced play by play data such as location, speed and type of batted balls to determine how many outs each player was expected to make and how many he actually did make in comparison to the average player.

Lichtman says that his system is very similar to the Fielding Bible +/- system except that it uses STATS data rather than BIS data. It actually includes a few additional features that the +/- system does not use such as ground ball/fly ball tendencies of pitchers and runner/out combinations. The complete methodology is described in two articles written in 2003.

Many analysts still consider UZR to be the gold standard of range metrics. Unfortunately, Lichtman is now a consulting statistical analyst for the St. Louis Cardinals and does not publish the UZR as freely as he would otherwise. He did release data for the top three and bottom three at each position in each league for 2007 and these figures are included on the Tangotiger site.

Table 1 shows the UZR runs saved per 150 games for Tigers in 2007 (RSAA/150). Table 2 shows how all the different methods discussed so far compare on RSAA/150. It reveals that UZR agrees with the other methods that Curtis Granderson, and Brandon Inge save runs and that Miguel Cabrera and Carlos Guillen give up runs. The methods continue to be all over the place on Sean Casey and Magglio Ordonez. UZR favors both of them.


Table 1: Run saved per 150 games according to UZR

Pos

Player

Inn

RSAA/150

1B

Casey

989

8

2B

Polanco

1,209

N/A

3B

Inge

1,310

12

3B

Cabrera

1,311

-28

SS

Guillen

1,074

-24

SS

Renteria

1,019

N/A

LF

Monroe

806

N/A

CF

Granderson

1,285

18

CF

Jones

645

N/A

RF

Ordonez

1,221

14


Table 2: Runs saved per 150 games: comparison among range metrics

PLAYER

RZR

ZR

PMR

+/-

UZR

Granderson

34

10

19

18

18

Jones

24

9

13

42

N/A

Inge

13

12

16

18

12

Polanco

12

4

9

8

N/A

Monroe

9

7

-12

N/A

N/A

Renteria

5

-9

-4

N/A

N/A

Ordonez

-5

10

-3

N/A

14

Casey

-8

8

-15

N/A

8

Cabrera

-13

-16

-26

-20

-28

Guillen

-18

-7

-18

N/A

-24

Twitter

Blog Archive

Subscribe

My Sabermetrics Book

My Sabermetrics Book
One of Baseball America's top ten books of 2010

Other Sabermetrics Books

Stat Counter