Sunday, June 27, 2010

Is Valverde Being Used Optimally?

There is no question that Jose Valverde has been awesome this season.  He has surrendered runs in only two of his 33 appearances and has an ERA of 0.56.  He has allowed just 11 hits and has a 29/12 strikeout to walk ratio in 32 1/3 innings.  This has translated into 17 saves in 18 opportunities. While he has been an indispensable piece of the Tigers bullpen this year, I've had the sense that the Tigers best reliever has not been used optimally.

It's the nature of the modern bullpen that a closer will often enter games in non-pressure situations.  Valverde has pitched in eleven games where the Tigers had a four run lead or greater and one in which they were behind by multiple runs.  In many of these cases, he was used simply because he needed work.  He has also has had four saves when he came into the game  with a three run lead.  Thus, he has had only 13 saves where he entered the game with a lead of one or two runs.  Have his talents been wasted in the specialized closer role?

New statistics have been developed in recent years which are better than ERA and saves for measuring reliever performance.  The win probability added (WPA) statistic gives relievers credit based on the effect each batter faced has on the team's probability of winning.  These probabilities vary depending on the game state before and after each play.

Eldon and Harlan Mills applied Player Win Averages – an early version of WPA – to the 1969 season using play-by-play data purchased from the Elias Sports Bureau.  One interesting result was that the leading pitchers in player win average were relievers Tug McGraw of the New York Mets and Ken Tatum of the California Angels.  This was noteworthy at the time, as
relievers were not valued by fans and media in 1969 as much as they are today.

WPA works as follows.  Suppose Valverde comes into the game in the top of the eighth with a two run lead, no outs and a runner on first.  There is a 0.837 (83.7%) expectancy that an average team will win a game given that situation. Now, assume that he strikes out the first batter.  There is now one out and the probability of winning increases to 0.884 (88.4%).  Subtracting the win expectation before the strikeout from the win expectation after the strikeout gives us the value of the play in terms of the probability of winning added by the strikeout, that is, 0.047 (0.884 – 0.837) or 4.7%.

Finally, suppose the next batter, facing Valverde, doubles home a run.  That gives the Tigers a one run lead with a runner on second and one out.  The probability of winning goes down to 0.774 so he loses .110 (0.774 – 0.884) points on that batter.  Summing all the gains and subtracting all the losses for all the batters Valverde faces during a season yields his WPA.  WPA is especially useful for relievers, because of the impact their innings typically have on the outcomes of games. 

The top American League relievers in terms of WPA in 2010 (according to FanGraphs) are listed below:

Daniel Bard, Bos 2.52
Rafael Soriano, TB 2.42
Jose Valverde, Det 1.71
Joakim Soria, KC 1.67
Darren Oliver, Tex 1.61

Valverde ranks a distant third on this statstic but what is most interesting to me is that two of the top five relievers are not closers.

WPA helps to measure reliever performance but it doesn't really get at the main question here which is whether Valverde has been used optimally. Ideally, you would want your best reliever to pitch in the most high pressure situations.  This can be answered using the leverage index concept developed by Tom Tango.  Leverage index (LI) measures how critical a given plate appearance is in determining the final result of a game by looking at the difference in win probability between the best and worst case scenarios.

Tango assigns a value of one to an average game situation.  Higher-leverage situations have values of more than one and lower-leverage situations have values less than one.  Each game scenario is then given a leverage index relative to the average situation.  A leverage index of two, for example, means that the given at bat has twice as much impact on the outcome of the game as the average at bat.  Leverage indexes are averaged over the batters faced by a pitcher to arrive at leverage index per plate appearance (pLI).  If we are just concerned with the index when the pitcher enters the game, then we would use gmLI. 

The AL gmLI leaders in 2010 are listed below:

David Aardsma, Sea 2.08
Daniel Bard, Bos 2.07
Jon Rauch, Min 1.97
Jonathan Papelbon, Bos 1.95
Neftali Feliz, Tex 1.94

Not surprisingly, four of the top five pitchers are closers.  However, Jose Valverde is only 19th in the league with a 1.49 gmLI.  This tells us that many other relievers are seeing higher impact situations than Valverde.  Even more telling is that Valverde does not lead his own team in gmLI:

Ryan Perry 1.70
Jose Valverde 1.49
Phil Coke 1.37
Joel Zumaya 1.29

Perry has been used in higher impact situations than Valverde and Zumaya and Coke are not far behind.

According to the numbers, Valverde has not been used as well as he could have been.  This is not a criticism of Jim Leyland.  All managers use their closers pretty much the same way.  It's more a question as to how well the system works when arguably the league's best reliever is not being used in high leverage spots very often.

Instead of having Valverde enter a dozen or more games in very low impact situation just to get work, wouldn't it be better if Leyland picked his spots using him only when the game was on the line?  I'd rather see him enter a tie game in the eighth inning or with the bases loaded in the seventh than see him get a three out save with nobody on base and three run lead.

I know the argument that relievers need to have roles in order to feel comfortable.  However, John Hiller, Aurelio Lopez, Willie Hernandez and countless others had success without such strictly defined roles.  It will take a lot of courage for a manager to go against the system but I think there is a better way than the status quo.


  1. AnonymousJune 27, 2010

    Valverde has an ERA of .56, not 1.57

  2. Thanks. I shouldn't be making long posts late at night. I'll fix it before too many people see it. :-)


  3. AnonymousJune 28, 2010

    Ummm...Rafael Soriano is a closer. 18 saves already this year.

  4. AnonymousJune 28, 2010

    Nevermind, I misread "two of the top" as "the two top." Whoops. Nice article.

  5. Politically you could never get away with this, but I have been a proponent on and off of using other relievers occasionally when there is a 3-run save on the line, if your closer can use the rest. Of course, that costs your closer his stat, which he reasons will costs him money at contract time. But I think at times it would be an optimal use of your staff, contingent on recent usage for your closer.

  6. It won't happen any time soon, but with more and more organizations becoming stat based, I've got to think they will rethink the closer role eventually.



Blog Archive


My Sabermetrics Book

My Sabermetrics Book
One of Baseball America's top ten books of 2010

Other Sabermetrics Books

Stat Counter