Posted by Brian Joseph on Saturday, August 15, 2009 at 12:47 am

How do we measure catcher’s defense? Who knows, really. While there’s plenty of data available, it is rare to find anyone taking a stab at it.

As I was working on this series and thinking how to look at catchers in a different way, I came across chuckb’s post at Beyond the Box Score called “An Attempt to Capture Catcher Defense.” At the same time, I was working on my own way to measure catcher’s defense. The similarities were evident. Using some of the same logic and with some help from our own Rob McQuown on determining what values to use (because Rob is more knowledgeable than me when it comes to that stuff!), here’s what I came up with and how I got there:

(Note: Before digging in, it’s important to note that these numbers were derived using only 2009 statistics. For a more accurate account and more accurate averages, years of data needs to be processed in order for a more precise measurement. At some point, an effort will be made to do so but, for now, this is where I am at in the research. As there will be efforts made to make this the best possible measurement of a catcher’s defensive impact, any input would be greatly appreciated. Feel free to e-mail Brian Joseph at bjoseph-at-comcast.net with comments, suggestions, hate mail, etc.)

CATCHER’S DEFENSIVE IMPACT RATING (CDIR): How It Was Calculated

Unlike chuckb’s attempt at BtB, the basis of this measurement of a catcher’s defensive impact utilizes six statistics made available by Baseball-Reference:

• Innings (Inn)
• Stolen Base Opportunities (SBO)
• Runner Bases Added (RBA)
• Runner Kills (RK)

In smaller part, the following data also is used to figure some of the values:

• Stolen Bases (SB)
• Caught Stealing (CS)
• Caught Stealing by Catcher (CScatch)

First, data was pulled from the Baseball-Reference website on August 12th for games through August 11th and that was the data used throughout the research.

For each catcher, the first number calculated was base runner runs (BrR). Since using RBA instead of SB, PB and WP and RK instead of CS, the value of an RBA and RK were different than the numbers used in chuckb’s analysis. Again, this used only 2009 data so these values may have to be adjusted at some point but for our purposes each RBA penalized our catcher -0.252 runs and each RK awarded our catcher +0.467 runs.

Obviously, we don’t want catchers to benefit from or be penalized by more base runners seen than average. So, based on innings played and league average Stolen Base Opportunities (SBO), expected SBOs were determined and baserunner runs (BrR) were adjusted accordingly. Let’s call it aBrR. Then, using each catcher’s innings divided by 9 to determine Catcher Games, aBrR/Gm were calculated. Here are the Top 5 and Bottom 5:

Top 5 aBrR/Game (Min. 400 Innings Caught)

1. Ryan Hanigan, Cincinnati Reds (-0.07)
2. Gerald Laird, Detroit Tigers (-0.08)
3. Yadier Molina, St. Louis Cardinals (-0.10)
4. Ramon Hernandez, Cincinnati Reds (-0.11)
5. Omir Santos, New York Mets (-0.11)

Bottom 5 aBrR/Game (Min. 400 Innings Caught)

32. Miguel Olivo, Kansas City Royals (-0.33)
33. Josh Bard, Washington Nationals (-0.34)
34. Nick Hundley, San Diego Padres (-0.35)
35. Jorge Posada, New York Yankees (-0.39)
36. Mike Napoli, Los Angeles Angels (-0.42)

Also, taking a look at backups with innings of 100-399 innings, here are the Top 5 and Bottom 5:

Top 5 aBrR/Game (Min. 100-399 Innings Caught)

1. Kenji Johjima, Seattle Mariners (-0.04)
2. Francisco Cervelli, New York Yankees (-0.04)
3. Dave Ross, Atlanta Braves (-0.08)
4. Brian Schneider, New York Mets (-0.08)
5. Raul Chavez, Toronto Blue Jays (-0.09)

Bottom 5 aBrR/Game (Min. 100-399 Innings Caught)

26. Mike Redmond, Minnesota Twins (-0.32)
27. Robinzon Diaz, Pittsburgh Pirates (-0.32)
28. John Buck, Kansas City Royals (-0.34)
29. Jose Morales, Minnesota Twins (-0.36)
30. George Kotteras, Boston Red Sox (-0.43)

No one came out on the positive end in this category with over 52 innings compiled. This is in line with chuckb’s analysis since he breaks out the same numbers into BR Runs and Miss Runs. Of the players valued by the BtB post, none were positive when adding these two numbers together.

Next, the impact of the catcher’s throwing ability on runners was measured. The BtB post did something similar and called it Rep Runs but I thought this impact needed to be measured in two ways. First, in the number of additional DPs this creates and then in the number of SBAs it reduces. We’ll start with DPs.

In order to figure out DPs added, the first step was to determine how often runners stayed in a DP situation. Using situational plate appearances from 2008 and accounting for situations captured in SBO (runner on 1st, runner on 2nd, runner on 1st and 2nd, runner on 1st and 3rd), 50% of those situations were with 0 or 1 out and 14.5% of those PAs result in a DP.

To figure out SOFA (initially named for a player Staying On First due to the catcher or Stay On First Adjustment), I took SBO and subtracted Runner Bases Advanced plus Runner Kills (adjusted for CS not due to catchers) and multipled that by an adjustment of 0.5. Even though SOFA doesn’t mean what it initially meant, the acronym worked so well, it had to stay.

Then, to determine expected DP (eDP), I took (SOFA*0.145). Based on league averages, the number of DPs should work out to 0.93/Catcher Game and lgDP for each player was determined. Finally, eDP (adjusted for pitching) was compared to lgDP and catchers with higher eDP were rewarded and lower eDP were penalized.

Using research presented in an article by John Walsh at The Hardball Times and after a slight devaluation, the overall reward for a DP for this exercise is 0.45. In this case, adding a DP rewards the catcher 0.45 runs of defense and costing a DP penalizes the catcher 0.45 runs. I know, I know. Rocket science, right? Actually, Walsh settled on a value of 0.47 rounded up to 0.5 but the situations looked at here do not involve bases loaded ones as his did which slightly reduces the overall value of a DP. Hence, 0.45!

Here’s the Top 5 and Bottom 5 Double Plays Added Runs/Game (DPAR/G):

Top 5 DPAR/Game (Min. 400 Innings Caught)

1. Omir Santos, New York Mets (0.020)
2. Yadier Molina, St. Louis Cardinals (0.017)
3. Gregg Zaun, Baltimore Orioles/Tampa Bay Rays (0.013)
4. Ivan Rodriguez, Houston Astros (0.013)
5. Joe Mauer, Minnesota Twins (0.011)

Bottom 5 DPAR/Game (Min. 400 Innings Caught)

32. Josh Bard, Washington Nationals (-0.011)
33. Russell Martin, Los Angeles Dodgers (-0.013)
34. Miguel Olivo, Kansas City Royals (-0.017)
35. Mike Napoli, Los Angeles Dodgers (-0.018)
36. Jorge Posada, New York Yankees (-0.028)

Here’s the Top 5 and Bottom 5 between 100 and 399 innings caught:

Top 5 DPAR/Game (100-399 Innings Caught)

1. Brian Schneider, New York Mets (0.021)
2. Michel Hernandez, Tampa Bay Rays (0.016)
3. Jason LaRue, St. Louis Cardinals (0.015)
4. Chris Coste, Philadelphia Phillies/Houston Astros (0.012)
5. Landon Powell, Oakland Athletics (0.010)

Bottom 5 DPAR/Game (100-399 Innings Caught)

26. Ramon Castro, New York Mets/Chicago White Sox (-0.008)
27. John Buck, Kansas City Royals (-0.013)
28. George Kotteras, Boston Red Sox (-0.018)
29. Robinzon Diaz, Pittsburgh Pirates (-0.018)
30. Jose Morales, Minnesota Twins (-0.022)

Finally, an adjustment for limiting stolen base attempts was made. To value a stolen base attempt, there were a few steps. First, CS% based on catcher’s CS only (CSctch) was used. That number came out to be 22%. Using RBA and RK values from earlier and a 22% CS%, catchers were awarded 0.0938 runs per stolen base attempt prevented (Let’s call it SAP!).

chuckb in his BtB post did something similar and called it Rep Runs. In that calculation, chuckb used inning averages. The SAP Runs (SAPR) differ from chuckb’s Rep Runs because it is based on Stolen Base Opportunities (SBO) and adjusted for pitching rather than based on Innings. One step chuckb takes that I didn’t was to use each catcher’s CS% in his Rep Runs calculation. CS% is so volatile throughout the season, I thought using that measure would make for too many fluctuations in SAPR. Plus, something like CS% is not really predictive. For example, if you have a hitter with a .300 batting average, it’s doubtful your prediction of him getting 3 hits in 10 at-bats is going to be accurate.

Here’s the Top 5 and Bottom 5 Stolen Base Attempts Prevented Runs/Game (SAPR/G):

Top 5 SAPR/Game (Min. 400 Innings Caught)

1. Yadier Molina, St. Louis Cardinals (0.042)
2. Ivan Rodriguez, Houston Astros (0.026)
3. Omir Santos, New York Mets (0.025)
4. Jason Kendall, Milwaukee Brewers (0.021)
5. Chris Snyder, Arizona Diamondbacks (0.018)

Bottom 5 SAPR/Game (Min. 400 Innings Caught)

32. John Baker, Florida Marlins (-0.020)
33. Mike Napoli, Los Angeles Angels (-0.024)
34. Nick Hundley, San Diego Padres (-0.027)
35. Jason Varitek, Boston Red Sox (-0.030)
36. Jorge Posada, New York Yankees (0.040)

Here’s the Top 5 and Bottom 5 SAPR/G for catchers with 100-399 innings:

Top 5 SAPR/Game (100-399 Innings Caught)

1. Jason LaRue, St. Louis Cardinals (0.044)
2. Humberto Quintero, Houston Astros (0.032)
3. Landon Powell, Oakland Athletics (0.024)
4. Jesus Flores, Washington Nationals (0.022)
5. Michel Hernandez, Tampa Bay Rays (0.021)

Bottom 5 SAPR/Game (100-399 Innings Caught)

26. Dave Ross, Cincinnati Reds (-0.022)
27. George Kotteras, Boston Red Sox (-0.024)
28. Robinzon Diaz, Pittsburgh Pirates (-0.025)
29. John Buck, Kansas City Royals (-0.030)
30. Ronny Paulino, Florida Marlins (-0.036)

CATCHER’S DEFENSIVE IMPACT RATING (CDIR): The Results Are In

Once this data was crunched, the three categories of runs — Base runners (aBrR), Double Plays Added (DPAR) and Stolen Base Attempts Prevented (SAPR) — were added together for each catcher. This number is referred to as Catcher’s Defensive Impact Rating (CDIR).

Next, the runs per true games caught were figured and converted to 120 games. Although chuckb’s attempt at BtB used 150 to keep in line with UZR/150, the fact that catchers don’t play 150 games in a season, converting to 150 felt inaccurate. Instead, 120 games was a better fit. This is represented by the column titled CDIR/120.

Finally, one last calculation figured each catcher’s CDIR/120 above the average for catchers. This is referred to as CDIRaa/120 in the chart and how the catchers are ranked.

Here’s the list for catchers with 400 or more innings caught:

Here’s the list for catchers with 100 to 399 innings caught:

ANALYSIS & OBSTACLES: Overall, the list fits well with the scouting reports for catchers. Yadier Molina comes out on top with Ryan Hanigan and Gerald Laird right behind him. The list is also similar in order to chuckb’s list at BtB with some exceptions. Also, the first table captures a few names that chuckb did not and the second table contains the primary backups throughout the leagues.

This was a hot topic at BtB when chuckb posted it on the site. Many were excited to see an attempt. Some pointed to other attempts made. Others (including UZR creator MGL) were skeptical in its value.

The arguments against attempts to measuring catcher’s defense usually revolve around how much is the catcher and how much is the pitcher. Then again, it is perfectly acceptable for the best defensive measurement we have to be perfectly happy with not adjusting for positioning (even though it impacts a majority of defensive plays.)

Another argument is passed balls vs. wild pitches. How do you treat both? In this case, both were treated the same which might be harsh to a catcher. However, the biggest argument against fielding % is typically that errors are not objective and based on a scorer’s decision (among other reasons). In this case, the decision was made to treat passed balls and wild pitches the same to eliminate scorer’s subjectivity. Is it fair? Absolutely not. But it’s the best we have… yeah, that’s the ticket!

To make a case for catcher’s actually having a decent effect on defense, let’s take a look at Padres catchers Nick Hundley and Henry Blanco. If you look at traditional catching statistics, Nick Hundley is pretty awful while Henry Blanco is average to above. chuckb’s analysis at BtB was cruel to Mr. Hundley but didn’t calculate Blanco’s defense. CDIR also thought Hundley was brutal (-18.57 CDIRaa/120) while Blanco actually worked out pretty positively (+8.76 CDIRaa/120).

How could this be? Maybe Blanco just benefits from the luck of handling all of the left-handed pitchers the Padres have. Oh wait! The Padres don’t have many left-handed pitchers! Besides, for such a drastic difference, the numbers would have to be very lopsided. Of all plate appearances handled by Hundley and Blanco, Hundley handled LHP 2.3% of the time while Blanco handled LHP 6.7%. Yes, Blanco benefited slightly… but not enough to sway the results as far in favor of Blanco as they are. Also, runners were 3-for-3 against Hundley vs. LHP. That’s a whopping 0% CS% for Hundley when he had the advantage of a lefty on the mound. Good ol’ Henry had 0 SB attempts against him in the 6.4% of the time a LHP was on the mound.

We also know that the Padres have a few RHPs who are particularly brutal at holding runners on — Jake Peavy, Chris Young and Cla Meredith — and those three both had Hundley and Blanco as a battery mate. 21.0% of Hundley’s PAs came with one of the three on the mound while 22.7% of Blanco’s PAs came with one of those on the mound. In this case it was Blanco who was at the disadvantage.

Here’s the PA data:

Based on these numbers, Blanco didn’t draw a major advantage over Hundley this season yet he was above average (ranked #10 amongst backups) while Hundley was below average (ranked #34 amongst regulars).

The estimate usually shared in these arguments against catcher defense is that catchers only account for 30% of the overall defense with little impact on CS% and pitchers account for the other 70%. Personally, this is not something I buy.

Either way, the mistake would be to ignore the possibility of catcher’s defensive impact rather than prove or disprove its existence. In this isolated case with a very small sample size, the Hundley/Blanco comparison suggests that catchers may have more than a negligible impact on defense when it comes to limiting stolen bases and throwing runners out. (Not to mention the possible impact on double plays).

Is there more work to be done here? Absolutely! There are a handful of problems to recognize and eventually handle:

• Sample size and averages based on a single year
• Determining whether or not to measure PBs and WPs differently
• Adjustment for pitcher splits

But it’s a start… not that it hasn’t been tried before.

A couple of things related to the above exercise. First, thanks to Rob McQuown for his help throughout this attempt. Rob’s ability to put up with me going on and on about catcher’s defense might be award-worthy. In addition, he was extremely helpful in figuring out the run values. I know that at one point in one of my initial efforts Rob used an analogy comparing measuring catcher’s defense to an ornamental gate at a mansion in a low crime neighborhood.

Thanks to Rob’s help and some tweaks, I’m personally ready to call this a little more than an ornamental gate. It’s not a security guard in a high crime high rise but it’s something. And as I’ve claimed before, the defensive metrics we have at the ready for other positions are still in their prehistoric phases. Once Hit f/x becomes fully functional and readily available, it’s reasonable to expect that there will be great strides in the advancement of defensive metrics.

As mentioned before, there’s other stabs out there at measuring catcher’s defense. I referenced one in particular that I thought was the best I’ve seen. My work captures some items that chuckb’s work doesn’t. Plus, it takes a deeper look at more catchers which gives us the Hundley/Blanco comparison. Although the work was done independently, a comparsion of the chuckb attempt and this one shows some significant tweaks and advancements on his effort. Most notably factoring in the impact a catcher has on DPs is something that really needs to be included.

It’s interesting that both chuckb and I had similar methodologies but both were done independently. One advantage I had over chuckb was to read some of the comments at BtB on his method and use that to inspect my own work. Also, looking at some of the other stabs out there, none come as close to truly capturing the defensive run impact of a catcher.

Future steps in this area will get us closer to understanding the impact of a catcher’s defense. I have a habit of selling myself poorly but others who I shared this with prior to posting felt the work was a big step in the right direction. Hopefully, you’ll feel the same way too.