Statistics Are Not Like Bikinis
Posted by Bill Baer on Sunday, June 21, 2009 at 2:59 pm
Statistics are like bikinis. What they reveal is suggestive, but what they conceal is vital.. . .
There are three kinds of lies: lies, damned lies and statistics.
I have no idea who the founders of the quotes are, but I know that they do not consider the human factor in the application of statistics. (Isn’t it ironic that a Sabermetrician is asking others to consider the human factor?)
What are statistics? From Wikipedia:
Statistics is a mathematical science pertaining to the collection, analysis, interpretation or explanation, and presentation of data.
Read all of the items following “pertaining to” and individually consider the human factor. Here are some.
Collection: Sampling bias
Analysis and interpretation/explanation: Conflicting interests, fundamental misunderstanding of the subject or of statistics
Presentation: Truncating graphs (other types of statistical vandalism)
Human beings are imperfect. We get jealous; we get angry; we get ecstatic; we get depressed. Sometimes, we can be petty in acting on our emotions. For instance, if I’m a part of PETA and I really hate people who eat meat and wear fur coats, I might join a protest where I spray paint fur coats to demonstrate my anger.
Similarly, if I don’t like teams who spend a lot of money in baseball, I might try to influence other people to think the same way as me by creating a graph that illustrates how much of a stranglehold big-spenders have in the game, and posting it on my blog, even if that graph is intentionally dishonest.
Recognizing our limits as humans when it comes to the sciences is important in understanding and trusting science. That is, actually, the beauty of science: a system of speculation, experimentation, and knowledge-enhancement that is self-critical. Take a religion as an example. When a discovery is made that contradicts what is in the Bible, does religion adapt its story for truth? No.
Think of all of the discoveries that have been made in science as far back as you want to go. Now try to think of any fundamental truths that science has ignored. None. Science isn’t petty where it is unable to admit fault and change its tune; people are. Science is a great system, but it is unfortunately used by imperfect human beings, so there are going to be some problems from time to time.
I say all of this, of course, in regard to the article that Brian wrote yesterday about Sabermetrics. Some of the criticism was valid, but he didn’t identify the correct target, which should have been us, the fallible humans. His criticisms (such as Sabermetrics being subjectively objective) didn’t put any chinks the armor of Sabermetrics because Sabermetrics — essentially one of baseball’s sciences — is adaptable. If there’s a problem with an aspect of Sabermetrics, go ahead and change it.
Additionally, Sabermetrics — contrary to the claims of many who don’t trust it — is not adhered to religiously. Just because PECOTA says Matt Wieters is going to smoke some American League pitching doesn’t mean that he is, in fact, going to smoke some American League pitching. We can choose to accept what various analyses say, or we can reject them. Just because one prefers DIPS to ERA doesn’t mean one is bound to the conclusions reached via DIPS and cannot utilize ERA in analysis.
I don’t write this as a jab back at Brian, what with me being someone who utilizes Sabermetrics. I welcome and enjoy reading well-written, well-researched criticisms of the sciences because it can only help us improve. It is important, though, to identify who or what is really at fault, and in the case of Brian’s criticisms, it’s the people using the science and not the actual science itself.
Finally, we need more than people simply identifying the errors; we need suggestions to improve what we already have. It is definitely important to point out that, for instance, some versions of FIP don’t account for batted ball rates, as Derek Carty pointed out in an article he wrote for The Hardball Times. What is just as important though, in improving the sciences of baseball, is providing alternatives (as Carty did), or else you’re just going to leave people at square one.
The Sabermetric community is not inclusive. I consider myself a part of it now and I was met with no resistance whenever it was that I joined. I didn’t see any bouncers, and I didn’t have to put my name on a list. Serious criticisms of Sabermetrics are always welcome even if you don’t consider yourself a Sabermetrician. Further, any Sabermetrician who is not willing to put his beliefs up to a test does not understand the difference between science and belief systems. So, criticize away.
Hopefully, this clears up a bit of what’s usually lost in translation between those who are “in” and those who are “out”. The biggest thing to heed is that when humans do anything, there’s a good chance they’re going to mess up. Sabermetrics is no different.
I think you made my point. I have no problem with sabermetrics. I do have a problem with those who employ the usage of it turning it into a mean and nasty verbal war.
I think a lot of it has to do with most sabermetric arguments taking place on the Internet. Typically, when I play live poker, no one berates me and calls me a fat a**, idiot donkey who sucks at life when I turn a straight after calling their big raise on the flop with an open-ended straight flush draw… however, online, that guy will fill up the chat box with much worse commentary when playing online poker.
I’m sure if most sabermetric arguments took place in person rather than in a place where Internet muscles can be flexed without consequence, sabermetrics would be able to move a step ahead. And it wouldn’t hurt if hardcore sabermetricians didn’t snicker every time someone not established in the “science” of sabermetrics quoted a statistic other than batting average, runs, RBI or HR.
My point about Park Factors wasn’t that they should be dismissed, it was that when using them to devalue Matt Holliday’s value, it is often forgotten that he had a hand in putting up the numbers that earned the Colorado ballpark it’s hitter-friendly reputation. That’s all.
I do happen to agree with the “lies, damned lies and statistics” quote in the beginning of your article. Because what I’ve learned over the years is that no matter what you say, someone will find a statistical argument that proves you wrong… unless, of course, you are trying to say Adam Eaton is a great pitcher.
Some things are just impossible.
I think you made my point. I have no problem with sabermetrics.
If that was the case, then you should have written a generalized article about ideologues and not about Sabermetrics.
You’ll find ideologues in every walk of life. See: Hannity, Sean. People who are devoted to ideas or people and not truth are the people who are going to display the behavior you described. It says nothing about the majority of people who belong to a certain group or the validity of what they are espousing.
Also, I think you’re making strawman arguments. Who is snickering when you use AVG, HR, RBI? Who is berating you for not adhering to Sabermetric tenets?
There’s a difference between critical analysis (i.e. “batting average is flawed because it doesn’t properly account for a hitter’s ability to draw walks, a tangible, repeatable skill”) and outright bashing.
No, I shouldn’t have written about ideologues because that’s not what I wanted to write about. I wanted to write about how sabermetrics can be used for evil and purposely left my stance on sabermetrics vague so that I could draw out very nasty and mean spirited comments to prove my point of how those who discuss sabermetrics act when they don’t like something. How’d that work out for me? I think it worked out just fine.
My “snickering” comment wasn’t about AVG, HR, RBI, it was about when a network like ESPN begins to use OPS in their broadcasts and comment about it, some in the sabermetric world make nasty comments about their assumed lack of understanding.
As far as batting average being flawed, I don’t think anyone ever said batting average did anything but measure how often a player is expected to get a hit per at-bat and at-bats do not account for walks so batting average arbitrarily chooses to ignore walks, it is not flawed. It is only flawed by those who use it incorrectly.
I wanted to write about how sabermetrics can be used for evil and purposely left my stance on sabermetrics vague so that I could draw out very nasty and mean spirited comments to prove my point of how those who discuss sabermetrics act when they don’t like something.
That’s not exclusive to Sabermetrics; any belief system will have these ideologues. Even baseball traditionalists act this way (see: Conlin, Bill).
By targeting Sabermetrics specifically, you come off as accusing the science and the people that use it of uniquely being detrimental, as opposed to people just being people. There is nothing specifically you can say of the science of Sabermetrics or the people that use it that is unique to those areas specfically.
some in the sabermetric world make nasty comments about their assumed lack of understanding.
Again, these people you are talking about (any chance you could cite some examples?) are not exhibiting behavior unique to Sabermetrics. Additionally, it’s not an assumed lack of understanding. Joe Morgan talks about clogging bases and home runs not being rally-starters. That’s a lack of understanding. Joe can play better baseball with his pinky finger than I ever can or ever could, but he does not understand fundamental concepts such as “more baserunners = more runs.”
Harold Reynolds on the MLB network is another ideologue who is too intellectually lazy to learn about that which he criticizes. I can’t find a video for you (copyright claim), but before the season, he didn’t like PECOTA’s projection for Kevin Youkilis (rightfully so, in my opinion). He went off on a rant against stats that demonstrated just how little he knows about the subject.
It’s like someone saying they don’t like mustard on their hot dogs without ever having tried mustard before.
As far as batting average being flawed, I don’t think anyone ever said batting average did anything but measure how often a player is expected to get a hit per at-bat and at-bats do not account for walks so batting average arbitrarily chooses to ignore walks, it is not flawed.
It’s flawed in the sense of, “Why would you use AVG when you have OBP?” OBP has everything AVG has plus walks, HBP, etc. which are skills (getting hit by pitches is a repeatable skill; see: Biggio, Craig and Utley, Chase).
Batting average has very little use in any area of baseball analysis.
Boys will be boys, right? Yeah… that’s often the same argument used in defense of hazing rituals at fraternities (and sororities) that turn horribly wrong. Sorry, just because a lot of people do something, it doesn’t make it acceptable to me.
Check out Harold Reynolds’ blog on MLBlogs which has drawn a lot of snarky comments because he criticized OPS which is as flawed as any other statistic yet right now, it is sexy so we love to use it and damn anyone that attacks it.
As far as your statement “Why would you use AVG when you have OBP?”… i offter this as an alternative… Why would you use one or the other when you have BOTH?
Here’s why I say that (I know, I know, Batting average has little to do with any area of baseball analysis but humor me for a second.)
If you are facing Kevin Slowey, would you rather have Ichiro (.332 BA/.377 OBP lifetime) or Jim Thome (.278 BA/.406 OBP). In the case of Slowey who doesn’t walk very often, I think I want Ichiro… yet if you base your decision solely on Thome’s OBP then you discount Ichiro’s impressive BA (I know, I know… meaningless, right?)
Sorry, just because a lot of people do something, it doesn’t make it acceptable to me.
I wasn’t defending the behavior, I was saying that your argument comes off as accusatory towards Sabermetricians as if the behavior is unique to them and only them. Ideologues come from all walks of life. You’ll find people making snarky comments about people who don’t drink their coffee at Starbucks, or people who choose to read Time over Newsweek, or watch Judge Joe Brown.
he criticized OPS which is as flawed as any other statistic yet right now, it is sexy so we love to use it and damn anyone that attacks it.
That’s not the point. Criticizing OPS is just fine, but it’s how he does it that draws the criticism. Example: I don’t know squat about cricket. If I wrote an article here about how cricket is stupid and people that play cricket are stupid, that’d be pretty ignorant.
That’s Harold Reynolds criticizing advanced statistics. He hasn’t done any of his own studies on it nor has he cited any by anyone else. He simply doesn’t like it because it’s something new he has to learn about, and that’s hard for someone who was a professional Major Leaguer for twelve seasons. It’s hard to admit that you don’t know as much about something as people who haven’t stepped on a baseball field since high school.
Why would you use one or the other when you have BOTH?
OBP has everything batting average has plus more. It’s like having a toaster oven and a toaster. One makes just toast, while the other makes toast, can heat up pizza and fries, and hey, you can broil a steak, too.
If you are facing Kevin Slowey, would you rather have Ichiro (.332 BA/.377 OBP lifetime) or Jim Thome (.278 BA/.406 OBP). In the case of Slowey who doesn’t walk very often, I think I want Ichiro… yet if you base your decision solely on Thome’s OBP then you discount Ichiro’s impressive BA (I know, I know… meaningless, right?)
The answer is far more nuanced than that and we need more information. What inning is it? What’s the score? Who’s on base? What are Thome and Suzuki’s strengths and weaknesses? What are the pitcher’s strengths and weaknesses?
Using one tool and one tool only (be it OBP or AVG or VORP) to make decisions is poor form.
I never said the behavior was exclusive to sabermetrics specifically. Honestly, I’ve done more sabermetric research than 80% of those who actually read and buy into sabermetrics. And that’s not being pompous that’s just fact.
Thinking you understand the statistics does not entitle you to be an asshole to those you feel don’t just because you find them on the other side of your argument. The same goes if you think your $9 cup of coffee from Wawa tastes better than the one I get for $1 at Wawa. This isn’t directed at you, btw… just a general comment on the usual way people decide to respond to anyone dismissing anything sabermetrics-wise.
If you read Harold Reynolds’ comments and came to that conclusion then I doubt we can see anything eye-to-eye on this one. Harold Reynolds is very intelligent and knows a lot about the game. Reynolds isn’t criticizing advanced statistics… he’s criticizing OPS! Why is one knock on anything sabermetrics a knock on all things sabermetrics.
If I don’t like apple pie, it doesn’t mean I don’t like all desserts, right? I think those who value OPS often overvalue it… there can be vast differences between two players with an .800 OPS. One could be .400/.400 or .200/.600 and those players are very, very different. Plus, there are many other arguments I could put out there about the value of OPS.
Reynolds also makes good points AND he has the perspective of actually playing the game. And yes, that does have value despite the popular opinion of a large segment of the sabermetric community.
And if my argument is why not use BOTH then wouldn’t you think I would be on board with using all of the data available? What I was saying is that you can’t just dismiss that BA has little value when it is obvious that it can provide value in situations and I gave an example of where it wouldn’t hurt to know that data.
And while OBP has everything batting average has plus more, certain pitchers just don’t walk guys (Slowey, Maddux, etc.) and whether a guy has the ability to draw a walk is diminished. Your toaster/toaster oven analogy is pretty good for that one because any toast afficianado knows that toast made in a toaster is always better than toast made in a toaster oven.
I wasn’t saying that you said that the behavior was exclusive to Sabermetricians; I was saying that you implied it by writing this article about Sabermetrics specifically. If you went onto an Xbox 360 forum and said, “Gosh, I hate it when people flame me for having a PS3,” you’re going to have a bunch of Xbox 360 fan boys wondering why you’re accusing them.
I agree with you about the behavior, but I think you’re being A) a bit too emotional about it and B) making the exception the norm. Lots of people exhibit poor manners. Dwelling on it is just going to drive you nuts. I’ve been cut off in traffic twice in the last three days, but I don’t think that every driver on the road is an a-hole. Most people who fancy Sabermetrics are your typical run-of-the-mill people who aren’t looking to stir up any trouble.
Regarding Reynolds, perhaps he made a quote I wasn’t aware of, but the comments he made that I was talking about referred to PECOTA. If you had watched the video, you would see that Reynolds had never heard of PEOCTA until five minutes before the show aired. I doubt he’d ever read Baseball Prospectus. His rant was a steaming pile of ignorance. Yes, there’s perspective to be gained from his experience as a former player, but he has no value when talking about Sabermetrics because he’s read absolutely nothing on the subject.
Sabermetricians have already discovered that OPS is flawed because it weights OBP and SLG evenly. I forget the exact numbers, but OBP is something like 1.5 times more valuable than SLG as it relates to run scoring.
I’d rather know a player’s BABIP, his K/BB rates, and his batted ball rates than his batting average. You can’t use batting average to make judgments about future success; it merely tells you about the past. Case in point: Carl Crawford. His batting average rose every year since his rookie season, so everyone thought he’d hit .330 last season. He hit .273.
Walk rates aren’t good for just knowing how many times a hitter draws a walk. You can also infer a hitter’s place discipline. If a pitcher’s getting tired, or if the umpire has a tight strike zone, or if the pitcher loses control pitching out of the stretch, etc. then I’d go with the OBP guy over the AVG guy.
Googled it and found the OBP/SLG thing I was talking about:
http://www.beyondtheboxscore.com/story/2006/2/12/133645/296
This makes OBP about 53% more important than SLG, a fairly typical result.
I wrote about the relationship between statheads and non-statheads today, picking on the statheads a bit (I’m one myself.) We can definitely do some things to improve how people look at us and stats.
http://www.beyondtheboxscore.com/2009/6/22/919854/defending-harold-reynolds
(Hopefully this isn’t seen as spam, as it’s relevant.)