But "sanity is not statistical," as our friend, Winston, remarked in George Orwell's book 1984.
What this means is that we cannot rely only on statistics to govern how we do things. We need to balance it with human intelligence (and spirit). Too often I hear horror stories of client requests to "measure everything!" and even more often nothing significant comes of it.
But if statistics can provide us with so much relevant information, where exactly does it fall short?
1. Show me the money.
In the book Super Crunchers, the author talks about a company that uses statistics to determine which movies will be huge hits - by only reading scripts. The accuracy is astounding and some movie studios use this method to determine not only which films to make but also to get suggestions on how to improve to make the movie more profitable. Pretty cool, right?
But then I think about all the terribly unsuccessful movies that I love. I can't imagine if movies like 'Army of Darkness' or 'Fight Club' were never made. Just because a movie bombs in the box office doesn't mean that it's not good. On the flip side, just because a movie makes millions of dollars, it doesn't mean it is good (Twilight anyone?).
2. There is no social media 'To Do' list.
Unfortunately, measurement seems to be something that people like to have but then do nothing with. It's as though they think, "Social media measurement? Check. Okay, item number two...Pick up dry cleaning."
The purpose of measurement is to find out where we can improve, what we can do better and what is working. There is no point in measuring something if you aren't going to do anything about it.
3. It's creepy and unfair.
My husband recently received an e-coupon from AIR MILES for yogurt. Well, not just ANY yogurt. The exact brand of yogurt we buy on a weekly basis. They know this because when we buy it we swipe our AIR MILES card to get points.
Some of you may think that they are providing us with a customized experience. But I know that they did a very complicated algorithm to figure out the minimum discount they could give us based on our purchasing habits. That means that there is probably someone else out there who got a discount for the same yogurt but their discount was for 25% more (or less). How fair is that? If I went into a store and they charged different amounts for the same product, no one would stand for it.
It's also big brother-ish. I don't want AIR MILES (or anyone) to be that aware of my purchasing habits. It actually makes me not want to use my card when I purchase yogurt or to start buying a different brand. But they probably have enough data to predict that.
4. It encourages organizations to ignore the outliers.
I attended a Data Mining forum where a speaker was talking about how he developed a system for a bank. The system enabled tellers to immediately identify who was a high value customer (HVC) just by pulling up their profile. The purpose was to let tellers know instantly who they should be giving exceptional customer service.
It reminded me of when I was working as a dealer at the casino in Niagara Falls Ontario. We would see all sorts of customers, high value and low value. But, as dealers, we were trained to treat them all the same. Each one of them was to get exceptional customer service. While the casino had other ways of rewarding HVCs, increased customer service was not one of them. Every person that walked through our doors was to be treated like an HVC.
The bank used measurement to focus only on the very successful parts of their business, while ignoring what they should improve. Instead of using statistics to decide who should get the best customer service, perhaps they should be figuring out why a person isn't an HVC and how to convert them. I can be certain that providing exceptional customer service is just a start.
5. It's about people.
While it's obvious that companies need to start looking at how they do things based on statistics (not just intuition), it's easy to fall into a false sense of security. It's easy to rely on statistics to tell us the best way to do things or that things are just being done. But we need to remember that a good measurement strategy starts with good questions and questions come from intuition. Statistics can give us the answers we need, but we need people to act on those answers.