Anecdotes Beat Data
Bezos: I have a saying — when the data and the anecdotes disagree, the anecdotes are usually right. It's usually not that the data is being miscollected. It's usually that you're not measuring the right thing.
If you have a bunch of customers complaining about something and at the same time your metrics look like they shouldn't be complaining — doubt the metrics.
When the data and the anecdotes disagree, the anecdotes are usually right. You're measuring the wrong thing.
The Phone Call
Bezos: We had metrics that showed customers were waiting less than 60 seconds when they called our 1-800 number. But we had a lot of complaints that it was longer. Anecdotally it seemed longer — I would call customer service myself.
One day we're in a meeting. We get to this metric in the deck. The guy who leads customer service is defending the metric. I said — OK, let's call. I picked up the phone, dialed the 1-800 number, and we waited in silence. It was over 10 minutes.
I said — OK, let's call. We waited in silence. It was over 10 minutes.
Seek Truth Even When It's Uncomfortable
Bezos: It dramatically made the point that something was wrong with the data collection. We weren't measuring the right thing. That set off a whole chain of events where we started measuring it right.
That's an uncomfortable thing to do. But you have to seek truth even when it's uncomfortable. You have to get people's attention. They have to buy into it and get energized around really fixing things.
You have to seek truth even when it's uncomfortable. Get people's attention and get them energized around fixing things.