Beware of all green dashboards
The danger of using metrics as proxies for truth
My friend Niklas texted me “yo, Jeff Bezos is talking about metrics on the Lex Friedman podcast.”
Like a moth to a flame, I tuned in.
Bezos observes how over time companies find themselves blindly managing to a metric. They lose connection with the underlying truth the metric is supposed to serve as a proxy for.
A proxy, in my simple mind, means a “stand in” or “representative” for someone or something else. In this case, the metric is a representative for an underlying theme in the business - like keeping customers happy… or expanding wallet share… or making sure employees like working for you.
I’ve time stamped the clip and quoted it below:
“One of the things that happens in business, probably anything where you have an ongoing program and something is underway for a number of years, is you develop certain things that you’re managing to.
The typical case would be a metric, and that metric isn’t the real underlying thing. And so maybe the metric is an efficiency metric around customer contacts per unit sold or something like that. ‘If you sell a million units, how many customer contacts do you get or how many returns do you get?’
And so what happens is a kind of inertia sets in where somebody a long time ago invented that metric and decided,
“We need to watch for customer returns per unit sold as an important metric.”
But they had a reason why they chose that metric, the person who invented that metric, and decided it was worth watching. And then fast-forward five years, that metric is the proxy.
The proxy for truth.
Let’s say in this case it’s a proxy for customer happiness, but that metric is not actually customer happiness. It’s a proxy for customer happiness.
The person who invented the metric understood that connection.
Five years later, a kind of inertia can set in and you forget the truth behind why you were watching that metric in the first place…
And so you’ve got to constantly be on guard and it’s very, very common. This is a nuanced problem. It’s very common, especially in large companies, that they’re managing to metrics that they don’t really understand. They don’t really know why they exist, and the world may have shifted out from under them a little and the metrics are no longer as relevant as they were when somebody 10 years earlier invented the metric.
Inertia is a bitch. It’s easier to go along with the way things are, and continually buy into a metric that’s already been vetted and gained valuable social approval, rather than upsetting the apple cart.
Note: I’ve always thought that’s a stupid saying - I’ve never witnessed anyone violently assault an apple cart.
So it begs the question - why don’t we want to call BS on a metric?
Why don’t I want to call your metric ugly?
Other than laziness, the biggest reason people don’t want to change the way they measure the business is social discomfort. No one wants to be the buzz kill during a Friday business review who basically says, “Hey you know what we should do? Gut the kitchen we built in 2011 (which has a perfectly good working stove and a decent quartz counter top) and put in some new cabinets.”
But enduring businesses are comfortable having those uncomfortable conversations. In fact, they’re necessary for survival.
Lex asks:
What does it take to have a culture that enables that in the meeting? Because that’s a very uncomfortable thing to bring up at a meeting…
Bezos jumps in:
You have just asked a million-dollar question. So if I generalize what you’re asking, you are talking in general about truth-telling and we humans are not really truth-seeking animals. We are social animals.
Let’s take you back in time 10,000 years and you’re in a small village. If you go along to get along, you can survive. You can procreate. If you’re the village truth-teller, you might get clubbed to death in the middle of the night.
Truths don’t want to be heard because important truths can be uncomfortable, they can be awkward, they can be exhausting.
They can make people defensive even if that’s not the intent.
But any high performing organization, whether it’s a sports team, a business, a political organization, an activist group, I don’t care what it is, any high performing organization has to have mechanisms and a culture that supports truth-telling.
One of the things you have to do is you have to talk about that. You have to talk about the fact that it takes energy to do that. You have to talk to people, you have to remind people, “It’s okay that it’s uncomfortable.”
Literally tell people, “It’s not what we’re designed to do as humans.”
This is by no means a newsletter about social norms and the psychology of belonging. I will leave that for people with liberal arts degrees and yogi’s with better hip flexibility.
But measurement is, at it’s core, a form of truth telling. And a CFOs job is to surface signals on the business - good, bad, or indifferent - regardless of the social pressures to shuffle things under the rug.
If you are the one who changes a retention formula, and the result isn’t great, the board may, in the words of a wise CEO I know:
“Place you in the center of the room and beat you with pool noodles until they feel better about churn.”
And no one wants to be beaten with a purple flotation device.
Be a buffalo - run towards the storm
Buffalos are the only mammal who, when they detect a storm, run directly INTO it.
Seems counterintuitive, right? Not really - they know that by running towards the storm, they are able to get through it faster. And they also minimize their exposure to the most dangerous, and longest, parts of the storm, like the leading edge, where the winds are the strongest.
A lot of us are not buffalos when we look at metrics. We get a gut feeling that something is off - maybe the anecdotes we are hearing around the office aren’t matching the signals coming from the numbers. We feel like the earth may have shifted below the metrics we were using as proxies.
This disconnect is the sign of a storm brewing. But rather than investigating further, and running towards the storm, we often shy away, and move in the opposite direction.
Afterall, “Dave from Strategy & Ops” really likes that metric. And I like Dave. And Dave will help me get promoted.
But as a result, the storm earns more time to gather force, and we wake up one Wednesday in October and discover we have a leaky bucket.
Also, Dave no longer works here.
Beware of all green dashboards
Someone once said to me,
“Beware of all green dashboards.”
I think what they meant was,
“There’s probably something you are not measuring if all your metrics are saying things are perfect. Because running a startup is hard.”
Being in a storm sucks. But it’s better to go through a category 2 today rather than a category 5 tomorrow. And that starts with saying the uncomfortable truths.
Some suggestions:
The most senior person in a meeting about metrics speaks last
Removes the social pressure to agree with what someone higher than you said
The KPI dashboards leadership teams rely on are revisited and stack ranked on a quarterly or annual basis
This creates a set time to change the metrics you measure towards, so calling someone’s metric “ugly” doesn’t come out of left field
Reports are sent on an automated basis
Once you lock in your numbers to report on, schedule their reporting so disconnects surface in an automated way - that way an individual doesn’t have to surface it manually and attach their name to the bad news.
Yes, we want all green dashboards, but ironically, having them isn’t a good way to get there.
A word from our sponsor Brex
Out with the old — manual, time-consuming expense processes — and in with the new — AI-powered, automated spend management.
Brex is how finance people get non-finance people to spend smarter. Their AI-powered cards + payments + software approach is more intuitive and scalable than anything out there, and empowers employees everywhere to make better $$$ decisions and finance leaders like you to close the books faster.
Discover how smart cards, automatic receipts, automated approvals, global travel, and real-time visibility in one place boost accuracy, efficiency, and compliance. It’s one total spend platform and a universe of benefits.
Not sick of me yet? Let’s Run the Numbers
Me and fellow CFO Kurtis Hanni answered reader questions including:
What’s your relationship with money?
What keeps CEO’s and CFO’s up at night?
How do you get teams to buy in on a metric’s definition?
Can either of us dunk? (spoiler alert: no)
Spotify:
Apple:
Quote I’ve Been Pondering
I read somewhere that a photo of Nikita Khrushchev’s historic shoe-pounding incident at the UN revealed that he was still wearing both his shoes. A third “for-pounding-only” shoe? That’s calculation.
-What They Don't Teach You at Harvard Business School by Mark H. Mccormack
Nice article. Great insights. Although I believe it's the bison the one that runs against the storm, not the buffalo!
Love this, CJ. It's critical to draw a distinction between what's factual (this metric is X) to what's true (that means Y). The former is an output, the latter is a narrative that we can derive action from.