The False Fix of Redefining Metrics

At some point, every company wrestles with an uncomfortable metric. Whether it’s churn, CAC payback, or activation rates, sometimes the numbers don’t paint the brightest picture. And to complicate things, it gets really dark when you don’t understand why.

I find it’s this second point - ambiguity - that pushes people to redefine metrics in a misplaced search for a solution.

Take a past run-in I had with churn. When we noticed the rate creeping up, the first question was to ask why these customers weren’t sticking around. Unfortunately the answer wasn’t very clear. There was a ton of noise and no clear through line. We turned over about 17 rocks and felt no more confident (some would even argue more confused) than we were before.

What happened next?

We started looking at the metric itself, questioning whether it was really fair to count every customer who left. I mean, if Twilio only counted customers spending over $5 a month, and Asana set the bar at $5,000, didn’t we have good reason to take a similar approach?

Maybe we should only count customers spending over $2,500 a month—after all, if they weren’t spending at that level, were they really “core” customers?

And to be fair, there was some logic here! Customers spending under that threshold likely weren’t as engaged with the product. Many were tire kickers, trying out a “free to them” feature. But this approach quickly started to feel like an escape route, allowing us to avoid a harder question: why weren’t these customers getting enough value? And were we actually dealing with a churn problem, or an onboarding issue?

If these customers never made it past a first “test” order—maybe it was because they’d never received the full experience of what we offer. Excluding them from our churn calculation was applying WhiteOut to a nasty picture.

At the end of our meeting, our CEO cracked a joke that struck a little too close to the truth:

It’s one thing to adjust metrics for external reporting (to an extent…), but it’s entirely another thing to do it internally just to feel better.

What do I mean?

External metrics often need to be scrubbed to give investors a clean view of growth and stability. If excluding a few confusing edge cases will make the story clearer, AND it will make the numbers look stronger, then many CFOs will do what every good catcher does - subtly reframe the pitch as a strike over the plate.

But when we start applying these “framed” metrics internally, we risk making operational decisions based on a skewed reality.