The False Fix of Redefining Metrics
Don't like it? Just change how it's measured.
Mercury simplifies your financial operations by powering them from the one thing a business needs: a bank account.* And now, Mercury offers Invoicing so you can send invoices, track what you’re owed, and get paid the way you want — all in one place.
Customize your invoices by adding your logo and preferred color scheme, view who owes you what and take action from there, and get paid via credit card, Apple Pay, Google Pay, wires or ACH.
(And that’s good news… because while I won’t judge a book by its cover, as a SaaS CFO, I’m a bit of an invoice snob.)
*Mercury is a financial technology company, not a bank.
Banking services provided by Choice Financial Group and Evolve Bank & Trust; Members FDIC.
Don’t Call it a Come Back
At some point, every company wrestles with an uncomfortable metric. Whether it’s churn, CAC payback, or activation rates, sometimes the numbers don’t paint the brightest picture. And to complicate things, it gets really dark when you don’t understand why.
I find it’s this second point - ambiguity - that pushes people to redefine metrics in a misplaced search for a solution.
Take a past run-in I had with churn. When we noticed the rate creeping up, the first question was to ask why these customers weren’t sticking around. Unfortunately the answer wasn’t very clear. There was a ton of noise and no clear through line. We turned over about 17 rocks and felt no more confident (some would even argue more confused) than we were before.
What happened next?
We started looking at the metric itself, questioning whether it was really fair to count every customer who left. I mean, if Twilio only counted customers spending over $5 a month, and Asana set the bar at $5,000, didn’t we have good reason to take a similar approach?
Maybe we should only count customers spending over $2,500 a month—after all, if they weren’t spending at that level, were they really “core” customers?
And to be fair, there was some logic here! Customers spending under that threshold likely weren’t as engaged with the product. Many were tire kickers, trying out a “free to them” feature. But this approach quickly started to feel like an escape route, allowing us to avoid a harder question: why weren’t these customers getting enough value? And were we actually dealing with a churn problem, or an onboarding issue?
If these customers never made it past a first “test” order—maybe it was because they’d never received the full experience of what we offer. Excluding them from our churn calculation was applying WhiteOut to a nasty picture.
At the end of our meeting, our CEO cracked a joke that struck a little too close to the truth:
“Problem solved! Just change the definition if you don’t like the result!”
It’s one thing to adjust metrics for external reporting (to an extent…), but it’s entirely another thing to do it internally just to feel better.
What do I mean?
External metrics often need to be scrubbed to give investors a clean view of growth and stability. If excluding a few confusing edge cases will make the story clearer, AND it will make the numbers look stronger, then many CFOs will do what every good catcher does - subtly reframe the pitch as a strike over the plate.
But when we start applying these “framed” metrics internally, we risk making operational decisions based on a skewed reality.
Said differently, it’s like if you were to avoid asking your pitcher to go back and practice actually throwing f’ing strikes so you didn’t have to frame.
The Slippery Slope of External vs. Internal Metrics
It’s easy to understand how this happens—and sometimes it’s not even intentional. Companies look to benchmarks from leaders like Twilio or Asana, assuming they’re taking the “right” approach to metric definitions. After all, if a major player is counting only customers who reach a certain threshold, doesn’t that make sense for us too?
The problem is, just because a benchmark makes sense for one company doesn’t mean it translates to others. For instance, Twilio might filter for higher-spending customers because their platform usage reliably scales with customer success. For us, though, that wasn’t necessarily the case. Instead of being a meaningful metric, the benchmark became a convenient excuse—a way to feel better about churn without really tackling the root causes.
The Real Fix: Digging Deeper
If a metric is consistently coming up short, here are a few ways to approach these uncomfortable metrics without resorting to shortcuts:
Get granular with your data: Segment your customers within the metric. If churn is high in one specific group, like new users or smaller accounts, dig deeper to see if you can spot trends in their journey. Drill. Down.
Focus on the journey, not just the result: Sometimes what looks like a [churn] issue is actually an [onboarding] or [product education] problem. Metrics tell you what is happening, but it’s up to you to uncover why.
Benchmark with purpose: Instead of picking numbers based on what other companies are doing, consider what thresholds or spending levels are meaningful to your unique business model. Benchmarks should inform—not justify—your approach.
Moving Forward
Next time a metric tempts you to redefine it just to feel better, take it as a signal to roll up your sleeves.
Real progress doesn’t come from adjusting the scoreboard. And sometimes, the metric you’re tempted to tweak is the very key to that transformation, challenging you to go beyond surface wins and commit to the work that turns short-term metrics into lasting momentum.
(And if all else fails, just make your footnotes really small)
Run the Numbers
Apple | Spotify | YouTube
I interviewed a +25 year vet of HarbourVest, one of Boston’s preeminent private markets firms. Scott Voss and I discussed:
Succession planning at Venture Capital Firms
The rise of continuation vehicles and side car funds (and how they work)
How much of private market investing is an access game vs a diligence game
This episode had perhaps the most signal per minute if you want to learn about the private tech landscape, and the structures people use to invest.
Quote I’ve Been Pondering
“It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so.”
-Mark Twain
So well said! I've gone through these battles myself. Trying to convince leaders there's no value in lying to themselves.