The Dangers of Vanity Metrics

If Pied Piper fell victim to vanity metrics, so can you

Vanity metrics are dangerous.

They might make you feel all warm and fuzzy about your business today…

But focusing on them will KILL you in the long run.

I asked 13 of the smartest CEOs and Investors I know for examples of vanity metrics that you should AVOID at all costs.

But first off - what are vanity metrics?

Vanity metrics are numbers that make you look good, but have no material impact on your ability to make good decisions and improve your business.

They are superficial and fail to drive durable revenue growth.

Per Ben Yoskovitz, Founder and Managing Partner of Highline Beta and writer of the superb Substack Focused Chaos:

A good metric has 4 key qualities:

❝

Understandable: A good metric is one that’s easy for everyone to understand and track. That allows it to be part of the company’s common language.
Comparative: A good metric allows us to compare things over periods of time to see trends. This is often what we think of when we talk about cohort analysis. For example: Active Users vs. Active Users/Month. If I tell you I have 10,000 active users, it’s difficult to know if that’s good or bad. If I tell you that last month I had 1,000 active users, that’s a 10x increase, and that looks pretty good!
Ratio / Rate: If you take a comparative number and then turn it into a ratio or rate, it becomes even more valuable. Using my example above, instead of Users/Month, I should track % Monthly Active Users. So last month I had 1,000 active users out of 2,000 that joined my platform, which is 50% of monthly active users.
Behavior changing: We already covered this above, but as a reminder: a good metric is one that you use to make decisions. Imagine looking at a metric and thinking to yourself, “If this goes up, stays the same, or goes down, I don’t know what I’d do differently.” ← stop focusing on that metric.

So with that context, let's look at some examples of dangerous vanity metrics:

1/ Daily and Monthly Active Users

From talking to Thiel Fellow and vertical SaaS founder Luke Sophinos (@lukesophinos), DAU and MAU are pretty useless without supporting information or specific rules that allow you to quantify them.

For example, if someone logs into an app every month and doesn’t use it for a specific purpose, then it’s pretty irrelevant.

They must be performing actions linked to the core product’s value to count.

2/ Net Churn

Per Niv Thanabalan (@itsnivt), the CEO of Integral Insights, watch out for subscription or membership based companies that only cite their Net Churn.

If you can't tell exactly how many gross customers are dropping out, you may be masking a larger problem through your client acquisition pace.

3/ Cumulative Metrics

From talking to everyone's favorite Fabio looking CFO, @OnlyCFO, cumulative metrics provide the prettiest looking charts but are among the worst in deriving business insights.

It's near impossible to get much from a chart that can't go down period over period.

It pains me to say that @asmartbear recently pointed this out for my favorite newsletter platform:

— # (#)

4/ Funding Raised

I asked Ben Yoskovitz (@byosko) the Founding partner of of Highlight Beta:

❝

"Is the amount of capital a founder has raised for their startup a vanity metric?"

His reply:

❝

"100% YES."

Why? The amount you raise doesn’t drive learning.

It might give you a chance to run more experiments and learn from them, but usually it leads to spending on stuff you don’t really need.

In other words, money can buy you time.

But it can't buy you execution.

Vanity Metrics: The Numbers We Hate to Love

If vanity metrics are bad, what makes a good metric? How do I know I'm tracking the right things? (#12)

www.focusedchaos.co/p/vanity-metrics-the-numbers-we-hate?utm_source=substack&utm_campaign=post_embed&utm_medium=web

5/ ESG Scores

From talking to Antonio Reza (@theantonioreza), Finance leader at Google, Environmental and Social Good (ESG) scores are potentially even more misleading than Net Promoter Scores (which we'll hit on later)

ESG scores are hard to understand (is 70 good?), you don't really know how they were derived (was it through a standardized questionnaire?) and they don't immediately translate to revenue, or really any operating metric.

But maybe you'll get a pat on the back?

I'm sure this will upset a few people, though.