Support for MM comes from Tropic, a cost-savings platform that automatically finds ways for businesses to reduce and control costs. By combining procurement workflows, supplier management, and benchmark data together in one place, Tropic makes savings opportunities easy to find and act on.
Before we get to one of the most tactical drops in Mostly metrics history…
So what exactly does the Fed do? How do interest rates impact the decisions that tech companies make? I sat down with Kyla Scanlon on Run the Numbers to talk about the “Vibecession” we find ourselves in, and what we think JPow eats for lunch.
Your company is only as healthy as your Churn says it is. That’s why I’ve asked my friend Olga Berezovsky, a data scientist and analytics expert, to teach us all the different ways a customer can leave, and help us avoid common mistakes when identifying and measuring Churn.
Churn has been a painful subject for many founders across all domains and industries this year. In 2023, users don’t retain as well as they did in 2019 or 2021. It’s an excruciating time to face the new churn baselines and accept that the cost of retaining a user today is higher than it used to be.
Whenever I calculate and report churn, I have to step back to understand the entire business model and its product offering. Churn rates for B2C, B2B, and SaaS differ significantly. The revenue models vary - and considering one-off transactions, recurring transactions, and freemium models are crucial for defining churn. You can’t simply apply or copy the churn reporting technique from one business to another, which makes using benchmarks so dangerous.
Analysts often refer to churn as any loss, such as cancellations, product abandonment, downgrade, or even attrition. However, each of these metrics require a different approach for reporting and analysis.
There are so many variations and flavors of churn metrics:
Customer churn
Customer churn rate
Customer retention
Customer retention rate
Net customer retention
Net revenue retention
Gross revenue retention
Revenue churn
Net Dollar churn
Gross Dollar churn
Logo churn
And many more. Some definitions may overlap, while others are more relevant to specific businesses than others.
What’s the difference between customer churn and revenue churn?
When analyzing churn, you have to differentiate between customer and revenue churn. For example, for subscriptions:
Customer Churn refers to subscribers who either cancel or downgrade their subscription plans.
Revenue Churn is the $ loss associated with customers who discontinue or terminate their subscriptions.
*If one customer can activate multiple products or subscriptions, you also would need to report Subscription churn as the total number of subscriptions deactivated.
In a nutshell, high customer churn doesn’t necessarily mean high revenue churn, and vice versa. This distinction is why LTV and ARPU exist. Churn itself won’t give you a fully comprehensive business overview, and needs to be evaluated alongside other metrics.
Regarding KPI ownership, data scientists or analysts are less likely to own revenue churn reporting or any revenue metrics. Typically, it falls under finance teams and accounting (thankfully!). Nevertheless, monitoring subscriptions and customer metrics should remain a priority for every analyst.
What’s the difference between voluntary and involuntary churn?
The voluntary vs involuntary churn concept is relevant for B2B and SaaS businesses measuring paying customers:
Voluntary churn means customers proactively cancel a subscription or end the contract. It was their decision to abandon the product.
Involuntary churn includes customers who fail to renew a subscription because of external factors, such as expired subscriptions, failed purchases, expired credit cards, service outages, and product bugs:
Some of these cases may potentially catch up (hopefully) through grace periods and billing retries (depending on the bank and payment service). However, these will likely be reflected in your next reporting period (e.g., next month), possibly as new subscriptions rather than renewals.
This is important because churn calculations are based on renewals, not new subscriptions. So even if customers return and your revenue remains unaffected, the involuntary churn rate may remain high.
For B2B SaaS, what’s the difference between churn and cancellations?
Churn is not cancelations. Churn includes most cancelations, but usually not all.
In most subscription platforms, users can set the end date of their subscriptions - e.g., end in 3 months for monthly plans, or the end date is “04-10-2024” for other subscription plans. So, these users never have to proactively cancel the subscription. It will be auto-ended for them. These users won’t show up in your churn metric if you build it on top of cancellations.
Also, remember that users may cancel before they are scheduled to renew their subscription. The customers might cancel but not churn yet. They don’t churn until the end of their payment period, which could be a few months ahead.
Why this is important is if users cancel but keep having access to your product and are still using your premium features for a time, you can potentially win them back.
For the same reason, with subscription-based products, I prefer to differentiate churn from a downgrade. I do not consider the downgrade to a cheaper plan as churn because:
You’re still making revenue from these users, and
There is a higher chance to win them back than a customer who churns entirely.
But many other analysts do. Again, every business churn can be different. So be cautious when using industry benchmarks.
In summary, churn includes both voluntary and not voluntary concepts. Users might churn without actively canceling their subscriptions. If you define churn based on cancellations, you will likely under-report it.
Should product and finance teams define it the same way? How might their view points differ?
From my standpoint, finance and product teams should have different definitions and perspectives on churn:
The finance team aims to oversee and analyze the company's financial strength. This comes down to granular, detailed, and segmented revenue, sales, and spending reports to oversee the business's health across various departments such as product, marketing, brand, engineering, customer support, and more. Measuring users who stop paying is crucial for finance teams.
On the other hand, the product team's goal is to build the best product that users will love, regardless of what % of them pay (or if any pay at all). Remember - in many B2C subscription-based products (e.g., Duolingo, Calm, Strava, etc.), only a small % of total users typically become paid customers. Product reports focus on understanding all user behavior and actions paid and not paid.
The role of the BI team is to bridge these two completely different worlds into one. They bring together and synthesize the context from finance and product perspectives.
For example, let’s say the analyst has to report the impact of a newly released premium feature to measure if it retains users better.
The product teams often don’t have the luxury of waiting a month (or a year) for a new subscription to renew before reporting on churn. And because churn includes an involuntary aspect, it won’t reflect in the new feature impact analysis anyway - unless there is a notable strong lift or decline, which is a rare effect.
Financial KPIs, like ARR or Churn rate, are ecosystem metrics. They are not designed to be sensitive because they are output metrics meant to safeguard reporting from fluctuation or noise. However, the product team must pick up these tiny noises in data to analyze the impact of a product release or a new feature adoption. That’s why analysts use clicks, page opens, and transactions, which are good at capturing immediate user responses, thus offering a more nuanced view of specific actions or behaviors. For this reason, the product team should use cancellations as a product metric and a proxy for churn to measure early user response to a new feature rollout.
Is churn always the opposite of retention? Are there any times this is not the case?
For B2B and SaaS,churn IS retention - it’s just inverted. But it’s the same metric. Churn shows users who failed to renew during the period, while retention reports on those who succeeded in renewing. It doesn't matter which metric you choose, as both will show the same movement (unless different groups of users are due for renewal in a given period, in which case retention will return different data, though the overall story remains the same).
Both churn and retention can be used for reporting users, subscriptions, and revenue. And both can be cohorted.
I commonly notice some flawed thinking that churn should be used for subscriptions, while retention is for paid customers or users. That being said, both churn and retention are used to measure:
revenue,
subscriptions
users
Both can be used to report raw numbers and rates. Both can be used for reporting Net and Gross.
As you know, CJ, regardless of what we “officially” send to the board or gods, as a team, we look into all levels and segmentation of churn and retention reporting: customer, subscription, and revenue.
For B2C free-to-use platforms with ads or one-off transactions (e.g., Meta, Airbnb, TikTok, eBay, etc), the story is different:
There is no actual “churn” term:
Typically, analysts use product abandonment, deleted accounts, deactivated profiles, or uninstalls to gauge total loss (because there are no “subscriptions” for such products, revenue reporting becomes a very different beast).
Retention becomes a different metric measuring user activity instead of paid tenure:
In such cases, retention is not the opposite of churn but simply different:
In B2C, retention speaks of your product's ability to keep users active over time. Read more: How to measure cohort retention.
What are some ways companies underestimate churn?
Mistake 1: Including new subscribers in the churn calculation.
New users, by definition, can’t churn within their first billing cycle, so they should be excluded from the denominator in your churn rate formula. They might cancel, but even then, new users will still have access to the premium features until their subscription expires (as is the case for many SaaS products). If you include new subscribers in your churn rate formula, you will underestimate churn.
Mistake 2: Including trials and promo codes in the churn calculation.
I remember working with a team a few years ago that counted trial users among the paid customers when calculating the churn rate and net new subscriptions. Their rationale was to measure how many users have access to the premium features and how many of them actually use premium offerings to develop a growth strategy based on feature utilization data. They succeeded in remarkable growth and engagement, but their churn was significantly understated, which caused bigger issues with revenue not meeting forecasts.
The trick is that there is a big difference between having access to premium vs paying for premium. Trials and freemium users should be excluded from SaaS waterfalls, as they can significantly pollute rates. For example, the trial activation rate can be high and stable, but the conversion rate from trials to paid subscriptions can be low or, even worse, fluctuate based on seasonality, app versions, and product experimentation, causing M/M variations.
Mistake 3: Mixing up monthly and yearly plans
I borrowed this insight from one of my favorite SaaS bloggers, Christoph Janz. In his 9 Worst Practices in SaaS Metrics, he emphasizes that you should not combine monthly and annual plans together in the churn formula. Since annual users do not renew every month, including them in the denominator would underestimate churn.
Merging different subscription types into one churn report is very common. It’s often referred to as “blended” churn (or “blended” retention). It serves as an indicator of overall subscription health, but I do not recommend it. The ratio between monthly and annual users can differ, so their actual churn rate patterns will be hidden. You won’t be able to tell whether your churn is increasing, decreasing, or remaining constant over time.
What are the six segments you should report on for any given month?
Many companies develop subscription waterfalls using Beginning-of-period (BOP) and End-of-Period (EOP) values, which I know many analysts are confused with. In other words, having a starting point for the initial sale is important to initiate the BOP values for calculating any Net New metric.
The challenge I often face is that getting clean and accurate data for the initial sale, especially those done many years ago, isn’t always easy in mature companies. Even if such data exists, automating SaaS waterfall reporting becomes difficult. The initial 300 sales might not have been recorded, or maybe even came with an M&A transaction and are not integrated with the rest of the sales data, or are stored in some Excel files.
Another challenge is that BOP and EOP values work for only one subscription type, for example, annual or monthly. However, if your product offers multiple subscription tiers (e.g., weekly, monthly, annual, bi-weekly), your analyst has to replicate the waterfall for each subscription type, which is a lot of manual effort. Then, you will need to tie them together to get the total number of churned users, net new, or all paid subscribers.
Over the years, I developed my own version of the SaaS waterfall, which is slightly different from the classic BOP and EOP, and is way simpler and more intuitive to understand and work with. As I’m trying to pass it to the next generation of analysts. Check it out below:
For monthly reporting, I group customers into 6 segments:
New customers: Brand new users who subscribed for the first time in a given period.
Customers up for renewal: Paying customers expected to renew their subscriptions in a given period.
Renewed customers: Customers who successfully renewed their subscription in a given period.
Churned customers: Customers who failed to renew their subscription in a given period (both voluntary and involuntary)
Customers who are not up for renewal:Paying customers who are NOT up for renewal in a given period (relevant for businesses with different subscription length periods).
Re-subscribed customers: Returning customers who previously churned and started another subscription. This is a subgroup of the #1 New customers segment. I would like to segment it out but keep it in my waterfall to report on the impact of the winback initiatives (if any).
Once these 6 customer groups are created, SaaS reporting becomes straightforward because all these groups are connected. Having data on some of the groups allows for easier calculation of others. And, the best part is that with these 6 segments, your churn rate formula becomes very simple:
The 6 segments for subscription metrics are my recommendation to optimize your reporting and validate it to ensure trust in your data. This method saved me more than once, and I find it more intuitive and easier to adopt than the classic BOP and the EOP framework.
If you haven’t, please subscribe to Olga’s fantastic newsletter so you don’t miss any game changing data insights.
True story - I was on the horn with Tropic’s VP of Marketing last week, talking newsletter stuff and shooting the breeze. It came up that Q4 is a busy season for more than just sales teams - as a CFO, I find myself buying a lot of software. And at that moment I had a proposal from Gong on the table.
The next thing you know, we were running a price check together on the deal. Fast forward and a few drop downs later (like literally under 90 seconds), I was looking at benchmarks for what someone like me should be paying for the tool, given the package and number of licenses that we wanted.
I’ve always said that he or she who holds the data holds the power. So I went back to Gong and got another 15% chopped off.
Don’t buy before you run a check - you can run one for free today.
