Explaining Why Your Forecast is Wrong: 3 Possibilities

How Confluent modernized billing to unlock speed, flexibility and new monetization opportunities

As a finance leader, you’ve probably felt it: billing becomes the bottleneck long before product does. That was Confluent’s story.

Their homegrown billing system worked, but every pricing update, product launch, or enterprise deal meant weeks (or quarters) of engineering lift. In this new case study, Confluent shares how switching to Metronome cut billing execution time in half, reduced on-call burden by 25%, and unlocked pricing innovations they couldn’t deliver before.

If billing is slowing your roadmap, or even blocking new GTM motions, this one’s worth reading.

Most writers are packing it in for the holidays, squirreling away their “good stuff” for January, and just checking the (in)box.

Nonsense, I say!

Quality hath no calendar!

And I didn’t hear no bell

Here’s a freshy for the players holding it down for the underground and closing Q4 deals.

Each June when the weather warms up, I pop on over to my in law’s place on Candlewood lake in CT. Stiff from winter and pale from staring at my ThinkPad, we go wake surfing. I have a love / hate relationship with this summer past time, as I’ve been trying to land a 360 for going on 8 summers now. It pains me to think I’ve had three kids and three jobs in that time. But just like clockwork, as soon as we leave the dock, without warming up or even stretching, I jump into the water and try to grip it and rip it.

I’ve landed it to 99.7% completion at least 997 times. But close is only good in horseshoes and hand grenades.

Why haven’t I been successful? Some of it comes down to skill. Some of it’s the equivalent of a fat finger on a keyboard. But most of the margin, as my father in law likes to say, can be chalked up to “Time on the board.”

To land a 360 you’re trying to forecast where your body weight, the wave, and the boat will be, while moving 11 MPH. And it’s hard to have any consistency to a craft if you’re a tourist across different boats, drivers, and conditions.

Similarly in business, you need time on the board when you’re attempting to forecast performance.

Rama Katkar, current CFO of Notion, and former senior finance leader at Credit Karma and Instacart, told me as much on the RTN podcast.

As the old adage goes, every model will be wrong. It’s just a matter of how wrong.

But what’s the best way to explain the why?

Rama shared a three-part framework for helping finance teams understand forecasting variances. It's (mostly) mutually exclusive. And what I love about it is how it takes the "sting" or "blame" out of the equation. It reframes the discussion around how we can get better at reflecting the levers in the business.

Here’s her framework for explaining why your forecast is wrong.

There was an error in the model. And if there's an error in the model, congrats, you're not the first. A CFO once told me how they accidentally modeled their debt EBITDA covenants using subtraction rather than addition. Like a whale, they breached. I bet whatever you did is not that bad. So fix it.

Mistakes happen when you’re engaged in heroics, as John McCauley, CFO of Calendly, once told me on RTN.

So where this comes up a lot is in the early stages of company building. Your intuition around forecasting as a financial analyst, combined with the still-ramping intuitions of your product and marketing leaders, can be faulty. No one has put in much time on the board yet.

You try to build a deterministic model showing that if we use X as inputs, we will see Y as outputs.

You may even work with data science to try to connect the assumptions between finance and product. So it’s not for lack of trying (just like my 360’s).

But early on you don't have the time series data required to bridge the gap between expectations and reality. You're still learning and building that intuition collectively, and your assumptions are a little bit off… which get multiplied through a few thousand times.

Anyone who’s gotten their product’s average deal size wrong knows this.

While it hurts, this type of miss is actually valuable; it shows you exactly where to refine your assumptions and strengthen the model next time.

The third type of forecasting mistake is an operational decision.

Often you set a plan (Rama said her team does full year modeling but sets most of the operating plan, at least in engineering and product design, for the half). You make a decision that you thought you were going to launch XYZ feature on this date. But then you don't feel comfortable with the product quality and you pull back.

That's obviously going to impact the revenue in your model because you pushed it out. But that's an operational decision, not a mathematical one, and it's the right decision for the business.

You may also do something that increases revenue or expenses, such as opportunistically launching a sales team in Asia after your customers pull you there. Also not in the model. But also not the “wrong” thing to do.

The issue at hand is not really if your model is wrong (it is); it's “would I make the same decision next time?”

If you would, then you are still doing the right thing.

The best part about this framework is it allows you to identify the cause, without attacking the person.

I’ve personally made a mistake by going at someone who “caused” the error, like the CMO, who totally whiffed on the number of registrations we expected to get from a big conference investment… which was a key input to our sales pipeline… and resulting revenue forecast for Q4.

Fuming, now I’m stuck trying to plug a revenue gap and a CAC Payback period that’s upside down.

Stepping back, the reality was we didn’t have enough time on the board as a management team. We were all less than a year into our respective roles. And I relied on naive assumptions that produced a bad output (a level 2 error). There weren’t any “bad people involved”. Just not enough conference seasons (or summers) on the board.

And as the business matures, the level of rigor you can take to your forecasting increases. For one, you have a larger revenue + customer base to spread your error tolerance over. Salesforce is really great at forecasting because they have 150,000 companies and a revenue base of over $50 billion per year and like 20 years of trends. The surface area to hide a forecasting error is much wider than a Series B company launching their second product.

With scale (and practice), the wave starts to feel less like something you must combat, or overcome, and more like your playground. You go from trying to merely stay in the sweet spot where you can feel the propulsion to exploring the crest, sides, back, and front of the wave.

And at that point, you’re landing 360’s at a predictable clip. You’ve had time on the board (and invested in a Mastercraft x24 with hydraulic steering).

Brian Brown, CFO at Rocket Companies (the folks behind Rocket Mortgage and Redfin) joins me to demystify the company’s business model and how it supercharges the American home buying experience.

On this episode we discuss:

Rocket is a staple of the American home buying experience and this was a bucket list conversation with a thoughtful leader (and amazing storyteller).

Kyle Poyar and I are joined by Des Traynor, Intercom’s Co-founder and Chief Strategy Officer, to discuss their outcome based pricing model, and why customer marketing has just morphed into VC thirst traps.

We also discuss:

Wishing you stop setting VC thirst traps,

CJ