Bad metrics rule the world around me.

The two biggest problems in decision-making:

1. Measuring the wrong metric
2. Measuring the metric wrongly

If before you make any decision, you ask yourself what metric/s you’re using to take that decision and 1. if they’re the right metric, and 2. if you’re measuring them correctly, you will always make good decisions.

By metric, I do not only mean stuff with numbers. Your customers' job-to-be-done is a metric. So is your brand perception. So are the many qualitative factors around doing a career switch that cannot be put into numbers.

Additionally, not every metric has the same meaning or can be measured in the same way across businesses or problems. Based on your product, market, and business model, the way you measure, your priorities and the way you look at the world, your metrics and the nuance behind them can vastly change.

Let's start with the nuance hidden in a super-quantitative metric like churn and move on to increasingly qualitative metrics.

Put simply, churn measures the % of customers you lose over a given period.

And like any other metric, it is wise to examine the assumptions that founders use in a churn calculation since it is possible to tell a very tall tale by misrepresenting the impact of churn.

Founder: “We have a very high churn rate, but as soon as we turn on email marketing to our user base, people will come back.”

Investor: “Yes, of course. The reason that people leave our service and don’t come back is that we have not been sending them enough spam. That makes total sense to me, too.”

The founder here is clearly an example of a person who knows that churn is an important metric but is clueless about what causes churn and how to solve for it.

Here are some nuances you need to understand before you measure or solve for churn:

1. The higher the Customer Acquisition Cost (CAC), the more important lower churn becomes. You can't have high CAC along with high customer churn. You're essentially spending big sums of money to acquire a customer who doesn't stick with you for long. It doesn't make for great unit economics.
2. Consequently, acceptable churn rate varies by business model and industry. Generally, if you're selling long-term subscriptions, your CAC will be higher but your churn will be lower as customers get tied into your product for an extended period of time.
3. But the trade-off here is that the longer the purchase cycle of the product, the longer you have to wait to measure true retention. Casper, the mattress company, faced this exact problem. A mattress has a long purchase cycle, i.e., once the customer buys it, they won't be buying it again for at least the next 10 years. To measure if customers are coming back, you have to wait for 10 years.
4. If you try to be more aggressive with your marketing strategy, not only will your CAC rise, but your churn may also rise, as a more aggressive program will likely capture customers of a lower quality.
5. For virtually all businesses, new customers will have a higher churn rate than mature customers. Some form of segmentation is necessary to have a useful churn rate.
6. Lag Time — By the time you see an increase in your churn rate, it may be six or eight months after the point in time when you actually failed the customer. Anything you do now will be too late to influence your churn.

Similarly, even measuring metrics like CAC and LTV can get complicated.

Let's take the example of Netflix, who like any subscription company, measures unit economics by just one crucial calculation: the cost of customer acquisition relative to the lifetime value of a customer.

Unfortunately, this number is incredibly hard to calculate for them, even internally!

A simple way to measure LTV/CAC is to treat product costs as fixed, see how much money is being spent on acquiring new customers and then calculate a) the annual gross profit from that customer, and b) how likely the customer is to quit the service.

But for Netflix, neither the product development cost is fixed (note that product development, too, impacts retention and LTV), nor the customer acquisition cost is clear cut as the biggest chunk of their CAC is producing/acquiring content.

And content is something that impacts both acquisition and retention. So, it's very hard to measure LTV/CAC for Netflix as the product and content decisions it makes and the money it spends on them impact both LTV and CAC metrics simultaneously.

Now, let's move on to the impact of measuring the wrong metrics or measuring them wrongly has on brand.

When it comes to intangible things like brand, the wrong metrics can create an illusionary sense of either security or ineffectiveness.

For example, content that gets the most engagement isn't necessarily the content that will help your brand long term and vice versa. I think that when it comes to brand, focusing on likes and engagement is focusing on the wrong metric.

This gets truer as your brand gets increasingly niche, and might not hold as true for mass consumer brands that are meant for everyone.

We have written more along these lines in The Personal Branding Trap.

Additionally, calculating the quality of a piece of content would also have to include factors like perceived value and memorability of the content and how coherent it is with the brand.

What is coherence? By coherence, I mean:

Does it reinforce some existing brand trait or does it come across as disconnected from the larger brand?

Apple doing topical marketing or making memes might gain a lot of engagement, but it would slowly kill the brand they've painstakingly built over the years. Similarly, Zomato doing serious informative posts would still get lots of engagement because of their existing reach, but it would look disconnected from the kind of tongue-in-cheek, humorous, and relatable content they've been putting out so far.

So, it isn't quite clear-cut if what you're measuring is necessarily a good proxy for the real problem you want to solve.

And this problem also affects problem-solving on a deeper, creative level where you might build the wrong product entirely if your evaluation of the real problem to be solved — the real job to be done — isn't correct.

Houston airport kept receiving complaints about the wait time at luggage collection for one of their routes, so they switched the gate to make the walk there five minutes longer and the complaints stopped. They realised that the real problem was boredom, not reducing wait times. A traveller would rather walk for 5 minutes than wait for 5 minutes. Also, the longer walk to the luggage collection area doesn't feel as much of a problem to the traveller as it fits their existing mental model of airports being huge spaces, versus having to wait for collecting their luggage, which comes across as an operational problem.

A similarly famous example is that of a high-rise hotel receiving complaints about their elevator being too slow. Instead of fixing the elevator speed, they decided to install a mirror to keep guests occupied. And the complaints dropped. Once again, they were able to identify the real metric to solve for: boredom.

The problem of identifying wrong metrics is even rampant in careers where people seek advice on completely wrong metrics without understanding the real problem.

As Apoorva Govind writes,

"Change doesn't always mean having to quit your job. It could be something as simple as changing the project you are working on or switching teams. People often ask how long should they stick around in a job that's not working out. In my opinion, time is the wrong metric here. Three months can be too long if you know this job isn't for you. Ten years could be too short if your objectives are being met."

As always, identifying the right metric and then measuring it correctly is key to all good decision-making. If you start with the wrong metric and seek advice on it, you're actually solving for a non-problem.

That's why getting to the root of the real problem is the shared responsibility of the person asking for advice and the person offering it.