If you have an existing product but you’re just getting started with product analytics then it is easy to get lost working out what you need to measure. It’s easy to gravitate to a vanity metric like active users or revenue but each of these are lagging indicators. When you’re starting out you need to cut straight through to the leading indicators so you can get value from your analytics fast and start influencing outcomes.
The purpose of this post is to help you quickly workout what to measure when you’re getting started.
To help you workout what to measure we’re going to walk through:
- Where teams new to analytics usually get stuck
- How to work out what to measure
- The 3 metrics to start with
There’s a bit of an assumption with this post that you already have a product up and running and you’re not planning a new product.
Where teams new to analytics get stuck
Most teams starting out with analytics head in one of two directions: only measuring lagging indicators or trying to measure everything.
Let’s unpack each direction.
Why only measuring lagging indicators fails
If you’re only measuring lagging indicators then you’re headed for trouble. You just might not recognise it yet.
What is a lagging indicator? It’s a measure that is the result of other actions and metrics. Some examples of this are: number of customers, time in app per user, revenue and daily active users. Each of them usually increases or decreases because of something else you or your product are doing.
You can start to see the problem emerging here. You may be seeing great improvements in, say, weekly active users so you all celebrate. This success hides the fact that you don’t actually understand your users. Then, suddenly, it starts to decrease and you don’t know why. Only once you dig in do you realise that you should have been measuring something closer to why people used your product each week.
Pro Tip: revenue (or even better profit) is really the main metric that matters ultimately so don’t take anything I’m saying as a message to dump thinking about this.
Why trying to measure everything fails
Some teams, in admirable enthusiasm or some directive from the top, embark on a course of measuring everything from the outset.
Endless meetings are held, multiple analytics tools are over-analysed for whether they’ll meet current and future requirements. Then an Important Project is initiated with steering committees to deliver an Analytics Platform. Systems are integrated, engineers are chosen, account reps are met with and everything seems On Track.
Then the team needs to measure what matters but they have too much data and (usually) the product/customer needs have shifted. The team becomes overwhelmed with noise.
They also become overwhelmed with recent work that has immediately become legacy – they aren’t getting the data they need in the way they now realise they need it.
So how do you workout what to measure so you don’t fall into these traps?
Working out what to measure
When you’re getting started with product analytics you can workout what to measure by following some simple tactics that you’ll need to share and agree with your team and stakeholders. These tactics are:
- Follow product principles:
- take an outside in approach,
- release early,
- mature through iteration
- disciplined prioritisation.
- Focus on 3 key areas first: your customers job-to-be-done, activation and system performance.
- Get familiar with Pirate Metrics so you know where you’re headed
The 3 Metrics to Start With
The three key areas you need to focus will need to be tailored to your unique situation. You’ll need to do some research, thinking and debating as well as some experimentation to get the right answers.
Just realise that no-one gets them right the first time, iteration is the key. Get started and then evolve.
Let’s go through each metric area and discuss how you can tailor it.
Metric: Measuring your customer’s job-to-be-done
You need to understand your customer’s core Job-to-be-Done and measure it, ideally the steps they took as well as the outcome they achieved.
This will give you a clear measure of the value you are providing to your customers.
If you don’t understand your customer’s Job-to-be-Done or know about this concept then click through the links. Going into detail is a bit beyond the scope of the post, they’re entire topics in their own right.
Metric: Measuring Activation
Now that you’re measuring value to your customer, you need to be able to measure how well you’re able to provide a new customer or user with value. Using the Pirate Metrics framework this is called Activation.
Measuring Activation is about measuring the steps involved in converting a potential new customer into a customer that has received value – achieved their job-to-be-done. Activation usually ends up as a funnel.
Metric: Performance
The performance (response times, up-time) of your product must be measured early on. This is a hidden gem that often gets lost or ignored because it can be seen as not customer facing.
You don’t need to measure everything at the start, just the performance of the functionality involved in your Job-to-be-Done and your Activation funnel.