A common mistake when defining metrics is beginning the process by asking “what metrics do I need?” A “metrics first” approach results in (an endless) list of metrics that takes you out of focus.
In contrast, a “questions first” approach helps you first identify the questions you want answered by the metrics, and only then look for the appropriate metrics, even if “you don’t know anything about metrics”
Sidenote: many people think that because they don’t know, or aren’t familiar with the name of some metrics they don’t know about metrics.
But there’s nothing further from the truth: you don’t need to know that there’s something called ROI (return on investment) to want to compare the benefit you got out of a specific investment in time/effort/money, like how well a certain design performed when compared with the effort (time, resources) it took to produce.
And you don’t need to know that there’s a metric called CAC (customer acquisition cost) to want to know how much it costs to acquire new customers.
The former (ROI or CAC) are technical definitions, the latter are the practical or intuitive definitions. Ultimately, the intuitive definition is more important, because the point is to know what you are trying to measure regardless of its technical name. In fact, different people may use different names for the same metric.
But going back to the main point… I came up with the “Questions first, data later” approach after seeing many teams struggle with the “data first” way of working.
Questions first
Begin the process by thinking about the questions you want to answer. This allows you to:
- Choose the relevant metrics
- Avoid analysis paralysis
- Make better sense of the metrics you choose
- Identify your data sources (not everything is in Google Analytics).
Data later
Once you know what questions you need to answer it’s time to think about metrics, even if “you don’t know anything about metrics”.
And when you start to think about the metrics, one thing that can help you is to think about the “core actions”, which are the main actions or activities that a user needs to do to complete the outcome you are trying to measure. Once you figure out the core actions, the metrics flow effortlessly.
An example
If we wanted to answer the question “How do we know if users are interested in our product offering?” possible core actions would be:
- Users explore the site and products
- Users go to product pages
- Users spend time on product pages
And if we translate core actions into metrics, we could define the following metrics:
- Product pages/visit
Note here that we are only considering product pages, not just any page, because we are focusing on the core action. Counting all pages per visit would not help us answer the question. - Time on page (as supporting metric)
- Scroll (as supporting metric)
- Navigation Flow (to see if visitors explore different products and categories)
Conclusion
- In the process of defining performance metrics, the first thing we need to do is understand what questions we want to answer with those metrics.
- Once we have the questions, the next thing is to define the core actions that would answer those questions.
- And once we define the core actions, simply translate them into metrics.
- It is OK to use the “intuitive” definition of the metric if we don’t know the technical name. What matters is that we have the metric, not how we call it.
You can download the “Questions first” template here.
If you use it, I’d love to hear how it worked for you!
A common mistake when defining metrics is beginning the process by asking “what metrics do I need?” A “metrics first” approach results in (an endless) list of metrics that takes you out of focus.
In contrast, a “questions first” approach helps you first identify the questions you want answered by the metrics, and only then look for the appropriate metrics, even if “you don’t know anything about metrics”
Sidenote: many people think that because they don’t know, or aren’t familiar with the name of some metrics they don’t know about metrics.
But there’s nothing further from the truth: you don’t need to know that there’s something called ROI (return on investment) to want to compare the benefit you got out of a specific investment in time/effort/money, like how well a certain design performed when compared with the effort (time, resources) it took to produce.
And you don’t need to know that there’s a metric called CAC (customer acquisition cost) to want to know how much it costs to acquire new customers.
The former (ROI or CAC) are technical definitions, the latter are the practical or intuitive definitions. Ultimately, the intuitive definition is more important, because the point is to know what you are trying to measure regardless of its technical name. In fact, different people may use different names for the same metric.
But going back to the main point… I came up with the “Questions first, data later” approach after seeing many teams struggle with the “data first” way of working.
Questions first
Begin the process by thinking about the questions you want to answer. This allows you to:
- Choose the relevant metrics
- Avoid analysis paralysis
- Make better sense of the metrics you choose
- Identify your data sources (not everything is in Google Analytics).
Data later
Once you know what questions you need to answer it’s time to think about metrics, even if “you don’t know anything about metrics”.
And when you start to think about the metrics, one thing that can help you is to think about the “core actions”, which are the main actions or activities that a user needs to do to complete the outcome you are trying to measure. Once you figure out the core actions, the metrics flow effortlessly.
An example
If we wanted to answer the question “How do we know if users are interested in our product offering?” possible core actions would be:
- Users explore the site and products
- Users go to product pages
- Users spend time on product pages
And if we translate core actions into metrics, we could define the following metrics:
- Product pages/visit
Note here that we are only considering product pages, not just any page, because we are focusing on the core action. Counting all pages per visit would not help us answer the question. - Time on page (as supporting metric)
- Scroll (as supporting metric)
- Navigation Flow (to see if visitors explore different products and categories)
Conclusion
- In the process of defining performance metrics, the first thing we need to do is understand what questions we want to answer with those metrics.
- Once we have the questions, the next thing is to define the core actions that would answer those questions.
- And once we define the core actions, simply translate them into metrics.
- It is OK to use the “intuitive” definition of the metric if we don’t know the technical name. What matters is that we have the metric, not how we call it.
You can download the “Questions first” template here.
If you use it, I’d love to hear how it worked for you!
