🧪 The Growth Experiment Process
Jun 08, 2021 • 5 min • Growth
Growth doesn't fit nicely into one part of the funnel like a product or marketing team might. Instead, growth spans many parts of the funnel and can mean lots of different things depending on the company. But there are some commonalities worth talking about, notably an experiment-driven approach. I referenced this idea previously in What it Means to Work on Growth:
One commonality across growth tasks is an experiment-driven approach. The best teams quickly and effectively prioritize the work that will move their north star most. They develop hypotheses, test them, and evaluate results. This is why Steven Dupree described growth as the scientific method applied to KPIs, which I like a lot.
Growth teams rapidly iterate and run experiments while more traditional product teams work on larger feature developments and initiatives. Once you accept this, the natural next question is, "How does this experimentation process work?"
🔬 Step 1: Quantitative analysis a.k.a "The What"
First things first, you need to understand the answer to the seemingly simple question, "How do we grow?" This seems straightforward, but most companies would be surprised at the lack of alignment if they surveyed a random sample colleagues. If you don't have a whiteboard-level quantitative model of how your product grows, start there. Break down your user acquisition channels, activation rate, and long-term retention curve.
This allows you to identify the things that are worth working on. It gives you an intuition for the highest leverage points in your model. Especially at smaller companies where resources are limited, it's important to have a means of prioritization.
Once you have your growth model and an idea of the key opportunities, it's time to dig one level deeper. Perform data analysis and segmentation to discover trends and identify specific problems within these areas that you want to improve. Once you have a set of well-defined problems and questions to answer, it's time to move on.
🧠 Step 2: User psychology a.k.a "The Why"
This leads us to more questions: Why is this problem occurring today? What is going through the user's head? It's important to start here rather than jumping to solutions right off the bat. If you don't have a solid sense of a user's psychology around the problem, your chances of solving it are going to be low.
There are a few ways to achieve this, namely user research through interviews, surveys, and product sessions. There's a ton of advice on conducting good user interviews, so I won't dive in deep there, but you should come away from this exercise with a set of several hypotheses. These are your guesses at why the problem specified before is happening.
🧮 Step 3: Experiment design a.k.a "The How"
Now it's time to get into the fun part: Generating ideas for solutions. These are your predictions as to what will address each hypothesis. You can go into as much detail as you want, but I would recommend at least brainstorming a few different solutions per hypothesis.
Once you have a list of solution candidates, stack rank them based on your confidence, ease of implementation, and potential impact. Create a more detailed experiment spec for whichever one sits at the top of that list, and get to work on shipping it!
I'm glossing over plenty of experiment design details, which the data scientist in me doesn't appreciate, but that's not the point of this post! The point is the process, and it's important to note that it's cyclical. Once your experiment results are in, evaluate your primary metric. If successful, how can you double down on this hypothesis? If unsuccessful, dig into why this is the case and consider a follow-up experiment with a different hypothesis or solution.
Regardless of the outcome, your analysis and learnings feedback back into your growth model and prioritization for the next experiment. And so the process starts all over again.
📚 Case Study
Before I go, I thought it would be helpful to run through a quick version of this for a mock scenario: You are the growth lead at a project management app like Asana or Basecamp. You have a step in your onboarding flow that allows users to invite teammates to their project.
- Opportunity: Most of our team invites comes from the onboarding flow
- Problem: 55% of users engage with the form but only 3% invite
- Question: Why do such a low percentage of engaged users send an invite?
- Hypotheses: 1) Users don't understand the benefit of inviting teammates at this stage 2) Users don't want to invite via email and would rather send a link in their messenger 3) Users want to invite but the friction of typing in emails is too high
- Prioritize: Users want to invite but the friction of typing in emails is too high
- Solutions: 1) Add in a copy link field 2) Allow users to import Google contacts 3) Autogenerate the company email domain
My views on the growth process aren't novel by any means. It's discussed plenty elsewhere, and this isn't an exhaustive example by any means, but I hope it's helpful nonetheless. Happy experimenting!
Thanks for reading! If you enjoyed this post, you can subscribe in the form below to get future ones like this one straight to your inbox. If Twitter is more your speed, you can follow me there as well. 🔥