My colleague Nicolas wrote a great guide with guidelines on how to do cohort analyses which I’d like to share with the readers of this blog. Thanks, Nicolas, for allowing me to hear guest release it. Without further ado, it is here! At Point Nine we believe that the only path to get a real sense of user retention and customer lifetime is doing a proper cohort analysis.
Much has been said and written about them and Christoph has a published a great template and guide on this issue if the concept is not used to you. With this Q&A I want to focus on a few of the more practical questions that may arise when you are actually applying a cohort evaluation for your startup. After near to two years of dealing with SaaS companies and doing much of the analysis I have learned that in most cases there is absolutely no perfect step-by-step method. Now let’s enter it!
Q: Which users must I include in the bottom quantity of the cohort? You can find two parts to the answer as it depends upon what you would like to measure. If you want to find out your overall user retention and have a free plan, month then you should include all signals of a specific.
However if you want to compute your customer lifetime value, you should only go through the true amount of paid conversions. I only count an account as a paid one when the user has or will be charged for a period. If you provide a 30-day trial offer for example, wait to find out if an individual changes into a paying plan before you include him in the cohort.
This way the quantities won’t be biased with users that truly never covered your service. When possible without too much effort, it’s also advisable to try to eliminate all ‘friend plans’ which you have directed at friends, your team or investors. If they’re not paying, they are not representative for the true cohorts. Q: How do I treat churn within the first / base month?
There will vary approaches here, month into consideration is the most accurate representation of fact but in my view taking churn within the first. Which means that in your first month the retention could be significantly less than 100%, if people cancel their paid subscription within that month. I really do this because I don’t want the analysis to exaggerate churn in the next month and understate it in the first / base month. After all the known reasons for churning in the first 1-4 weeks could be very different than after 5-8 weeks. Q: Should I treat team and specific accounts in different ways?
But when team programs make up a substantial part of your paid accounts, or your product has a very different user experience when an entire team uses it, you should probably take a look at both kind of accounts separately. Findings could include that united team accounts are much more active, month than specific plans churn less and see a lower drop-off in the first.
Q: What about annual vs. Again, if you are concentrating on how active your users are over their lifetime it is OK to mix both plans. If you just want to see how lots of the people that registered still keep coming back after X months, no need to split hairs. If you’re however focused on churn, you should only take a look at paid accounts that could have churned for the reason that month.
- Twin Cities Hospital (FL)
- Interact with client and other stake holders
- Add subtlety with your ties
- Evaluate student geographic data
This is one of the 9 Worst Practices in SaaS Metrics and means that you should exclude all annual programs that aren’t expiring in the respective month. Including these in the denominator would in any other case skew churn amounts. Q: Given that I’ve it, what can I take from it away? The two most obvious take-aways are depicted in this (KISSmetrics) retention grid. Moving horizontally you can see how the retention of a cohort decreases within the user’s lifetime.
Interesting here’s where the highest drop-offs occur and if the figures stabilise after a couple of months. Vertically, you can (ideally) see how the retention of your cohorts change over the product lifetime. Assuming you are not twiddling your thumbs while catching up with House of Cards or sipping Mai Tai’s at the beach once your product launches, you should see a noticable difference in user retention with young cohorts as the product improves.