The number of active users of an online service can grow for months while the service is actually getting worse. All it takes is for new sign-ups to arrive faster than old users leave. Overall metrics do not show this dynamic. Cohort analysis does, because it separates people by when they arrived and follows each group over time.
What a cohort is
A cohort is a group of users who share an initial event in the same period. The most common is the acquisition cohort: everyone who signed up in January, everyone in February, and so on. For each cohort you measure how many are still active after a week, a month, three months.
The initial event and the definition of "active" need choosing with care:
- for a newsletter, the initial event is subscribing and "active" might mean opening at least one message in the month;
- for a web application, the initial event is registration and "active" means performing the main action, not just logging in;
- for an online shop, the initial event is the first purchase and "active" is a subsequent purchase.
The retention table
The result is shown in a triangular table. Each row is a cohort, each column a period after joining.
| Cohort | Sign-ups | Month 1 | Month 2 | Month 3 |
|---|---|---|---|---|
| January | 500 | 42% | 31% | 27% |
| February | 620 | 40% | 29% | 25% |
| March | 800 | 35% | 24% | — |
| April | 950 | 30% | — | — |
The table is read in three directions:
- along a row: how a cohort's activity declines over time. Usually the drop is steep at first and then levels off;
- down a column: whether more recent cohorts retain better or worse than earlier ones;
- along a diagonal: the effects of an event in a specific month, which hits different cohorts at different ages.
In the example sign-ups grow every month, but month-one retention falls from 42 to 30 per cent. Total active users rise, yet each new cohort is less engaged than the last. Perhaps new campaigns bring less interested people, perhaps a product change made the first experience worse.
The curve that flattens
The most valuable piece of information is where a cohort's curve levels off. If a quarter of sign-ups are still active after three months and that value stops falling, the service has a core of people who find it worthwhile. If the curve keeps sliding towards zero, the service loses everyone over time and growth depends entirely on new acquisition.
Cohorts beyond acquisition
Cohorts can be built on other criteria:
- traffic source: do people from organic search stay longer than those from a promotion?
- first action taken: do people who complete their profile on day one come back more often?
- product version: do people who signed up after the new onboarding behave differently?
The last use is particularly helpful: comparing cohorts before and after a change is a simple way to observe its effects over time, although it does not replace a controlled test.
What you need to do it
Cohort analysis requires recognising the same person over time. It therefore works well with registered users, list subscribers or customers with an internal identifier, and is poorly suited to anonymous cookieless analytics, which by design do not link visits on different days.
In my work I prefer to compute it directly on the application database, with a query that groups users by sign-up month and counts their activity in later months. That way the data does not have to pass through a third-party tool and the definition of "active" stays under control.
SELECT date_trunc('month', u.created_at) AS cohort,
date_trunc('month', a.date) AS activity_month,
count(DISTINCT u.id) AS active
FROM users u
JOIN activity a ON a.user_id = u.id
GROUP BY 1, 2
ORDER BY 1, 2;
Mistakes to avoid
- Cohorts that are too small: with thirty people per cohort, percentages swing widely by pure chance.
- Incomplete periods: the last month is not over yet and shows artificially low values.
- An overly broad definition of active: counting a simple automatic login inflates retention.
- Confusing growth with health: more active users in total does not mean people stay longer.
In short
- A cohort groups users who share the same initial event in the same period.
- A retention table is read by rows, by columns and by diagonals.
- The point where the curve flattens shows whether a stable core of users exists.
- Cohorts require recognising the same person over time, so registered users or customers.
- Avoid cohorts that are too small, incomplete periods and overly broad definitions of activity.
