Unfortunately, exactly how we measure retention isn’t too simple at all. And if we’re going to start testing and optimizing our apps in order to change our retention rates, we should make sure we know what they mean. And if we’re going to measure retention against ‘industry averages’ (an approach which I have huge issues with, but let’s save them for another day), we should make damn sure we’re all talking about the same thing.
There are, in fact, an almost limitless number of ways in which retention rates can be measured. But let’s focus on just one for now. You might, in common with much of the industry, be comfortable with the idea of Day 1 retention being defined as “the percentage of players who return to play the game on Day 1”, with Day 1 being the day after Day 0.
The Time Zone Conundrum
That sounds fair enough. But is it? Are we really saying that a player who first plays at 11.59pm on the 18th September is retained for a day if he comes back at 12.05am on the 19th? To me, that doesn’t sound right - but if it was a consistent, comparable measure I could live with it.
Unfortunately, it isn’t. Take, for example, a company with two apps and a single ‘game time’ set to EST. One app helps users find restaurants in New York. The other does the same in San Francisco. Which one has the highest Day 1 Retention rate? The answer should be obvious - and the first person to explain why in the comments gets a Swrve T-shirt. Hint - it’s got nothing to do with the quality of the app - and isn’t that what we’re supposed to be measuring?
The point here is that if we set retention to simple calendar days we are not measuring the same thing for different users in different apps. And that makes for a flawed metric.
The alternative (and yes, this is how we do it at Swrve, although we do also offer the simpler version above) is to make Day 1 retention relative to the user rather than the calendar. Our definition is any session between 24 and 48 hours of the first, but of course you could define that time lapse in some other way if you chose.
However you did it, this approach to retention means that every user, in every app, is being measured in exactly the same way. I don’t believe anyone should expect anything less from an analytics and optimization platform. And with the data problem sorted, we can then go about getting those people to stick around and moving the needle!