Don’t take Pokemon Go for dead: a model of product growth
October 2, 2016 1 Comment
In the last month, some people wrote about the decay in active users for Pokemon Go after its first month, in a tone that presents the game as likely a mere fad – with article on 538, cinemablend and Bloomberg, for example. “Have you deleted Pokémon Go yet?” was even trending on Twitter. Although it is of course certainly possible that this ends up being an accurate description for the game, I posit that such conclusions are rushed. To do so, I examine some systemic reasons that would make the Pokemon Go numbers for August be inevitably lower than those for July, without necessarily implying that the game is doomed to dwindle into irrelevance.
Others have made similar points before – see this article and the end of this one for example. However, in the spirit of TheEGG, and unlike what most of the press articles can afford to do, we’ll bring some mathematical modeling into our arguments.
The three systemic reasons that can make the Pokemon Go numbers for August lower than those for July, without necessarily implying that the game is doomed to dwindle into irrelevance:
- The first reason, and perhaps most obvious one, is that even if they have gone down from the crazy times after launch, the numbers for Pokemon Go are still incredibly successful when compared to other games. Looking at them on September 5, the day this paragraph is first being written, we have a daily Pokemon Go revenue just for iPhone and just for the US of 2 million dollars, with around 150K signups of such users as well. Both of these are easily enough to top the rankings, specially in the signups category, where Pokemon Go triples its persecutor. This becomes even more astonishing once one considers the small size of Niantic (between 11 and 50 employees according to LinkedIn).
- Being a bit more subtle in our considerations, it is also important to realize that Pokemon Go received a large number of players in rural and suburban areas due to the press coverage.These people would never have heard of the game was it something that got popular by word of mouth, or targeted advertising. At the moment, the game, like many other location-based apps, doesn’t cater to them. It is then certainly understandable that they are choosing to leave the game. To appreciate the implications of this for user growth, imagine an extreme case where the monthly drop-off rate of suburban and rural users was 85%, and that they represented 40% of the user base in the first month. Take it that the drop-off rate for other users (who live in cities) was 5%. Then, the monthly drop-off rate after the month where the rural and suburban users joined would be around 35-40%. However, as all of the rural and suburban users leave the app, the drop-off rate will turn into a much more decent 5%. This means that extrapolating the decay from the month with all the rural and suburban users in the app to later months would result in extremely inaccurate and overly pessimistic predictions.
- Somewhat related to the previous point, the fact that Pokemon Go reached so many of its potential users at the same time due to press coverage is something that would make its immediately following numbers decay. In slightly more formal terms, this is because a fraction of the new users is bound not to like the app, and since there are so many of them, this interferes with normal user growth patterns, and in fact obscures them. Why is this is not immediately obvious, but can be seen in the model accompanying this entry.
Model for product growth
This model that we define and simulate is of course not completely accurate, but it suffices to demonstrate our points:
- A product has a potential market of M users. A fraction of S of them are in suburban and rural areas, while the rest are in cities.
- Due to spontaneous product discovery, a percentage D of the potential market that still has not tried the app will give it a try. This affects equally both suburban/rural and urban areas.
- For each of the suburban/rural and urban categories, we have the following behavior, with different parameters according to the suburban/rural vs urban split:
- To reflect virality, each user will talk to V random people within the potential market for their density category. If those who they talk to haven’t given a try to the product yet, they will do so with probability J.
- A fraction K of users that use the product on a given day will use it again on the next day.
- A fraction R of users that have used the product in the past but not the previous day will use it again.
For each of the suburban/rural and urban categories, this gives rise to the following recursive equations
Assume now that we have M = 100 million people, a discovery rate D = 0.05%. S corresponds again to 40%. For cities, we have parameters for virality V = 0.25 and J= 50%, a reactivation rate R = 0.2%, and a daily retention rate K = 90%. For rural areas and suburbs, consider parameters for virality V = 0.1 and J = 40%, a reactivation rate R = 0.05%, and a daily retention rate K = 20%. These parameters are within what would be considered realistic for a highly successful mobile application, as far as I know.
Using a spreadsheet to implement the model, we can see that the user rate curve for such a product during its first year would be the following:
We can see that the user numbers rise until they slightly decay and reach a steady state of around 5.5 million users.
However, assume now that on day two after the product launches we have an event such as press coverage that suddenly attracts 40% of the potential market. Furthermore, take it that event affects uniformly users in both the urban and suburban/rural categories. The user growth curve would then be (note that the y-scale has changed):
We can see that while the user numbers eventually reach the same steady state, someone looking at the initial user numbers will just see a drastic decay.
Of course, this is not to say that there were no wrong decisions taken by Niantic during the first months after the Pokemon Go release. However, I suspect that those bad decisions can still be fixed, and to make a judgement on the future of the game, we should wait until more significant releases do happen. The thrill of catching legendaries could very well make a lot of the users that signed up to use the app for a while again. And of course, so could significant attempts to increase the population density ranges for which the app presents an attractive experience.
- I believe this is a reasonable estimation of the initial percentage of suburban and rural users for Pokemon Go in the US. Rural population in the US is around 15% of the total. Suburban population is trickier to estimate, but the media put it around 50% a couple of decades ago, so let’s say that rural and suburban together can be around 60% now. Even if the users who discover the app due to word of mouth are mostly in cities, media appearances will tend to move the initial fraction of rural and suburban users towards that 60% of the wider population. Our partial compromise of 40% can be sanity-checked by the fact that 538 take the fraction of rural users for Pokemon Go (for July, I assume) to be around 11%, using SurveyMonkey data.
- Notice an analogy here to Artem’s previous discussion of the evolution of resistance to chemotherapy. The treatment is ‘absence of marketing campaign’, and the suburban/rural population is sensitive, while the urban one is resistant. The question for marketers becomes: what can we learn about optimal media campaign timing from dosing schedules? The question for mathematical medicine becomes: is there a rich data source of resistance dynamics in media campaigns?