Beauties and Delusions of Daily Metrics

Photo by Jason Coudriet on Unsplash

DAU (daily active users), WAU and MAU are types of metrics that show how many unique users interact with your product on a particular day, week or month. It is a pulse of growth and good proxies for engagement and stickiness.

To generalise, I will call DAU metrics all metrics that have DAU or alternatives as a denominator.

For example, ARPDAU = Daily Revenue / Daily Active user. It is a DAU metric because it shows the average revenue per daily active user, which is how much money a user generates on average. The same works for the average number of sessions or time spent in an app or a website.

DAU metrics help you spot disruptions and improvement. It is their beauty. If today you release a new bold feature that affects the existing user base, you will see it tomorrow in DAU metrics. And if things fall apart after the release, you can react quickly.

However, for each great beauty, there is a delusion, or two:

  1. Selection bias. You lose non-engaged and churned users from the sight of DAU metrics.
  2. Aggregation bias. DAU has a mixed nature. It is never the same and full of different types of users. Many factors affect the final number. Two of those are the product itself and the marketing activity, especially paid. Old users are churning while marketing is renewing DAU with fresh users changing the mix. Every. Single. Day.

That makes DAU metrics “impure” and ineffective for actionable Product Analytics. They are measures of growth but weak indicators of product health.

Imagine you make changes to your product every week or month. A portion of these changes are harmful and nudge down engagement and revenue per user.

When marketing functions well, you have a growing flow of new users, DAU will continue to go up. ARPDAU can stay the same or even increase. But the creeping problems will compound inside the existing user base ruining your growth potential.

To avoid this issue, you should control all changes you make via experiments, use Cohort Analysis and scrutinise your DAU with segmentations. It is what Product Analytics is all about. There is no all-embracing method, only trade-offs. You always have a problem to solve, a lever to find, and a small win to praise.

I’m grateful you make it to the end. Thanks for reading. If you want to give feedback or say “hi”, feel free to DM me on Twitter or send an email.

Beams of appreciation,
John

Originally published at https://johnhellion.com.

Data Scientist