How do you measure product usage habits?
Motivation: for the “service” aspect of SaaS products to be significant users need to find value in an app and develop habits of usage. In the Product-Led Growth world we talk about habit loops, and the triggers that bring users back to our app over and over again. I wanted to understand and measure habits, and hit upon an analytical model that uncovers “habit signals” in product usage data. Being able to measure the habits from the product perspective (“which parts of the product deliver the most value to users?”) as well as the user perspective (“which users have developed the strongest habits?”) is immensely valuable in growing a SaaS product.
Successful software products are acquired by people for regular use. My phone is full of apps that I have picked up because they do something for me (provide directions, tell me the temperature outside, play music), and because it is actually full (64GB is not enough anymore) I have to remove the least used, least useful apps in order to add new ones. From the perspective of the teams building those apps they see me along a journey:
Acquisition–I make the choice to delete an unused app and install their new one. I have been “acquired” as a new user.
Activation–I begin to explore the product and look for value, and I will keep it if I come to some feeling about how it is useful to me. What job is it going to do for me? When I figure that out I’m not an “activated” user of the product.
Retention–I come back again and again, and form habit loops around the product. If I use it enough a lot of the taps and clicks I do will become habitual. I couldn’t tell you where the feature I use is (on a menu somewhere), but I can find and use it while having a conversation and making lunch for my kids.
Conversion to a paid account is not important in trying to assess the health of a user base, from a usage perspective. In the user’s experience having to go from a free trial or freemium to paid is friction, and if they have not formed any habits purchasing is speculative. If they do not quickly realize the value your product provides them they will churn. See my predictive Product-Qualified Lead scoring model for a conversion oriented product analysis.
Here I want to focus on retention, and go beyond Monthly, Weekly, Daily Active Users (MAU, WAU, DAU) and the common retention curve graphs. These are common SaaS performance metrics, but in order to meaningfully drive growth we need to dig deep and figure out what users are coming back to do over and over (the “use” aspect of “user”!) We are not random in how we use products, and after an exploratory phase (when we figure out what a product does) we use them to do things (where “doing” is understood to have a functional as well as emotional and social components). This post will start with assumption that we have users of a product already and we want to understand their habits, and what they actually do when they pick up our app every day or week (then as a next step we could use that knowledge to encourage new users in similar directions, or focus the efforts of our product and development team). How habits are formed is a fascinating topic as well (see this post for a good discussion of habit formation around software products).
Defining Habits
From Oxford Languages:
Habit: a settled or regular tendency or practice, especially one that is hard to give up.
From Merriam-Webster:
Habit: a usual way of behaving: something that a person does often in a regular and repeated way.
From Wikipedia:
Habit: is a routine of behavior that is repeated regularly and tends to occur subconsciously.
A habit, if viewed externally, appears as a reoccurring pattern, often with some notion of “automatic” and without explicit intention. Pattern itself is a reoccurrence of consistent elements over a space or time (Wikipedia: “A pattern is a regularity in the world…the elements of a pattern repeat in a predictable manner”). If we think about habits in our daily lives they can be described, typically, as a sequence of events that define this pattern:
Example: I drink 2 cups of hot tea every morning. I go to the kitchen, I pick up the kettle, I walk to the sink, I fill up the kettle (often, even if it already has enough water), I take it to the stove, put it on the front-left burner, turn the element on to max, walk to the shelf and take my favorite ceramic mug, put the mug on the counter, open the jar of tea bags, take one bag, put it in the mug, wait for the water to boil, and on and on.
I do this same sequence of actions almost subconsciously every morning. Someone viewing me everyday would be able to ascertain the “pattern” after they saw it repeated several times over several days. We could then say: Ben is a retained user (he is DAU of tea kettles), and has a strong habit-loop around tea making. [Interestingly, there is a lot of debate around whether we can self-report habits, since they are often stimuli based responses performed unintentionally. So this external observer is necessary, and in this case my family confirms my tea drinking habit. 🫖]
Sequential dependency–the precise order of actions in a habit is often not as significant as the co-occurrence of the actions. I can put the tea bag in my mug before I turn the kettle on, or after I’ve already poured the boiling water in the mug. The habitual pattern involves both actions but they can happen in a variety of orders. The common approach to catching this in Machine Learning is to count the number of occurrences of an action (also called a “feature”) within a “sample” of the pattern. We can then compare distances between vectors of features to determine similarity and look for clusters of patterning.
Habits in software usage follow the same patterning paradigm, and with the right processing can be discovered and used for a variety of business decisions and tasks (lead qualification, opportunity identification, onboarding performance, account health and churn risk, etc.) For example: once we have analyzed for and identified a few primary habit patterns product users have formed, we can compare new users and watch as their usage settles into the same patterns. If their usage remains chaotic (incoherent, in a sense) then they aren’t likely to be interested in purchasing. If they match the patterns of long-term users we can start making assumptions about their likelihood to cancel or churn (from a usage perspective). Additionally, we can test hypotheses on where the value in the product really is, and what keeps people coming back to use it again and again.
Example: I am a DAU of Spotify for listening to music. I open the app on my phone, I tap the Home button, and look at my 6 most recently played playlists. If one of those seems interesting I tap it, select Play or Shuffle-play, wait until I hear music, and turn my phone off. If I am looking for something new I start scrolling down, viewing all the recommendations, or use the search function until I arrive at a suitable selection. There are many other features and actions I can take in the app, but I only use a handful and a typical “session” for me involves 2-6 taps or clicks before I have obtained the value I am seeking: music to accompany working or cooking or driving.
In part 2 (TBD!) I will describe how to pre-process user data and perform a pattern analysis, identifying “habit signals” that can give us deeper insights into how people use a product and which feature combinations are most critical in delivering value. I will get into figuring out how users are using the product, which features are really valuable, and what habits we can see them building around the product that make it sticky.
Dr. Ben Smith is a Data Scientist and thinker, fascinated by the appearance of computers in our daily lives, creativity, and human struggles. He has had the privilege to think, learn, and write at the University of Illinois, the National Center for Supercomputing Applications, the Cleveland Institute of Art, Case Western Reserve U., IUPUI, and at Boardable: Board Management Software, Inc.
If you have feedback or questions please use Contact me to get in touch. I welcome thoughtful responses and constructive critique.