I have had the opportunity to apply my knowledge about customer loyalty/experience measurement in the area of mobile application development/testing. Today's post is about the measurement of user loyalty toward mobile applications.
Does the 5-Point/Star Rating Measure User Loyalty?
The 5-point/star rating scale has become the default mobile app loyalty metric in mobile app stores. In a global survey of mobile savvy users, Mob4Hire, a mobile market research and testing firm, found that mobile app users, on average, need to see at least a 4-star rating (out of 5 stars) before they download/purchase mobile applications. Mobile apps with higher star ratings are more likely to be downloaded compared to mobile apps with lower star ratings. While download intentions are a good indicator of mobile app quality, there are other facets of user loyalty toward mobile applications that drive the success of mobile applications.
Components of User Loyalty
Retention Loyalty: the extent to which users will remain a user and/or do not use mobile apps of competitor
- How likely are you to continue using <mobile app name> on your mobile phone? (0 - Not at all likely to 10 - Extremely likely)
- How likely are you to use a similar application by a different company? (0 - Not at all likely to 10 - Extremely likely) - Reverse coded so higher scores reflect higher loyalty
- How likely are you to recommend this mobile application to your friends/relatives? (0 - Not at all likely to 10 - Extremely likely)
- Overall, how satisfied are you with <mobile app name>? (0 - Extremely dissatisfied to 10 - Extremely satisfied
- How likely are you to use other applications that are created by the developers of <mobile app name>? (0 - Not at all likely to 10 - Extremely likely)
- How likely are you to increase the frequency with which you use <mobile app name>? (0 - Not at all likely to 10 - Extremely likely)
MobExperience Measures User Loyalty
MobExperience, Mob4Hire's mobile usability testing solution, helps companies asses and improve their customers’ mobile user experience. Mob4Hire employ their proprietary sample of mobile savvy testers (Mobsters) to give companies real feedback about their mobile app / Web site. The mobsters download, install, use and evaluate the mobile app / Web site, providing ratings on 5-point star rating (called MobStar rating here), retention, advocacy and expansion loyalty, along with seven mobile user experience areas (today's blog post focuses on the loyalty measures).
Combining data from six (6) MobExperience projects (N = 151), I correlated the MobStar rating (If <mobile app name> was available for download in an app store, how would you rate it? - 1 Star to 5 Stars) with each of the user loyalty questions. The correlations of the MobStar rating with each of the above user loyalty questions are:
- Continue using: r = .59
- Use similar app of competition: r = -.04
- Recommend: r = .67
- Overall satisfaction: r = .70
- Use additional apps: r = .55
- Increase usage: r = .46
When User Loyalty is Important, 5-Star Ratings are not Enough
There are a few key conclusions we can make about the pattern of correlations. These are:
- Star ratings of mobile apps reflect different components of user loyalty. The 5-point/star rating scale are predictive of different types of user loyalty behavior intentions. Users who give higher overall star ratings of the mobile application also report higher levels of user loyalty toward the mobile application.
- Star ratings for mobile apps primarily reflect advocacy loyalty. While MobStar rating was correlated across different types of loyalty questions, the MobStar rating was correlated more highly with the overall satisfaction and recommend questions than other loyalty measures (expansion, retention).
- Different measures of overall mobile application quality are needed to assess different types of user loyalty. The different user loyalty questions provide unique and useful information about the quality of the mobile app. The relatively low correlation of each loyalty question with the MobStar rating suggests that the loyalty questions measure something distinct (and useful) apart from the MobStar rating. The inclusion of loyalty questions when evaluating a mobile app seems reasonable. To grow adoption of your mobile app, you need to maximize different types of user loyalty (recommendations, continued use and using additional apps). The 5-star rating system tells only part of the picture.
Similar to the improvement of customer loyalty measurement in customer experience management, mobile usability researchers and practitioners are exploring better ways to define and measure user loyalty toward mobile apps / Web sites. From user recommendations and increased usage of the app to downloading additional apps, user loyalty impacts both new user growth (advocacy) and existing user growth (expansion, retention). To improve different types of user loyalty toward their mobile applications, mobile application developers need to consider different ways users can demonstrate their loyalty and use tools that help them measure and improve specific types of user loyalty.