Top of Page

What does the 5-point/star mobile app rating tell us about user loyalty?

What do app ratings mean?I have had the opportunity to apply my knowledge about customer loyalty/experience measurement in the area of mobile application development/testing. Today's post is about the measurement of user loyalty toward mobile applications.

Does the 5-Point/Star Rating Measure User Loyalty?

The 5-point/star rating scale has become the default mobile app loyalty metric in mobile app stores. In a global survey of mobile savvy users, Mob4Hire, a mobile market research and testing firm, found that  mobile app users, on average, need to see at least a 4-star rating (out of 5 stars) before they download/purchase mobile applications. Mobile apps with higher star ratings are more likely to be downloaded compared to mobile apps with lower star ratings. While download intentions are a good indicator of mobile app quality, there are other facets of user loyalty toward mobile applications that drive the success of mobile applications.

Components of User Loyalty

Mob4Hire, to expand the concept of user loyalty beyond the 5-star rating system, incorporated the RAPID loyalty approach into their mobile app usability testing solution, MobExperience. The RAPID approach holds that there are three types of customer loyalty which drive different types of business growth. Using this approach, these loyalty components (and questions) were modified for mobile applications:

Retention Loyalty: the extent to which users will remain a user and/or do not use mobile apps of competitor

  • How likely are you to continue using <mobile app name> on your mobile phone? (0 - Not at all likely to 10 - Extremely likely)
  • How likely are you to use a similar application by a different company?  (0 - Not at all likely to 10 - Extremely likely) - Reverse coded so higher scores reflect higher loyalty
Advocacy Loyalty: the extent to which users will become advocates of the mobile app
  • How likely are you to recommend this mobile application to your friends/relatives? (0 - Not at all likely to 10 - Extremely likely)
  • Overall, how satisfied are you with <mobile app name>? (0 - Extremely dissatisfied to 10 - Extremely satisfied
Expansion Loyalty: the extent to which customers expand their use of the mobile app / developer
  • How likely are you to use other applications that are created by the developers of <mobile app name>?  (0 - Not at all likely to 10 - Extremely likely)
  • How likely are you to increase the frequency with which you use <mobile app name>? (0 - Not at all likely to 10 - Extremely likely)

MobExperience Measures User Loyalty

Overall Mobile Application Ratings (Star ratings) are Related to Different Types of User Loyalty

MobExperience, Mob4Hire's mobile usability testing solution, helps companies asses and improve their customers’ mobile user experience. Mob4Hire employ their proprietary sample of mobile savvy testers (Mobsters) to give companies real feedback about their mobile app / Web site. The mobsters download, install, use and evaluate the mobile app / Web site, providing ratings on 5-point star rating (called MobStar rating here), retention, advocacy and expansion loyalty, along with seven mobile user experience areas (today's blog post focuses on the loyalty measures).

Combining data from six (6) MobExperience projects (N = 151), I correlated the MobStar rating (If <mobile app name> was available for download in an app store, how would you rate it? - 1 Star to 5 Stars) with each of the user loyalty questions. The correlations of the MobStar rating with each of the above user loyalty questions are:

  • Continue using: r = .59
  • Use similar app of competition: r = -.04
  • Recommend: r = .67
  • Overall satisfaction: r = .70
  • Use additional apps: r = .55
  • Increase usage: r = .46

When User Loyalty is Important, 5-Star Ratings are not Enough

There are a few key conclusions we can make about the pattern of correlations. These are:

  1. Star ratings of mobile apps reflect different components of user loyalty. The 5-point/star rating scale are predictive of different types of user loyalty behavior intentions. Users who give higher overall star ratings of the mobile application also report higher levels of user loyalty toward the mobile application.
  2. Star ratings for mobile apps primarily reflect advocacy loyalty. While MobStar rating was correlated across different types of loyalty questions, the MobStar rating was correlated more highly with the overall satisfaction and recommend questions than other loyalty measures (expansion, retention).
  3. Different measures of overall mobile application quality are needed to assess different types of user loyalty. The different user loyalty questions provide unique and useful information about the quality of the mobile app. The relatively low correlation of each loyalty question with the MobStar rating suggests that the loyalty questions measure something distinct (and useful) apart from the MobStar rating. The inclusion of loyalty questions when evaluating a mobile app seems reasonable. To grow adoption of your mobile app, you need to maximize different types of user loyalty (recommendations, continued use and using additional apps). The 5-star rating system tells only part of the picture.

Similar to the improvement of customer loyalty measurement in customer experience management, mobile usability researchers and practitioners are exploring better ways to define and measure user loyalty toward mobile apps / Web sites. From user recommendations and increased usage of the app to downloading additional apps, user loyalty impacts both new user growth (advocacy) and existing user growth (expansion, retention). To improve different types of user loyalty toward their mobile applications, mobile application developers need to consider different ways users can demonstrate their loyalty and use tools that help them measure and improve specific types of user loyalty.

, , ,

4 Responses to What does the 5-point/star mobile app rating tell us about user loyalty?

  1. Keith Ching December 4, 2011 at 5:42 pm #

    Hi Bob,

    Continued from previous post...

    What aroused my curiosity are the advocacy loyalty questions. From the graph bar, it seems that the "Recommend app" is more linearly correlated to the 5-star scale as compared to the "overall satisfaction with app". However, the r value for the former (r=0.67) in comparison with the latter (r=0.70) appears to suggest otherwise.

    Thank you.

    Regards,
    Keith

  2. Keith Ching December 4, 2011 at 5:33 pm #

    Hi Bob,

    Thank you for the very comprehensive explanation.

    Is it correct to assume that the sample data used to derive the r values above are the exact same ones used to plot the bar graph (shown on this page)?

    In any case, would you be kind enough to share the data (for the graph & for the derivation of r) with me via email?

    Thanks again.

    Best regards,
    Keith

  3. Bob Hayes December 2, 2011 at 10:57 am #

    Keith,

    Thanks for your interest. The value r (or Pearson correlation coefficient) measures the linear relationship between two variables. Since there were 6 loyalty questions, I was able to calculate 6 correlation coefficients (r) using the MobStar rating as the other variable in those 6 correlations. Here is a good link that shows the calculations: http://en.wikipedia.org/wiki/Correlation_and_dependence

    The correlation can range from -1 (perfect negative relationship - as one variable goes up, the other goes up) to 1 (perfect positive relationship (as one variable goes up, the other variable also goes up). A correlation of 0 indicates there is no linear relationship between the two variables. The closer the correlation is to 1 (or -1) the more strongly the two variables are related.

    There are other measures of association you could use (Beta, r**2) to quantify the strength of the relationship they all will result in pretty much the same conclusion.

    Bob

  4. Keith Ching December 2, 2011 at 1:59 am #

    Hi Bob,

    I read this article of yours with much interest. Would you mind clarifying how did you come about deriving the values of variable r for the different loyalty questions?

    Thanks in advance.

    Best regards,
    Keith Ching

bob@businessoverbroadway.com | 206.372.5990

UA-23043697-1