Is the Importance of Customer Experience Overinflated?

Buffer

Companies rely on customer experience management (CEM) programs to provide insight about how to manage customer relationships effectively to grow their business. CEM programs require measurement of primarily two types of variables, satisfaction with customer experience and customer loyalty. These metrics are used specifically to assess the importance of customer experience in improving customer loyalty. Determining the "importance" of different customer experience attributes needs to be precise as it plays a major role in helping companies: 1) prioritize improvement efforts, 2) estimate return on investment (ROI) of improvement efforts and 3) allocate company resources.

How We Determine Importance of Customer Experience Attributes

When we label a customer experience attribute as "important," we typically are referring to the magnitude of the correlation between customer ratings on that attribute (e.g., product quality, account management, customer service) and a measure of customer loyalty (e.g., recommend, renew service contract). Correlations can vary from 0.0 to 1.0. Those attributes that have a high correlation with customer loyalty (approaching 1.0) are considered more "important" than other attributes that have a low correlation with customer loyalty (approaching 0.0).

Measuring Satisfaction with the Customer Experience and Customer Loyalty Via Surveys

Companies typically (almost always?) rely on customer surveys to measure both the satisfaction with the customer experience (CX) as well as the level of customer loyalty.  That is, customers are given a survey that includes questions about the customer experience and customer loyalty. The customers are asked to make ratings about their satisfaction with the customer experience and their level of customer loyalty (typically likelihood ratings).

As mentioned earlier, to identify the importance of customer experience attributes on customer loyalty, ratings of CX metrics and customer loyalty are correlated with each other.

The Problem of a Single Method of Measurement: Common Method Variance

The magnitude of the correlations between measures of satisfaction (with the customer experience) and measures of customer loyalty are made up of different components. On one hand, the correlation is due to the "true" relationship between satisfaction with the experience and customer loyalty.

On the other hand, because the two variables are measured using the same method - a survey with self-reported ratings, the magnitude of the correlation is partly due to the method of how the data are collected. Referred to as Common Method Variance (CMV) and studied in the field of social sciences (see Campbell and Fiske, 1959) where surveys are a common method of data collection, the general finding is that the correlation between two different measures is driven partly by the true relationship between the constructs being measured as well as the way they are measured.

The impact of CMV in customer experience management likely occurs when you use the same method of collecting data (e.g., survey questions) for both predictors (e.g., satisfaction with the customer experience) and outcomes (e.g., customer loyalty). That is, the size of the correlation between satisfaction and loyalty metrics is likely due to the fact that both variables are measured using a survey instrument.

Customer Loyalty Measures: Real Behaviors v. Expected Behaviors

The CMV problem is not really about how we measure satisfaction with the customer experience; a survey is a good way to measure the feelings/perceptions behind the customers' experience. The problem lies with how we measure customer loyalty. Customer loyalty is about actual customer behavior. It is real customer behavior (e.g., number of recommendations, number of products purchased, whether a customer renewed their service contract) that drives company profits. Popular self-report measures ask for customers' estimation of their likelihood of engaging in certain behaviors in the future (e.g., likely to recommend, likely to purchase, likely to renew).

Using self-report measures of satisfaction and loyalty, researchers have found high correlations between these two variables; For example, Bruce Temkin has found correlations between satisfaction with the customer experience and NPS to be around .70. Similarly, in my research, I have found comparably sized correlations (r ≈ .50) looking at the impact of the customer experience on advocacy loyalty (the recommend question is part of my advocacy metric). Are these correlations a good reflection of the importance of the customer experience in predicting loyalty (as measured by the recommend question)? Before I answer that question, let us first look at work (Sharma, Yetton and Crawford, 2009) that helps us classify different types of customer measurement and their impact on correlations.

Different Ways to Measure Customer Loyalty

Sharma et al. highlight four different types of measurement methods. I have slightly modified their four types to illustrate customer loyalty measures that are least susceptible to CMV (coded as 1) to measures that are most susceptible to CMV (coded as 4):

1. System-captured metrics reflect objective metrics of customer loyalty: Data are obtained from historical records and other objective sources, including purchase records (captured in a CRM system). Example: Computer generated records of “time spent on the Web site” or “number of products/services purchased” or "whether a customer renewed their service contract."
2. Behavioral-continuous items reflect specific loyalty behaviors that respondents have carried out: Responses are typically captured on a continuous scale. Example item: How many friends did you tell about company XYZ in the past 12 months? None to 10, say.
3. Behaviorally-anchored items reflect specific actions that respondents have carried out: Responses are typically captured on scales with behavioral anchors. Example item: How often have you shopped at store XYZ in the past month? Not at all to Very Often.
4. Perceptually-anchored items reflect perceptions of loyalty behavior: Responses are typically on Likert scales, semantic differential or "agree/disagree scale". Example: I shop at the store regularly. Agree to Disagree.

These researchers looked at 75 different studies examining the correlation between perceived usefulness (predictor) and usage of IT (criterion). While all studies used perceptually-anchored measures for perceived usefulness (perception/attitude), different studies used one of four different types of measures of usage (behavior). These researchers found that CMV accounted for 59% of the variance in the relationship between perceived usefulness and usage (r = .59 for perceptually-anchored items; r = .42 for behaviorally anchored items; r = .29 for behavioral continuous items; r = .16 for system-captured metrics). That is, the method with which researchers measure "usage" impacts the outcome of the results; as the usage measures become less susceptible to CMV (moving up the scale from 4 to 1 above), the magnitude of the correlation decreases between perceived usefulness and usage.

Looking at research in the CEM space, we commonly see that customer loyalty is measured using questions that reflect perceptually-anchored questions (type 4 above), the type of measure most susceptible to CMV.

Table 1. Descriptive statistics and correlations of two types of recommend loyalty metrics (behavioral-continuous and perceptually-anchored) with customer experience ratings.

An Example

I have some survey data on the wireless service industry that examined the impact of customer satisfaction with customer touch points (e.g, product, coverage/reliability and customer service) on customer loyalty. This study included measures of satisfaction with the customer experience (perceptually-anchored) and two different measures of customer loyalty:

1. self-reported number of people you recommended the company to in the past 12 months (behavioral-continuous).
2. self-reported likelihood to recommend (perceptually-anchored)

The correlations among these measures are located in Table 1.

As you can see, the two recommend loyalty metrics are weakly related to each other (r = .47), suggesting that they measure two different constructs. Additionally, and as expected by the CMV model, the behavioral-continuous measure of customer loyalty (number of friends/colleagues) shows a significantly lower correlation (average r = .28) with customer experience ratings compared to the perceptually-anchored measure of customer loyalty (likelihood to recommend) (average r = .52). These findings are strikingly similar to the above findings of Sharma et al. (2009).

Summary and Implications

The way in which we measure the customer experience and customer loyalty impacts the correlations we see between them. As measures of both variables use perceptually-anchored questions on the same survey, the correlation between the two are likely overinflated. I contend that the true impact of customer experience on customer loyalty can only be determined when real customer loyalty behaviors are used in the statistical modeling process.

We may be overestimating the importance (e.g., impact) of customer experience on customer loyalty simply due to the fact that we measure both variables (experience and loyalty) using the same instrument, a survey with similar scale characteristics. Companies commonly use the correlations (or squared correlation) between a given attribute and customer loyalty as the basis for estimating the return on investment (ROI) when improving the customer experience. The use of overinflated correlations will likely result in an overestimate of the ROI of customer experience improvement efforts. As such, companies need to temper this estimation when perceptually-anchored customer loyalty metrics are used.

I argue elsewhere that we need to use more objective metrics of customer loyalty whenever they are available. Using Big Data principles, companies can link real loyalty behaviors with customer satisfaction ratings. Using a customer-centric approach to linkage analysis, our company,TCELab  helps companies integrate customer feedback data with their CRM data where real customer loyalty data are housed  (see CEM Linkage for a deeper discussion).

While measuring customer loyalty using real, objective metrics (system-captured) would be ideal, many companies do not have the resources to collect and link customer loyalty behaviors to customer ratings of their experience. Perhaps loyalty measures that are less susceptible to CMV could be developed and used to get a more realistic assessment of the importance of the customer experience on customer loyalty.  For example, self-reported metrics that are more easily verifiable by the company (e.g., "likelihood to renew service contract" is more easily verifiable by the company than "likelihood to recommend") might encourage customers to provide realistic ratings about their expected behaviors, thus reflecting a truer measure of customer loyalty. At TCELab, our customer survey, the Customer Relationship Diagnostic (CRD), includes verifiable types of loyalty questions (e.g., likely to renew contract, likely to purchase additional/different products, likely to upgrade).

The impact of the Common Method Variance (CMV) in CEM research is likely strong in studies in which the data for customer satisfaction (the predictor) and customer loyalty (the criterion) are collected using surveys with similar item characteristics (perceptually-anchored). CEM professionals need to keep the problem of CMV in mind when interpreting customer survey results (any survey results, really) and estimating the impact of customer experience on customer loyalty and financial performance.

What kind of loyalty metrics do you use in your organization? How do you measure them?