Four Things You Need to Know about Your Customer Metrics

Buffer

What Customer Metrics Do You Use?

A successful customer experience management (CEM) program requires the collection, synthesis, analysis and dissemination of different types of business metrics, including operational, financial, constituency and customer metrics (see Figure 1).  The quality of customer metrics necessarily impacts your understanding of how to best manage customer relationships to improve the customer experience, increase customer loyalty and grow your business. Using the wrong customer metrics could lead to sub-optimal decisions while using the right customer metrics can lead to good decisions that give you a competitive edge.  How do you know if you are using the right customer metrics in your CEM program? This post will help formalize a set of standards you can use to evaluate your customer metrics.

Figure 1. Customer experience management is about collection, synthesis, analysis and dissemination of business metrics.

Customer Metrics

Customer metrics are numerical scores or indices that summarize customer feedback results. They can be based on either customer ratings (e.g., average satisfaction rating with product quality) or open-ended customer comments (via sentiment analysis). Additionally, customer ratings can be based on a single item or an aggregated set of items (averaging over a set of items to get a single score/metric).

Meaning of Customer Metrics

Customer metrics represent more than just numerical scores. Customer metrics have a deeper meaning, representing some underlying characteristic/mental processes about your customers: their opinions and attitudes about and intentions toward your company or brand. Figure 2 depicts this relationship between the feedback tool (questions) and the this overall score that we label as something.  Gallup claims to measure customer engagement (CE11) using 11 survey questions. Other practitioners have developed their unique metrics that assess underlying customer attitudes/intentions. The SERVQUAL method assesses several dimensions of service quality; the RAPID Loyalty approach measures three types of customer loyalty: retention, advocacy and purchasing. The Net Promoter Score® measures likelihood to recommend.

Figure 2. Advocacy Loyalty Index (customer metric) measures extent to which customers will advocate/ feel positively toward your company (underlying construct) using three items/questions.

Customer Metrics are Necessary for Effective CEM Programs but not Frequently Used

Loyalty leading companies compared to their loyalty lagging counterparts, adopt specific customer feedback practices that require the use of customer metrics: sharing customer results throughout the company, including customer feedback in company/executive dashboards, compensating employees based on customer feedback, linking customer feedback to operational metrics, and identify improvement opportunities that maximize ROI.

Despite the usefulness of customer metrics, few businesses gather them. In a study examining the use of customer experience (CX) metrics, Bruce Temkin found that only about half (52%) of businesses collect and communicate customer experience (CX) metrics. Even fewer of them review CX metrics with cross-functional teams (39%), tie compensation to CX metrics (28%) or make trade-offs between financial and CX metrics (19%).

Evaluating Your Customer Metrics

As companies continue to grow their CEM programs and adopt best practices, they will rely more and more on the use of customer metrics. Whether you are developing your own in-house customer metric or using a proprietary customer metric, you need to be able to critically evaluate them to ensure they are meeting the needs of your CEM program. Here are four questions to ask about your customer metrics.

1. What is the definition of the customer metric?

Customer metrics need to be supported by a clear description of what it is measuring. Basically, the customer metric is defined the way that words are defined in the dictionary. They are non-ambiguous and straightforward. The definition, referred to as the constitutive definition, not only tells you what the customer metric is measuring, it also tells you what the customer metric is not measuring.

The complexity of the definition will match the complexity of the customer metric itself. Depending on the customer metric, definitions can reflect a narrow concept or a more complex concept. For single-item metrics, definitions are fairly narrow. For example, a customer metric based on the satisfaction rating of a single overall product quality question would have the following definition: “Satisfaction with product quality”. For customer metrics that are made up of several items, a well-articulated definition is especially important. These customer metrics measure something more nuanced than single-item customer metrics. Try to capture the essence of the commonality shared across the different items. For example, if the ratings of five items about the call center experience (e.g., technical knowledge of rep, professionalism of rep, resolution) are combined into an overall metric, then the definition of the overall metric would be: “Overall satisfaction with call center experience.”

2. How is the customer metric calculated?

Figure 3. Two Measurement Criteria: Reliability is about precision; Validity is about meaning

Closely related to question 1, you need to convey precisely how the customer metric is calculated. Understanding how the customer metric is calculated requires understanding two things: 1) the specific items/questions in the customer metric; 2) how items/questions were combined to get to the final score. Knowing the specific items and how they are combined help define what the customer metric is measuring (operational definition). Any survey instructions and information about the rating scale (numerical and verbal anchors) need to be included.

3. What are the measurement properties of the customer metric?

Measurement properties refer to a scientifically-derived indices that describe the quality of a customer metric. Applying the field of psychometrics and scientific measurement standards (Standards for Educational and Psychological Testing), you can evaluate the quality of customer metrics. Analyzing existing customer feedback data, you are able to evaluate customer metrics along two criteria: 1) Reliability and 2) Validity. Reliability refers to measurement precision/consistency. Validity is concerned with what is being measured. Providing evidence of reliability and validity of your customer metrics is essential towards establishing a solid set of customer metrics for your CEM program. The relationship between these two measurement criteria is depicted in Figure 3. Your goal is to develop/select customer metrics that are both reliable and valid (top right quadrant).

Figure 4. Four Types of Reliability

While there are different kinds of reliability (see Figure 4), one in particular is especially important when the customer metric is made up of multiple items (e.g., most commonly, items are averaged to get one overall metric). Internal consistency reliability is a great summary index that tells you if the items should combined together. Higher internal consistency (above .80 is good; 1.0 is the maximum possible) tells you that the items measure one underlying construct; aggregating them makes sense. Low internal consistency tells you that the items are likely measuring different things and should not be aggregated together.

There are three different lines of validity evidence that help show that the customer metric actually measures what you think it is measuring. To establish that a customer metric assesses something real, you can look at the content of the items to determine how well they represent your variable of interest (establishing evidence of content validity), you can calculate how well the customer metric correlates with some external criteria (establishing evidence of criterion validity) and you can understand, through statistical relationships among different metrics, how your customer metric fits into a theoretical framework that distinguishes your customer metric from other customer metrics (e.g., How is the customer engagement metric different than the customer advocacy metric? – construct validity).

Figure 5. Evidence of criterion-related validity: Identifying which operational metrics are related to customer satisfaction with the service request (SR)

These three different lines of validity evidence demonstrate that the customer metric measures what it is intended to measure. Criterion-related validity evidence often involves linking customer metrics to other data sources (operational metrics, financial metrics, constituency metrics).

Exploring the reliability and validity of your current customer metrics has a couple of extra benefits. First, these types of analyses can improve the measurement properties of your current customer metrics by identifying unnecessary questions. Second, reliability and validity analysis can improve the overall customer survey by identifying CX questions that do not help explain customer loyalty differences. Removal of specific CX questions can significantly reduce survey length without loss of information.

4. How useful is the customer metric?

While customer metrics can be used for many types of analyses (e.g., driver, segmentation), their usefulness is demonstrated by the number and types of insights they provide. Your validation efforts to understand the quality of the customer metrics create a practical framework for making real organizational changes. Specifically, by understanding the causes and consequences of the customer metric, you can identify/create customer-centric operational metrics (See Figure 5) to help manage call center performance, understand how changes in the customer metric correspond to changes in revenue (See Figure 6) and identify customer-focused training needs and standards for employees (See Figure 7).

Figure 6. A useful customer metric (satisfaction with TAM) reveals real differences in business metrics (revenue)

Examples

Below are two articles on the development and validation of four customer metrics. One article focuses on three related customer metrics. The other article focuses on an employee metric. Even though this present blog post talked primarily about customer metrics, the same criteria can be applied to employee metrics.

In each article, I present the necessary information needed to critically evaluate each customer metric: 1) Clear definition of the customer metrics, 2) description of how metrics are calculated, 3) measurement properties (reliability/validity), 4) show that metrics are related to important outcomes (e.g., revenue, employee satisfaction). The articles are:

• Hayes, B.E.  (2011). Lessons in loyalty. Quality Progress, March, 24-31. Paper discusses the development and validation of the RAPID Loyalty approach. Three reliable customer loyalty metrics are predictive of different types of business growth. Read entire article.
• Hayes, B. E. (1994). How to measure empowerment. Quality Progress, 27(2), 41-46. Paper discusses need to define and measure empowerment. Researcher develops reliable measure of employee perceptions of empowerment, the Employee Empowerment Questionnaire (EEQ). The EEQ was related to important employee attitudes (job satisfaction). Read entire article.

Figure 7. Evidence of Criterion-Related Validity: Satisfaction with TAM Performance (customer metric) is related to TAM training.

Summary

A customer metric is good when: 1) it is supported with a clear definition of what it measures and what is does not measure; 2) there is a clear method of how the metric is calculated, including all items and how they are combined; 3) there is good reliability and validity evidence regarding how well the customer metric measures what it is supposed to measure; 4) they are useful in helping drive real internal changes (e.g., improved marketing, sales, service) that lead to measurable business growth (e.g., increased revenue, decreased churn).

Using customer metrics that meet these criteria will ensure your CEM program is effective in improving how your manage the customer relationship. Clear definitions of the metrics and accompanying descriptions of how they are calculated help improve communications regarding customer feedback. Different employees, across job levels or roles, can now speak a common language about feedback results. Establishing the reliability and validity of the metrics gives senior executives the confidence they need to use customer feedback as part of their decision-making process.

The bottom line: a good customer metric provides information that is reliable, valid and useful.