The Value of Opinion versus Data in Customer Experience Management

Buffer

Nate Silver of fivethirtyeight.com and author of The Signal and the Noise

Senior executives make business decisions based on different types of information. They can use their gut feelings to guide their decisions, they can use data to inform their decisions or they can use both.  Executives’ decisions can be guided by their prior experiences or an examination of a whole host of business data to help form their decisions. Let us take a look at a timely topic on the predicted outcome of the US presidential election that shines a light on the use of data in making predictions/decisions.

Opinion v. Data and the US Presidential Election

With the US presidential election only a few days away, political pundits put the race as a tossup. Anybody could win, their narrative goes; each candidate has a 50% chance of winning, the probability of a win for either is roughly the same as that of flipping a coin. Nate Silver, a statistician, however, thinks the outcome of the election is far from a 50/50 split. On his site, fivethirtyeight.com, Silver combines the results from many different polls with historical information about prior elections’ poll results to reach his forecasts/predictions about who will win.  Using these data, his statistical model says that Obama has an 85% chance of winning the election (as of 11/4/2012), far from a tossup. For example, out of 22 swing-state polls published on Friday, 19 has Obama in the lead, 1 has Romney in the lead and 2 result in a tie, a very unlikely outcome if the candidates were tied. With these polls, Obama leads in 86% of those polls.

Pundits have stated negative opinions about Silver’s predictions. Recently, Joe Scarborough, a pundit on MSNBC.com called Silver a “joke.”:

Nate Silver says this is a 73.6 (at that time) percent chance that the president is going to win? Nobody in that campaign thinks they have a 73 percent chance — they think they have a 50.1 percent chance of winning. And you talk to the Romney people, it’s the same thing,” Scarborough said. “Both sides understand that it is close, and it could go either way. And anybody that thinks that this race is anything but a tossup right now is such an ideologue, they should be kept away from typewriters, computers, laptops and microphones for the next 10 days, because they’re jokes.

Summarizing his response to these attacks, Nate Silver responded:

Nevertheless, these arguments are potentially more intellectually coherent than the ones that propose that the leader in the race is “too close to call.” It isn’t.  If the state polls are right, then Mr. Obama will win the Electoral College. If you can’t acknowledge that after a day when Mr. Obama leads 19 out of 20 swing-state polls, then you should abandon the pretense that your goal is to inform rather than entertain the public.

Using Customer Feedback Data

No matter your political ideology, I believe that these political polling data are useful in providing insight about the actual election results, a future event. While Silver’s exact formula for making predictions remains unknown, what he is essentially doing is averaging many different polling results to make his predictions. His prediction does not say that Obama is certain to win. What it is saying is that if a candidate has a margin of victory as big as Obama has in the polls right now, he wins 85% of the time. It is not a guarantee that Obama will win. But the odds are in his favor. That is all. What do the pundits have to offer? What is their track record? Time will tell. In the meantime, I side with Nate Silver and his use of real data.

Based on what I have seen in this political prognostication battle of opinion versus data, here are some practices that will improve the value of customer feedback data to your organization:

1. Use data to draw your conclusions, especially if there is a preponderance of the evidence about what will happen. I am big fan of using data, specifically customer data, to help companies make decisions/conclusions about how to best manage their company. Sure, judgement is important in making important decisions/conclusions, especially in times of uncertainty, but when the data overwhelmingly tell you what is happening, those data trump judgement in their importance to helping you make important decisions about your company.
2. Use hypothesis testing to make business decisions. Partisan pundits, by definition, will tend to support their side of the argument no matter what the data say.  In companies, executives will hold different beliefs about what needs to be done to move the business forward (e.g., where to invest in the company, product development strategies). These beliefs can be tested/supported/disputed using customer feedback data. This process is one of hypothesis testing, using data to rule out alternative explanations for what is occurring in your customer base while supporting other explanations for what is occurring.
3. Use different sources of data to draw your conclusions. You can be more confident in the result of many studies than any single study alone. Any single poll has a margin of error (sampling error), reflecting the fact that the results are based on a sample of the entire population; that is, the poll might not reflect what is happening in the population.  For the presidential election, states, however, have many different polls that study the same outcome. By taking the average of the different polls, you increase your confidence of the prediction/outcome. In the scientific literature, this process is referred to as a meta-analysis (aggregating many different studies of the same topic). A meta-analysis is essentially a study of studies. While there is never 100% certainty about polling data, you can still draw probabilistic conclusions about who will win the electoral vote based on the results of many different samples of likely voters.  In business, there are many different sources to tell you the health of the customer relationship, from relationship surveys and transaction-based surveys to social media sites to branded community sites. Looking at different sources of customer feedback will paint a more comprehensive picture of the real health of the customer relationship.
4. Segment your customers. While the national polling results show a close race, the winner of the popular (national) vote does not determine the winner.  The winner of the election is based on the number of Electoral College votes, which are determined state by state. Examining the differences across different customer segments helps you tailor your marketing, sales and service approaches to target specific customer groups’ needs. This practice of managing customer segments differently maximizes the return on investment of your business’ resources which helps the overall performance of the company.

Summary

Customer experience management (CEM) programs collect a lot customer data to help the business leaders guide their business decisions. Sometimes, opinion and data collide. Some people primarily rely on their personal opinions as a guide to their understanding of the world. Others primarily rely on the use of data as their guide. The prediction of the outcome of the US presidential election pits these two views against each other. Based on my examination of this political battle, I offer some ways senior executives can use customer feedback data to help them make decisions about their company.

I guess we will know Tuesday night (Wednesday morning) whether data beats opinion. I’m betting on data.

Here is an excellent interview of Nate Silver on Charlie Rose (his interview starts half way through the show) where he explains the value of using data to make predictions.