Do Your Metrics Measure Up?

Assessing the right customer satisfaction data is important if you want to improve your organization.

Published: Monday, May 3, 2010 - 10:05

ISO 9001 makes multiple references to the need to understand customer requirements and to monitor the extent to which those requirements have been fulfilled. In fact, the scope of the standard clearly indicates that its focus is primarily on ensuring that an organization has the processes that support its ability to fulfill customer requirements. Therefore, it stands to reason that gathering and analyzing information about how well an organization is doing in terms of meeting customer requirements would be an essential feature of its quality management system. Subclause 8.4 requires organizations to analyze data relating to (among other topics) customer satisfaction.

The questions now are:

What data do you gather to assess customer satisfaction?

What metrics will provide a credible and reliable barometer of what your customers think of your organization and the products you furnish?

It is possible to gather the wrong information. This can be the result of a failure to properly determine what indicators will provide the best information. Or it can also originate from a misdirected need to develop metrics that paint a specific picture—that portray the organization in a glowing, albeit inaccurate, light.

I had occasion about a month ago to accompany my mother on one of her many visits to have her hearing aid checked and adjusted. The instrument was not functioning properly—or as advertised. My mother was getting increasingly frustrated with the company’s failure to fix the problem. She was also tired of waging a battle with an individual who was making her feel as though the problem lay with her rather than the hearing device.

As we sat in the reception area, patiently waiting our turn, I glanced over at a fairly large poster displayed near the door. It apparently provided statistical testimony to their customers’ satisfaction. In very large letters at the top, it said, “Overall customer satisfaction 98%.” Below that in smaller type were the following customer satisfaction metrics:

95% Live demonstration of hearing instrument

99% Staff friendliness and professionalism

98% Ease of making appointment

97% Knowledge of hearing instrument

96% Convenient office location

I turned to my mother and asked her if she could read the large print at the top of the poster and she said, “Yes.” Then I asked if she could read the other text below, to which she answered, “No.”

Consider the fact that the vast majority of hearing aids are sold to older individuals. Many of them can probably not read the smaller text. Therefore, the only statement of customer satisfaction that they will be able to read easily will be the one posted in large print. So they, like my mother, will be left wondering what’s wrong with them if more than 95 percent of the other hearing-aid users are satisfied. This is a subtle deterrent to challenging the technician who repeatedly tells them that the fault doesn’t lie in the instrument but in how they’re using it.

Consider now the other indicators of customer satisfaction. Not a single one relates to the fit, form, or function of the product. Nor is there a metric relating to the test administered before the correct hearing aid is selected. The dubious 98-percent customer satisfaction isn’t inaccurate. However, it’s based on criteria that are irrelevant to the most important features of the product—its performance. The assertions are not completely unimportant. We do want to interact with professional individuals who have knowledge of the product. But, again, this has little relevance to the biggest indicator that will truly attest to customer satisfaction.

These kinds of metrics are construed to generate a specific result. The tactic is to determine the outcome in advance and then to find the data that will support the conclusion. This is generally done to influence a decision maker or to make someone look good.

Companies sometimes manipulate data in similar ways when measuring on-time delivery, customer complaints, or internal performance indicators. Other organizations may simply have grabbed onto convenient metrics so they’ve got something to show during an audit. Either way, the results are at best ineffective and at worse counterproductive and deceptive.

Selecting appropriate metrics is the only guarantee that the information you are using to make decisions is reliable. Take the time to select the metrics that will delineate the information you need to improve your organization and your ability to serve your customers. Anything else is just playing with numbers.

About The Author

Denise Robitaille

Denise Robitaille is the author of thirteen books, including: ISO 9001:2015 Handbook for Small and Medium-Sized Businesses.

She is chair of PC302, the project committee responsible for the revision to ISO 19011, an active member of USTAG to ISO/TC 176 and technical expert on the working group that developed the current version of ISO 9004:2018. She has participated internationally in standards development for over 15 years. She is a globally recognized speaker and trainer. Denise is a Fellow of the American Society for Quality and an Exemplar Global certified lead assessor and an ASQ certified quality auditor.

As principal of Robitaille Associates, she has helped many companies achieve ISO 9001 registration and to improve their quality management systems. She has conducted training courses for thousands of individuals on such topics as auditing, corrective action, document control, root cause analysis, and implementing ISO 9001. Among Denise’s books are: 9 Keys to Successful Audits, The (Almost) Painless ISO 9001:2015 Transition and The Corrective Action Handbook. She is a frequent contributor to several quality periodicals.