Featured Product
This Week in Quality Digest Live
Standards Features
Megan Wallin-Kerth
MasterControl’s Matt Lowe talks competition, data, and what quality does for a company
Using the CASCO Toolbox to repair and restore
Todd Hawkins
Good documentation is worth the effort you put into it
Rupa Mahanti
With data as the fuel of the digital economy, data governance is the much needed brake
Bryan Christiansen
Workplace policies that reduce accidents and enhance safety

More Features

Standards News
Creates one of the most comprehensive regulatory SaaS platforms for the industry
Yotrio and SunVilla to provide interactive, 3D-enabled assembly via BILT app
KCP25C grade with KENGold coating sets new standard for wear and productivity in steel turning
Takes action to mitigate risk of medical devices shortage
Standards, testing help autonomous vehicles drive safely
Offering production versatility and throughput with increased detection sensitivity and low energy consumption
New lines improve software capability and analysis
Automotive cybersecurity on Feb. 9, and AS9145 on Feb. 28

More News

Denise Robitaille


Do Your Metrics Measure Up?

Assessing the right customer satisfaction data is important if you want to improve your organization.

Published: Monday, May 3, 2010 - 10:05

ISO 9001 makes multiple references to the need to understand customer requirements and to monitor the extent to which those requirements have been fulfilled. In fact, the scope of the standard clearly indicates that its focus is primarily on ensuring that an organization has the processes that support its ability to fulfill customer requirements. Therefore, it stands to reason that gathering and analyzing information about how well an organization is doing in terms of meeting customer requirements would be an essential feature of its quality management system. Subclause 8.4 requires organizations to analyze data relating to (among other topics) customer satisfaction.

The questions now are:

What data do you gather to assess customer satisfaction?

What metrics will provide a credible and reliable barometer of what your customers think of your organization and the products you furnish?

It is possible to gather the wrong information. This can be the result of a failure to properly determine what indicators will provide the best information. Or it can also originate from a misdirected need to develop metrics that paint a specific picture—that portray the organization in a glowing, albeit inaccurate, light.

I had occasion about a month ago to accompany my mother on one of her many visits to have her hearing aid checked and adjusted. The instrument was not functioning properly—or as advertised. My mother was getting increasingly frustrated with the company’s failure to fix the problem. She was also tired of waging a battle with an individual who was making her feel as though the problem lay with her rather than the hearing device.

As we sat in the reception area, patiently waiting our turn, I glanced over at a fairly large poster displayed near the door. It apparently provided statistical testimony to their customers’ satisfaction. In very large letters at the top, it said, “Overall customer satisfaction 98%.” Below that in smaller type were the following customer satisfaction metrics:

95% Live demonstration of hearing instrument

99% Staff friendliness and professionalism

98% Ease of making appointment

97% Knowledge of hearing instrument

96% Convenient office location


I turned to my mother and asked her if she could read the large print at the top of the poster and she said, “Yes.” Then I asked if she could read the other text below, to which she answered, “No.”

Consider the fact that the vast majority of hearing aids are sold to older individuals. Many of them can probably not read the smaller text. Therefore, the only statement of customer satisfaction that they will be able to read easily will be the one posted in large print. So they, like my mother, will be left wondering what’s wrong with them if more than 95 percent of the other hearing-aid users are satisfied. This is a subtle deterrent to challenging the technician who repeatedly tells them that the fault doesn’t lie in the instrument but in how they’re using it.

Consider now the other indicators of customer satisfaction. Not a single one relates to the fit, form, or function of the product. Nor is there a metric relating to the test administered before the correct hearing aid is selected. The dubious 98-percent customer satisfaction isn’t inaccurate. However, it’s based on criteria that are irrelevant to the most important features of the product—its performance. The assertions are not completely unimportant. We do want to interact with professional individuals who have knowledge of the product. But, again, this has little relevance to the biggest indicator that will truly attest to customer satisfaction.

These kinds of metrics are construed to generate a specific result. The tactic is to determine the outcome in advance and then to find the data that will support the conclusion. This is generally done to influence a decision maker or to make someone look good.

Companies sometimes manipulate data in similar ways when measuring on-time delivery, customer complaints, or internal performance indicators. Other organizations may simply have grabbed onto convenient metrics so they’ve got something to show during an audit. Either way, the results are at best ineffective and at worse counterproductive and deceptive.

Selecting appropriate metrics is the only guarantee that the information you are using to make decisions is reliable. Take the time to select the metrics that will delineate the information you need to improve your organization and your ability to serve your customers. Anything else is just playing with numbers.


About The Author

Denise Robitaille’s picture

Denise Robitaille

Denise Robitaille is the author of thirteen books, including: ISO 9001:2015 Handbook for Small and Medium-Sized Businesses.

She is chair of PC302, the project committee responsible for the revision to ISO 19011, an active member of USTAG to ISO/TC 176 and technical expert on the working group that developed the current version of ISO 9004:2018. She has participated internationally in standards development for over 15 years. She is a globally recognized speaker and trainer. Denise is a Fellow of the American Society for Quality and an Exemplar Global certified lead assessor and an ASQ certified quality auditor.

As principal of Robitaille Associates, she has helped many companies achieve ISO 9001 registration and to improve their quality management systems. She has conducted training courses for thousands of individuals on such topics as auditing, corrective action, document control, root cause analysis, and implementing ISO 9001. Among Denise’s books are: 9 Keys to Successful Audits, The (Almost) Painless ISO 9001:2015 Transition and The Corrective Action Handbook. She is a frequent contributor to several quality periodicals.


Do Your Metrics Measure Up?

There is nothing wrong with the selected metrics. Beyond product quality, these metrics may well reflect customer satisfaction. What's missing is some measure of product effectiveness. Also missing is the recognition that each element contributing to customer satisfaction may not carry the same weight (business impact). In most cases if the product does not perform, good performance in any, or all, other categories is indeed a moot point.

playing with numbers

Look at the five "metrics". They all represent what the organization thinks they need to do to achieve customer satisfaction - the things they can control, like making sure they are polite and give a live demonstration. I agree these don't really measure customer satisfaction - but they do measure the organizations compliance with doing the things they think they need to do. And I could not fault them for wanting to verify that they are doing all those things.
Maybe we should suggest what they should measure. I'd like to see them monitor the purpose for each appointment, and publish the metrics of how many appointments were for people coming in to get their hearing aids adjusted and fixed. Then they could have a numerical assessment. Of course, it would not neccesarily be data about customer perception because it would be data about reasons for appointment. But so what? Maybe if we generate enough good ideas Denise and her Mom can take them in to her next appointment. With the new level of respect her Mom will command it is likely her level of service will improve, as will her frustrations and ultimately her hearing!