Featured Product
This Week in Quality Digest Live
Metrology Features
Jamie Fernandes
From design to inspection to supply chain management, AI is transforming manufacturing
NVision Inc.
Scanning plays a role in extending life span and improving design of A/C systems
Patrice Parent
Integral components of an electric vehicle’s structure are important to overall efficiency, performance
NIST
NIST scientists perfect miniaturized technique
Donald J. Wheeler
The more you know, the easier it becomes to use your data

More Features

Metrology News
Partnership will lead to comprehensive, integrated manufacturing and surface inspection solutions
Enables better imaging in small spaces
Helping mines transform measurement of blast movement
Handles materials as thick as 0.5 in., including steel
Presentation and publication opportunities for both portable and stationary measurement leaders
Accelerates service and drives manufacturing profitability
Improved readings despite harsh field conditions
Designed to meet standards, bolster detection, and mitigate work environment hazards
Machine vision market will return to single-digit growth in 2024 following declines in 2023

More News

Metrology

Measurement Data and Getting the Basics Right

Part 4 of our series on SPC in the digital era

Published: Wednesday, December 6, 2023 - 12:03

Parts 1, 2, and 3 of our series on statistical process control (SPC) have shown how data can be thoughtfully used to enable learning and improvement—and consequently, better product quality and lower production costs. Another area of SPC to tap into is that of measurement methods. How do we ensure that the measurement data we get provide us with accurate process and product insight? Read on to see how SPC techniques play this key role as well as generating actionable and profitable insights from your data.

Measurement consistency

Standard samples are routinely measured as part of measurement monitoring programs, and an essential property of such samples is that they don’t change over time. (If the standard sample is of value 10 on Day 1, it retains the value 10 on days 2, 3, 4... and so on.) A commonly used standard is a reference weight, because reference weights don’t change unless they are damaged or poorly handled.

In this first example, Figure 1 uses reference weight data over 14 days of continuous production, with a measurement performed by a production operator at the start of each shift, and three shifts operating daily. The consistency of these measurements is examined with a control chart of individual values, herein called a consistency chart. The term consistency chart is taken from Donald J. Wheeler’s book, EMP III (Evaluating the Measurement Process): Using Imperfect Data (SPC Press, 2006).

A graph with lines and dots  Description automatically generated
Figure 1: Consistency chart of reference weight measurements over 14 days

Consistency charts are interpreted in the same way as standard SPC control charts. So, with no signal of inconsistency found in Figure 1—no points beyond the limits, no sequence of nine or more points on one side of the central line, and no systematic patterns—measurement consistency is reasonably demonstrated. The limits on Figure 1 tell us what to expect should measurement consistency be sustained: Each new value to be in the range of 599.775–600.059 g.

Any single value outside this range is taken as a signal of measurement inconsistency. Possible causes would include a problem with the reference weight itself, some problem with the weighing scale, or the way the scale is used. A signal of measurement inconsistency is therefore a warning of a problem with the measurement method and, consequently, a concern about the data coming from that measurement method.

Measurement bias

An essential property of a standard sample is that it has an accepted value against which measurement data can be compared. The accepted value of the reference weight in this example is 600.0 g.

Having an accepted value allows the question of bias for the reference weight measurements to be examined. To put this in context, if your height is 177 cm and the measurements regularly put you incorrectly at 179 cm, the measurements are misleading due to the positive bias of 2 cm. Biases offer nothing of value, only misleading data and waste. When a bias is discovered, the wise course of action is to seek its immediate elimination. But first you have to detect the bias.

How to check for a bias

If a bias is present, the data will be routinely too high or too low. An easy way to get a “feel” for a possible bias is to indicate the accepted value on a histogram of the original values and judge whether the accepted value is close to the middle, or balance point, of the histogram or not. This is shown in Figure 2 for the reference weight data. Visually, we see that the histogram is “unbalanced”—too far to the left—in relation to the accepted value. With this we can suspect the presence of a bias.

A graph of a weight  Description automatically generated with medium confidence
Figure 2: Histogram of the weight data with the accepted value of 600.0 g indicated

SPC allows us to confirm detection of the bias in a simple, valid way. Figure 1’s consistency chart can be adapted by making the central line equal to the accepted value of the reference weight (600.0 g). This is shown in Figure 3. (Note: The 3-sigma distance from the central line to the upper and lower limits, of value 0.142 g, does not change.)

A graph of a graph of a graph  Description automatically generated with medium confidence
Figure 3: Adaptation of the consistency chart to detect the existence of a bias

Why is the bias confirmed? In the right chart of Figure 3, we now see signals, including:
• Five points that fall below the lower limit
• 25 consecutive points that are below the central line (starting on Day 6’s third shift)

How big is the bias? This is estimated as follows, showing that, on the average, the routine weight data will be approximately 0.1 g too low.

Bias = Baseline average – Accepted value = 599.917 – 600.0 = –0.083 g

Is it sufficient to compare to tolerances?

In practice, the routine check performed by the operators on the shop floor didn’t include the use of consistency charts. Instead, each time the reference weight was measured the result was compared to the tolerances of 599.80–600.20 g, as illustrated in Figure 4. This checking procedure judged all to be OK because each and every measurement obtained over the two-week period was inside the tolerance range. Yet, as shown above, not everything was OK with the measurements.

A screenshot of a graph  Description automatically generated
Figure 4: Table of how production operators recorded and interpreted the reference weight data

Unnecessary waste

Two examples of loss that could arise from a bias as shown above are:

1. Increased scrap level: In order to meet a legally required minimum of 599.50 g in each product unit, a checkweigher could falsely reject some slightly underweight units that, with correction for the bias, would in fact satisfy the legal requirement for minimum net content.

Example: A unit of measured value 599.45 g would be rejected, yet with knowledge of the bias, the expected value would be OK at 599.53 g (599.45 + 0.083 = 599.533).

2. Product give away: This bias could also affect overfilling, known as “give away,” which does nothing except hurt a manufacturer. Without corrective action on an unknown bias, the settings on a filler could end up being adjusted so that, on the average, approximately 0.1 g extra would be added to each packing unit. An example of the losses this could generate is shown in Figure 5. Losses in the ballpark of $20,000 may not bankrupt a manufacturer, and they may never be uncovered. But why risk losses of this kind from a source of waste so easily eliminated by the simple correction of a measurement bias? Moreover, add on the losses from other sources of bias in a factory and the losses could be much greater than $20,000. 


Figure 5: Potential losses caused by overfilling due to an unknown measurement bias

A better way

A better way was put into practice as part of the onsite support from corporate’s technical department (during what we call Week 2 in Figure 1). Training was given on the use of consistency charts, and the bias was identified and eliminated.

In Week 3 the shop-floor team started to use, and was being coached on, consistency charts. (While the data from weeks 1 and 2 were control-charted retrospectively, the data in Week 3 were charted close to real time, what one might call proper SPC.)

The appropriate limits used to assess ongoing measurement consistency in Week 3 are those seen in the right chart of Figure 3. Why?
• With the bias eliminated, the central line can be at the value of the reference weight (600.0 g)
• The 3-sigma distance (0.142 g) is the same as in Figure 1

The updated consistency chart is shown in Figure 6. As expected, given the consistency demonstrated during the first two weeks (see Figure 1), measurement consistency continued to be demonstrated in Week 3. Figure 6’s consistency chart shows the measurements to be both consistent and free of bias. These two characteristics of a measurement system are key enablers of data generating the best possible actionable insights into the production process.

A screen shot of a graph  Description automatically generated
Figure 6: Measurement consistency chart up to the end of Week 3

Measurement inconsistency

Inconsistent measurements are equivalent to a rubber ruler: With a rubber ruler, the measured dimension of a part could depend as much on the ambient temperature as the actual length of the part. Such measurements can’t be relied upon to generate actionable insights into the production process. Thus, the need for measurement consistency. (Whom reading this article hasn’t been frustrated, or lost time, or made a wrong decision, or observed waste due to problems with measurement data?)

Figure 7 presents new data, from Week 4, for the measurement of the reference weight. We see that something happened at the start of Week 4. On the second shift of Day 22, the first measurement, of value 600.188 g and plotted in red, was clearly above the upper limit, prompting follow-through. The shift supervisor noticed that the standard weight had some remnants of powder on it, causing the elevated weight reading. A new operator had not properly checked the state of the reference weight prior to measurement. An update to the SOP was prepared and the operator, now aware of the error, remeasured the reference weight, having brushed it down. With the repeat measurement of value 600.109 g falling comfortably inside the measurement consistency limits, the issue had been resolved. This learning was then shared.

A screen shot of a graph  Description automatically generated
Figure 7: Measurement consistency chart up to the end of Week 4

Would this error have been detected by the monitoring-to-tolerance criteria initially used (see Figure 4)? No.

Measurement inconsistencies unnecessarily degrade the quality of measurement data, risking wrong decisions, wasted time, and wasteful actions. The best way of detecting measurement inconsistencies is by using consistency charts, not performance tolerances.

Actionable insights from your data

A member of corporate’s technical department stayed onsite to support better use of control charts for net-content data for the 600 g packaging format discussed above. To start, data from the last run were reviewed. Filler B, in operation for this run, has three filling heads, and every 10–15 minutes an operator collected data on three consecutive units, one from each of the three filling heads. The data from each sampling were organized into 87 subgroups, each of three measurements, so that the process data could be examined on an average and range chart, as shown in Figure 8.

A graph of a graph  Description automatically generated with medium confidence
Figure 8: Average and range chart for the net content data in time order of production

The signals on both the average and range chart urge investigation into the detected changes. The production logbook was reviewed, with the key observation that at about seven hours into the run, a stoppage occurred and the feeding screw on Filling Head 1 had to be replaced. This stoppage occurred just after the data for Subgroup 40 had been collected.

The first signal on the average chart is a run of 12 consecutive points below the central line, corresponding to subgroups 30–41 and the time of the line stoppage. The points on the chart then visibly shift up by about 0.2 g. The second signal, at the end of the average chart, further substantiates the occurrence of a sustained upward shift.

The next step in the investigation was to disaggregate the data to look at the measurements for each filling head. Three control charts for individual values were used, one per filling head, as shown in Figure 9. Two sets of limits are shown on each chart to better investigate any possible effect from the stoppage and the change of the feeding screw in Filling Head 1.


Figure 9: Control charts for individual values for each filling head

Figure 9 makes clear that the upward shift identified in Figure 8 was attributable to Filling Head 1, with the change in the central lines indicating an increase of about 0.45 g per unit on the average for this head.

Further investigation revealed that the replaced feeding screw was from Filler A and had been mistakenly fitted. This “wrong” screw dosed too much powder into the packaging units, causing approximately 10 kg of finished product powder to be lost as “give away” over the remaining eight and a half hours of this production run.

The power of SPC to help discover the unknown is apparent in this example. Discovery promotes learning, and the learning generates actionable insights from which product quality can be improved and production costs lowered.

Working with a rapid method on the shop floor

In this second example, corporate’s technical team visited another plant in their quest to drive improvement throughout the organization. One of the points under evaluation was the potential to optimize moisture content in powder products. Starting first on the most-used production line, powder product samples were routinely taken every 30 minutes just before filling and measured for moisture content in a rapid method on the shop floor. These data were used as part of product release and process monitoring.

The master method for moisture measurement was located in the plant’s main laboratory. Approximately twice a week, production samples, having already been measured in the rapid method, were sent to the lab for measurement on the master method as part of the rapid method’s monitoring program. The data used and the monitoring tolerances applied are shown in Figure 10.

Diagram of a sample  Description automatically generated
Figure 10: Overview of difference values in the monitoring of a rapid measurement method

The corporate team (in January 2021) started their study by reviewing the data from 2020. A control chart of the difference values was created to get a first impression (see Figure 11). (As per Figure 10, a difference value is the difference between the rapid measurement value and master measurement value for the same sample.)

A graph on a white background  Description automatically generated
Figure 11: Control chart of the moisture difference values throughout 2020

With three excursions below the lower limit, and a long run of points above the central line in November and December, inconsistencies in alignment between the rapid and master methods are detected. These signals indicate deteriorated measurement quality that can only bring waste to operations.

The team’s investigation gave priority to the most recent signal from November and December. The lack of alignment between the rapid and master methods indicates that the moisture measurements from the rapid method were, on the average, almost 0.1 g/100 g too high over at least one and a half months (given consistency and no bias in the master method, which was verified by the laboratory). Due to the standard practice of comparing the difference values to the tolerances of ±0.18, nobody in the plant had been aware of this misalignment between the two methods at the end of the year.

Further investigation identified a change in raw material supply in mid-November as the most likely cause of the change in the rapid method measurements. A recalibration of the rapid method was carried out to realign the rapid method with the master method.

Three potential and negative consequences of the bias in the rapid method’s moisture measurements are:
1. Misclassification of product vs. specification limits
2. Unnecessary changes to process parameters during production
3. Product losses by making product of lower-than-expected moisture content, i.e., approximately 0.1 g/100 g below the target value

Further to Point 2, an estimation of the potential losses due to “lost product” during a six-week period is shown in Figure 12. Over one year, a loss of this kind could reach about $120,000 ($13,709 x 52 / 6) = $118,811). As with the other examples in this series, we again show how SPC can help manufacturers to improve quality, lower costs, and reduce waste.


Figure 12: Potential losses over six weeks caused a bias in the rapid method’s moisture measurements.

Summary

Good measurements are an essential foundation of excellence in manufacturing. Cracks in this foundation may be invisible to you, but be assured they will be there, and causing losses somewhere, if inconsistencies or biases are present in your measurement data.

As shown here, SPC techniques can play a key role in ensuring the measurement data we get provide us with meaningful process and product insight. With good measurement data at hand, you can generate the best possible actionable insights from your data to drive quality improvement, lower costs, and reduce waste.

Please share your thoughts on this topic: Do you use SPC techniques to monitor the consistency of your measurement methods? Do you have a story to tell of the waste you’ve suffered due to “bad” data?

Discuss

About The Authors

Scott A. Hindle’s picture

Scott A. Hindle

Scott A. Hindle has been using data to study and improve processes, and actively working in the field of SPC, for close to 15 years.

Douglas C. Fair’s picture

Douglas C. Fair

A quality professional with more than 35 years of experience in manufacturing, analytics, and statistical applications, Douglas C. Fair is the former chief operating officer at InfinityQS International, an SPC software company. At InfinityQS, he spent 25 years helping manufacturers around the world deploy SPC and benefit from its use. 

Fair holds a bachelor’s degree in industrial statistics from the University of Tennessee, and a Six Sigma Black Belt from the University of Wisconsin. He’s a regular contributor to various quality magazines and has co-authored two books on industrial statistics: Innovative Control Charting (ASQ Quality Press, 1998) and Quality Management in Health Care (Jones and Bartlett Publishing, 2004).

Comments

Measurement Data and Getting the Basics Right Part 4

Hi Scott,

Have done done similar many times and especially the application on making margarine, engine manufacturing, and bio-medical labs.

As mentioned earlier in my comments, I wish those U/LCL lines were reflecting the Voice of the Process and be dotted and not full lines.

Secondly, I find the placement of the Histogram to the right of the Control Chart is valuable for understanding the distribution but then on it, placing the U/LSL and seeing where the Control Cart readings fall into the Voice of the Customer.

Mike

Mike

Thanks for the comments and happy to hear you have successfully applied the ideas discussed above in your work.

Dotted/dashed lines .... In all these years I'd never paid much attention to the process limits being solid or dashed. I see in Shewhart's 1931 and 1939 books they are dashed. My main focus, or concern, has tended to be that the two voices (process & customer) are understood, i.e. why they are different and what they represent.

Measurement Data and Getting the Basics Right Part 4

Hi Scott,

Have done done similar many times and especially the application on making margarine, engine manufacturing, and bio-medical labs.

As mentioned earlier in my comments, I wish those U/LCL lines were reflecting the Voice of the Process and be dotted and not full lines.

Secondly, I find the placement of the Histogram to the right of the Control Chart is valuable for understanding the distribution but then on it, placing the U/LSL and seeing where the Control Cart readings fall into the Voice of the Customer.

Mike