Measurement Systems Analysis for Attributes, Part 2

The analytical method allows more detailed quantification of the gage standard deviation and bias

Published: Monday, July 12, 2021 - 12:03

The first part of this series introduced measurement systems analysis for attribute data, or attribute agreement analysis. AIAG¹ provides a comprehensive overview, and Jd Marhevko² has done an outstanding job of extending it to judgment inspections as well as go/no-go gages. Part two will cover the analytical method, which allows more detailed quantification of the gage standard deviation and also bias, if any, with the aid of parts that can be measured in terms of real numbers.

Part one laid out the procedure for data collection as well as the signal detection approach, which identifies and quantifies the zone around the specification limits where inspectors and gages will not obtain consistent results. The signal detection approach can also deliver a rough estimate of the gage’s repeatability or equipment variation. Go/no-go gages that can be purchased in specific dimensions, or set to specific dimensions (e.g., with gage blocks) do indeed have gage standard deviations even though they return pass/fail results.

Part one also covered hypothesis test risk analysis, which quantifies how well inspectors agree with each other (reproducibility), themselves (repeatability), and the standard, i.e., whether the part is, in fact, good or bad. A drawback of attribute agreement analysis, however, is that the outcome will depend on the status of the parts themselves, and specifically how many are in the “gray area” around the specification limit.

Analytic method for bias and gage variation

The analytic method (AIAG reference, pp. 145–149) shows how to estimate bias and variation for an attribute gage. It requires at least eight parts, with six in Zone II (the region around the specification limits where good parts may be rejected and bad ones accepted), and m measurements per part, with m ideally being 20. The probability of acceptance is defined as follows if the part is accepted a out of m times. Note that this is for the gage for the lower specification limit. If a separate gage is used at the USL, it must be assessed similarly, except the smallest measurements will be acceptances, and the largest ones rejections. The data are organized as follows:
• Second largest reference value for which a = 0 (no acceptances), P_a = 0. Note that we cannot plot a probability of 0 on a normal probability plot.
• Largest reference value for which a = 0, P_a = (a+0.5)/m
• a>0 but a/m<0.5, P_a = (a+0.5)/m
• a/m = 0.5, P_a = 0.5
• a<m but a/m>0.5, P_a = (a-0.5)/m
• Smallest reference value for which a = m (all accepted), P_a = (a-0.5)/m
• Second smallest reference value for which a = m, P_a = 1, but we cannot use a probability of 1 on a normal probability plot, either.

We therefore need at least eight suitable parts with 20 measurements per part. The measurement, as perceived by the gage, is the sum of:
• The reference value, or actual measurement
• The gage bias, if any, for the specification limit under consideration
• The gage standard deviation times a random standard normal deviate z. This can be obtained in Excel with NORMSINV(RAND()), but the random number will update every time we perform a calculation. Another method is to generate a table of random numbers u_ij from the uniform distribution between 0 and 1, and then use NORMSINV(u_ij), where i is the part, and j the measurement, e.g., 1 through 20 in this case.
• If the result is inside the specification limits, then the result is 1 for pass and otherwise 0 for fail. Add them to get the total acceptances.

I did this for the lower specification limit to get the results in the table below. Note that the first row is not usable because the acceptance probability of 0 does not have a finite standard normal deviate.

Reference values, acceptances, acceptance probability, and z score

The next step is to perform a linear regression (figure 1) of the reference value against the standard normal deviates of the acceptance probabilities. StatGraphics returns a slope of 0.00462 and an intercept of 0.4464. The AIAG manual defines the bias as the lower specification limit (0.45 in this case) minus the reference value measurement for which the acceptance probability is 50 percent, i.e., when z = 0 and therefore the intercept of 0.4464. The result is 0.0036 vs. the 0.005 that was used in the simulation; the go/no-go gage thinks the parts are 0.0036 units larger than they really are. This is probably how the 0.4475 part, which is 0.0025 below the LSL, managed to get accepted 15 out of 20 times.

Figure 1: Linear regression, normal probability plot

The AIAG manual adds that the repeatability (gage variation) can be estimated by dividing the difference between the reference values (range R) for which the acceptance probabilities are 0.995 and 0.005, respectively, by 1.08. This unbiasing factor is specific for 20 measurements, and the reference does not contain a table that provides it for other quantities. The standard normal deviates for these quantiles are 2.576 and –2.576, respectively, for which the fitted values are 0.4583 and 0.4348.

Then:

0.00423 compares favorably to the 0.004 used in the simulation.

In summary, then, the analytical method will return estimates of 1) the gage bias; and 2) the gage’s standard deviation. This is actionable information that tells us we may need to 1) recalibrate the gage; and/or 2) select a gage with less variation, if such is available, and/or 3) tighten the acceptance limits to protect the customer from poor quality. The latter procedure is known as guard banding.

Analytical method in StatGraphics

Using the same data in StatGraphics delivers the results shown below. The estimated bias is 0.0036, which is not a surprise. The unadjusted repeatability is similarly 0.00245 (vs. the 0.00235 calculated above), which, when divided by the unbiasing factor of 1.08, yields 0.00227. Note also that the acceptance probability is 50 percent not at the LSL, where it should be, but at 0.4464, which reflects the bias.

Gage Study for Attributes—Analytical Method—Ref value
Reference values: Ref value
Number of acceptances: Accepts
Number of trials: 20

Upper spec. limit: 0.55
Lower spec. limit: 0.45

Regression model
Innermost reference value with no rejections: 0.455
Innermost reference value with no acceptances: 0.4375
Z-score = –93.865 + 210.27*Ref value
R-squared = 97.189%

Bias analysis
P(acceptance) = 0.5 at 0.4464
Comparison spec. limit = 0.45
Estimated bias = 0.003599

Repeatability analysis
P(acceptance) = 0.005 at 0.43415
P(acceptance) = 0.995 at 0.45865
Estimated repeatability (unadjusted) = 0.0245
Estimated repeatability (adjusted) = 0.022685
AIAG t-statistic = 4.9657
P-value = 0.000085829

StatGraphics result, analytical method

The AIAG t-statistic with (20–1) = 19 degrees of freedom tests the null hypothesis that the bias is zero. In this case, we reject the null hypothesis to conclude that gage bias exists. StatGraphics also returns the gage performance plot in figure 2. The plot is 0.0036 units to the left of where it ought to be because the go/no-go gage thinks the parts are 0.0036 units larger than they really are. That is, removing the bias would move the curve 0.0036 units to the right to place the 50-percent acceptance point on the LSL. Note, however, that the performance plot might not apply to the USL if a separate gage is used to assess the USL. A plug gage, for example, has a “go” end that must fit into the hole to ensure that it is larger than the LSL, and a “no-go” end that must not fit at the USL.

Figure 2: StatGraphics gage performance plot

Jody Muelaner³ includes a similar study, this one with m = 25 measurements per part, for which the specification limits are 12.000 to 12.027 millimeters. This was a very well planned study with data from parts ranging from 11.995 to 12.005 mm, with 1 micron increments, around the lower specification limits. Note that we can’t estimate the gage standard deviation because, since m = 25, we don’t have the necessary unbiasing coefficient (1.08 for m = 20).

Analytic method, upper specification limit

The AIAG example applied the analytic method only to the lower specification limit. Recall that we are actually using two gages, so let’s try the same thing at the upper specification limit, as shown in the table below.

Reference value, acceptances, and z scores, upper specification limit

StatGraphics obtains the results shown below. Note that the t statistic of 0.61, whose P value (significance level) is 0.549, tells us we cannot reject the null hypothesis that the gage bias is zero, and none was simulated at the USL. Note also that the acceptance probability at the USL is close to 0.50, as it ought to be. Dividing the adjusted repeatability of 0.0194 by 5.15 estimates the gage standard deviation at 0.00377, which is close to the 0.004 used in the simulation.

Gage Study for Attributes—Analytical Method—Ref value
Reference values: Ref value
Number of acceptances: Accepts
Number of trials: 20

Upper spec. limit: 0.55
Lower spec. limit: 0.45

Regression model
Innermost reference value with no acceptances: 0.558
Innermost reference value with no rejections: 0.544
Z-score = 135.24 – 245.73*Ref value
R-squared = 95.079%

Bias analysis
P(acceptance) = 0.5 at 0.55038
Comparison spec. limit = 0.55
Estimated bias = -0.00037824

Repeatability analysis
P(acceptance) = 0.005 at 0.56086
P(acceptance) = 0.995 at 0.5399
Estimated repeatability (unadjusted) = 0.020965
Estimated repeatability (adjusted) = 0.019412
AIAG t-statistic = 0.60989
P-value = 0.54916

StatGraphics Results, USL

Figure 3 shows that the 50-percent acceptance point is almost exactly where it belongs—on the USL of 0.55. The portion at the LSL should be ignored because a different gage is in use there.

Figure 3: Gage performance curve, USL

This section has shown how to estimate the gage bias and standard deviation by means of the analytical method. The final question concerns what we can do with the information.

Guard banding

The purpose of any statistical exercise is to deliver actionable information. If we know that the go/no-go gage has equipment variation and/or bias, what can we do about it? Perhaps bias can be dealt with by recalibrating the gage, but variation will remain. If we know the costs associated with 1) shipping bad parts to the internal or external customer; and 2) rejecting good parts, we guard-band by setting the acceptance limits inside the specification limits.

We can, in fact, optimize the acceptance limits to minimize the total costs in question. This is done by double-integrating the joint probability-density function of the process (as quantified, for example, by a process capability study with real-number data) and that of the gage, given the part dimensions. That is, f(x)φ(y|x), where f(x) is the probability density function of the part dimension (given, for example, the process mean and standard deviation. φ(y|x) is the normal probability function of the measurement (y) that the gage returns, given x and the gage’s standard deviation.

As an example, nonconforming work that is generated (the integrals from a lower limit to the LSL and the USL to an upper limit) but is accepted (the measurement is between the specification limits) can now be calculated as follows:

f(x) need not be the normal distribution if the critical-to-quality characteristic follows a different one, but that of the resulting measurement should generally be normal. This allows conversion of the integral across φ(y|x) into an expression that uses the cumulative normal probability (Φ), for which Excel has a built-in function.

That is,

This reduces the job to a single integration that can be handled with a Visual Basic for Applications (VBA) function that uses Romberg integration. The lower and upper limits are selected to encompass the range in which parts can be expected realistically to be made.

For any set of tightened acceptance limits, we can now quantify the total of 1) the fraction of parts that are bad and are accepted times the cost of external failure; and 2) the fraction of parts that are good and are rejected times the marginal cost of replacing the parts. Excel’s Solver feature can even minimize this cost. If the parts are expensive, meanwhile, gages could be used to sort out borderline ones from clearly bad ones, and the former could then be measured with more sophisticated gages or instruments that return real-number data.

Other approaches also might come to mind, but the key takeaway is that, once we quantify the gage’s standard deviation, we can develop optimal remedies to any deficiencies in the measurement system.

References
1. Automotive Industry Action Group. Measurement Systems Analysis, 4th Edition, Section C: Attribute Measurement Systems Study, 2010.
2. Marhevko, Jd. “Attribute Agreement Analysis (AAA): Calibrating the Human Gage!” Statistics Digest, The Newsletter of the ASQ Statistics Division. Vol. 36, No. 1, 2017.
3. Muelaner, Jody. “Attribute Gage Uncertainty.” Engineering.com, 2019.
See also McCaslin, James, and Gruska, Gregory. “Analysis of Attribute Gage Systems.” 1976 ASQ Technical Conference Transactions, Toronto, for the analytic method.

About The Author

William A. Levinson

William A. Levinson, P.E., FASQ, CQE, CMQOE, is the principal of Levinson Productivity Systems P.C. and the author of the book The Expanded and Annotated My Life and Work: Henry Ford’s Universal Code for World-Class Success (Productivity Press, 2013).