Featured Product
This Week in Quality Digest Live
Statistics Features
Donald J. Wheeler
The more you know, the easier it becomes to use your data
Scott A. Hindle
Part 7 of our series on statistical process control in the digital era
Donald J. Wheeler
How you can filter out noise
Scott A. Hindle
Part 6 of our series on SPC in a digital era
Douglas C. Fair
Part 5 of our series on statistical process control in the digital era
Statistics News
How to use Minitab statistical functions to improve business processes
New capability delivers deeper productivity insights to help manufacturers meet labor challenges
Day and a half workshop to learn, retain, and transfer GD&T knowledge across an organization
Elsmar Cove is a leading forum for quality and standards compliance
InfinityQS’ quality solutions have helped cold food and beverage manufacturers around the world optimize quality and safety
User friendly graphical user interface makes the R-based statistical engine easily accessible to anyone
Collect measurements, visual defect information, simple Go/No-Go situations from any online device
Good quality is adding an average of 11 percent to organizations’ revenue growth
Ability to subscribe with single-user minimum, floating license, and no long-term commitment
Statistics

### It often pays to be dubious

Published: Thursday, June 30, 2016 - 15:22

Approaching the end of the school year means focusing on graduation rates, dropout rates, and other data suggesting trends for students. Opportunities for considering statistics abound, but one must examine the way that these statistics are actually used by asking the right questions about the data.

For example, as teachers finish state testing regimens and head into final exams, it may be useful to see data related to average pay for teachers. Is it going up? The following figures from 1969–2014 suggest that it is:

 1969–70 1979–80 1989–90 1999–2000 2009–10 2011–12 2012–13 \$8,626 \$15,970 \$31,367 \$41,807 \$55,202 \$55,418 \$56,383

Wow. Based on the numbers in this table, you could say that average teacher salaries have gone up by more than 700 percent since 1969.

What one sometimes forgets, though, is that inflation and other factors contribute to a need to calibrate salaries to current dollars, meaning what they represent in today’s terms. Here are the numbers, as calculated in 2012–13 dollars (the latest year for which the National Center for Education Statistics has reported):

 1969–70 1979–80 1989–90 1999–2000 2009–10 2011–12 2012–13 \$52,830 \$47,592 \$57,152 \$57,133 \$58,925 \$56,340 \$56,383

This adjusted data tells a different story, with average salaries not changing so radically over the years. But notice, too, that in these charts, the intervals are not consistent; the first four are 10-year intervals, while the last three are 2-year and 1-year intervals. “What are the intervals in the data collection?” is another question that must be raised. Comparing the jump from 1969–70 to 1979–80 is not the same thing as evaluating the difference between 2009–10 and 2011–12.

A control chart that reflects this data may give a greater sense of information from the data, but since the increments are irregular, the chart is as specious as the original data:

The comparison points to the necessity for being careful with statistics. As Mark Twain noted, “There are lies, damn lies, and statistics.” Statistical methods, when applied carelessly, distort what can be seen as reality. Sometimes this happens accidentally, and sometimes statistics are simply made up to support a pre-held conviction.

Here are some things to look for to verify that statistics are actually accurate:
1. If dollar figures are given over time (as in the example above) be sure that the adjustment for CPI rates or other economic factors have been factored in.
2. Check the vertical axis of any chart, to be sure that the intervals are consistent and reasonable. Statistics can be distorted by altering the consistency in the y axis. If, for example, some intervals are 5 and others are 10, be suspicious.
3. Verify the source of the data in the chart. It may be anecdotal, as in the case of charting the number of items that women have in their purses. If it comes from a reliable source, you may have more confidence in its accuracy.
4. Verify the operational definition for what is being measured (e.g., is a string bag considered a purse? What about a backpack? Or a wrist wallet?)
5. Look at the sample and the integrity of sampling methods that have been utilized. If a sampler surveys a group of his friends, for example, be suspicious. The sampling should be truly random, and large enough to suggest trends.
6. Use a control chart to assess the data for stability, using standard out-of-control tests that include patterns such as too many points around the mean.

Although applying statistics in a way that promises accuracy demands more than these few rules, applying these standards will provide a clue about the data and help evaluate its integrity.