How to Select Design of Experiments Software

During the 1920s, a British statistician named Ronald Fisher put the finishing touches on a method for making breakthrough discoveries. Some 70 years later, Fisher's method, now known as design of experiments, has become a powerful software tool for engineers and researchers.

But why did it take engineers so long to begin using DOE for innovative problem solving? After all, they were ignoring a technique that would have produced successes similar to the following modern-day examples:

• John Deere Engine Works in Waterloo, Iowa, uses DOE software to improve the adhesion of its highly identifiable green paint onto aluminum. In the process, the company has discovered how to eliminate an expensive chromate-conversion procedure. Savings: $500,000 annually.

• Eastman Kodak in Rochester, New York, learns via DOE software that it needs only to retool an existing machine instead of making a huge capital purchase for a new one. The solution means improved, light-sealing film-pack clips used by professional photographers. Savings: Setup time drops from eight hours to 20 minutes; scrap reduces by a factor of 10, repeatability increases to 100 percent and $200,000 is not spent on a new machine.

• Ski manufacturer K2 Corp. in Vashon, Washington, is hampered by a new ski's complex design that produces a 30-percent higher scrap rate. DOE finds the reason and the solution. Savings: Press downtime tumbles from 250 labor-hours per week to a mere 2.5.

 

Earlier this century, DOE lay virtually dormant due to the complex and tedious hand calculations it required. In later years, mainframe computers requiring programming skills beyond most statisticians' scope chugged through complicated DOE equations. It wasn't until room-sized computers became desktop PCs that affordable DOE software for the nonstatistician first appeared. Today, off-the-shelf packages commonly sell for $500-$1,500.

DOE now offers a highly versatile technique useful to industries as diverse as textiles, tires, advertising research and human resources. In fact, it works anywhere people need to learn about a system's inputs. Because of DOE's incredible efficiency and its ability to uncover synergistic interactions, its potential is virtually unlimited. Figure 1 shows more examples of companies experiencing continuous improvement with DOE. Given its impressive range and flexibility, what should potential buyers consider when selecting DOE software?

Critical software elements

Think of a statistical software package as a tool in your quality solutions tool chest. Carpenters, for example, rough in new timber for a deck extension with a ripping saw and a large hammer. In contrast, a job requiring precisely aligned windowsill corners needs a specialized tool -- a miter box -- to produce the best results. Similarly, although all-encompassing statistical software packages sometimes contain a DOE component, frequently these aren't honed for easy, readily understandable DOE applications.

General-purpose statistical software constitutes the staple tool for everyday analytical tasks. But sometimes the power and ease-of-use of a dedicated tool dramatically simplifies the job. When selecting a statistical software package, therefore, begin by looking at its accompanying manual. If the chapters on DOE don't make up the bulk of the information, consider adding to your toolbox a dedicated DOE program in addition to a wide-ranging statistical package.

When selecting DOE software, it's important to look for not only a statistical engine that's fast and accurate but also the following:

• A simple user interface that's intuitive and easy-to-use.

• A well-written manual with tutorials to get you off to a quick start.

• A wide selection of designs for screening and optimizing processes or product formulations.

• A spreadsheet flexible enough for data entry as well as dealing with missing data and changed factor levels.

• Software that randomizes the order of experimental runs. Randomization is crucial because it ensures that "noisy" factors will spread randomly across all control factors.

• Design evaluation tools that will reveal aliases and other potential pitfalls.

 

First-rate packages offer sharp graphics for statistical diagnoses and displaying responses. Graphics include residual plots, which confirm a model's statistical validity. Functions such as square root or log base 10 allow users to transform their responses, thus improving the statistical properties of the analysis.

Interaction plots, another series of graphics, show how combined changes in two factors produce effects greater (or less) than the sum of effects expected from either factor alone -- positive and negative synergy, so to speak. Friendly DOE software lets users simply click on the relevant interaction to reveal nonparallel lines indicating how one factor's effect depends on the other.

Design of Experiments Defined Design of experiments provides information about factors and their interactions in systems as varied as aluminum welding and zoo-animal cholesterol. It points out breakthrough solutions by thoroughly evaluating multiple elements within a process and has become a useful tool for quality professionals.   DOE is superior to traditional one-factor-at-a-time experimentation, which ordinarily requires many more time-consuming test runs to obtain less-than-perfect knowledge of a process. Because one-factor-at-a-time can't reveal critical factor interactions, it offers a shaky substitute for a designed experiment. This means, for example, that color-pen researchers using one-factor-at-a-time might never learn that red ink and blue ink interact to make purple -- a severe oversight.   The best DOE software packages today set up and analyze statistically sound DOEs with ease. They do this by fitting data into mathematical equations that predict outcomes for any combination of values.   Consequently, by using DOE software, engineers, scientists and researchers in virtually any industry can optimize responses and discover winning combinations. The advantages commonly include carving six to 12 months off production schedules and reaping annual savings of six figures or more.

Both 2-D and 3-D plots present maps of the anticipated changes in the outputs as the factors, or inputs, change. Some DOE software can become flashy here, so look for mature packages, which usually have the best qualifications for providing what experimenters need. Aim for software that lets you control the graphical display by selecting plot axes, variable ranges for response surface designs and contour levels. The convenience of right-click mouse functions for 2-D, and rotating or tilting functions for 3-D, can greatly simplify your experimentation. Also, look for multiple response optimization via graphical methods that reveal the "sweet spot," as well as numerical tools, such as desirability, for finding the peak.

Analyzing experiment results shouldn't be a chore. To make your work easy, DOE software should determine which terms must be included in the model; compute all coefficient values with associated standard errors and confidence values; check "lack-of-fit" statistics and provide other diagnostics, especially for outliers, so users can assess the model quality.

A sound DOE software package also offers users the capability to augment an existing design. Augmentation is beneficial for completing an experiment that initially didn't provide sufficient information.

Good software includes good training

The best DOE packages have excellent training workshops behind them. Mark Anderson, a principal of Stat-Ease Inc., estimates that only 10 percent of engineers use DOE. Most potential users are intimidated by statistics and DOE's perceived difficulty, he believes. In an effort to alleviate this apprehension, Stat-Ease offers an introductory three-and-a-half-day computer-intensive workshop in addition to several more advanced classes. Graduates of courses like these can save their employers substantial sums annually.

Users new to design of experiments concepts sometimes need a jump-start in basic statistics. If this sounds like you, look for a workshop, or at a minimum, study on your own frequency distributions such as histograms, normal distribution, and cause-and-effect diagrams; range and standard deviation; mean, median and mode; and confidence intervals. DOE novices shouldn't need statistical knowledge beyond the above concepts if they choose a DOE software package with an effective user interface behind it.

Telephone support also is important. Find out how much phone help you can expect with interpreting a variance's analysis, for example, or in determining whether you should "block" an experiment. A good package is only as good as its technical support and the statistical expertise of ready-to-help consultants.

What's best for you?

Your definition of good DOE software probably differs from your colleagues'. Perhaps you emphasize ease-of-use whereas someone else may look for power. In either case, DOE software should help users set up appropriate designs. Once they are run, the software should provide a way to analyze and understand the results. Ultimately, the software should tell you how to change the inputs in order to manipulate the outputs.

But more important than selecting a DOE package is making the decision to begin using DOE now. Then you can join the growing group who have generated huge profits for their companies via improved quality and process efficiency.

DOE vendors

An addendum to the following list of vendors appears below this list.

ADDENDUM

ProtoLife
57 Post Street, Suite 906
San Francisco, CA 94104
(415) 689-1171
www.protolife.com

 

DOE examples

About The Author