Predictive Performance Measurements

Going beyond red-yellow-green scorecards

Published: Wednesday, September 23, 2009 - 05:00

A previous article of mine in this newsletter, “NOT Transforming the Data Can Be Fatal to Your Analysis,” addressed the need for appropriate transformations and a predictive performance measurement system.

The statistical business performance charting (SBPC) methodology that was described in the article can, for example, reduce firefighting when the performance measurement system replaces organizational red-yellow-green scorecards, which often have no structured plan for making goal-setting improvement objectives.

This article describes how organizations can benefit from a SBPC scorecard or dashboard system, which can guide them to the most appropriate performance measurement system actions or nonactions in manufacturing and transactional processes.

Predictive process metrics and goal setting

Organizations often describe their business performance using a table of numbers, stack bar charts, and pie charts. Red-yellow-green scorecards may also be used for an assessment of how well a function is performing relative to established goals, where, in this tracking-to-goal system, a red-colored metric is a signal for attention since a performance goal is not being met, while a green-colored measurement indicates a goal objective is being achieved.

Traditional performance scorecards present historical information for some time frames with no predictive statement. Business decisions made through the use of these charts are not unlike driving a car by only looking at its rear view mirror.

What organizations need is a predictive metric reporting system. This futuristic assessment can then be utilized so that if expected future performance is not desirable, adjustments can be made. This is not unlike making an automobile driving adjustment using a steering wheel or brake/gas pedal, where this mechanical intervention is analogous to incorporating process improvement activities.

In W. Edwards Deming’s book Out of the Crisis (Massachusetts Institute of Technology, 1982), Lloyd S. Nelson stated, “If you can improve productivity, or sales, or quality, or anything else, by 5 percent next year without a rational plan for improvement, then why were you not doing it last year?” To me this addresses a fundamental problem with the implementation of red-yellow-green goal-setting scorecards, which can lead to much firefighting and management by hope.

When setting goals, it is important to remember that the output of a process (Y) is a function of its inputs (Xs) and the step sequence, which can be expressed as Y=f(X).

What is needed is an honest assessment of how well these process managed systems are addressing the overall needs of customers and the business as a whole. The reason for doing this is to determine which of the following actions or nonactions, as presented in table 1, are most appropriate for any given situation.

This statistical business performance charting (or 30,000-foot level) methodology provides a high-level view of how the process is performing so that organizations can move toward achievement of the “3 Rs” of business—everyone doing the right things, and doing them right, at the right time.

We are not attempting through SBPC performance reporting to manage the process in real time. With this performance measurement system we consider differences between sites, working shifts, hours of the day, and days of the week to be a source of common-cause input variability to the overall process.

The SBPC system has two steps in its reporting. The first step is to evaluate the process for stability. This is accomplished using an individuals chart where there is an infrequent subgrouping time interval so that input variability occurs between subgroups. For example, if we think that Friday’s checkout time in a popular grocery store could be longer than the other days of the week because of increased demand, a weekly subgrouping frequency could be most appropriate when tracking the checkout time process.

The second step in the SBPC system is to determine a region of stability. If there is a recent region of stability, a predictive statement can then be made by considering data from the most recent stable-process-performance time frame to be a random sample of the future. If data are continuous, this prediction statement can be presented using a probability plot, where normal and log-normal are the most common types of distribution plots.

Real-data example: Comparing red-yellow-green scorecards to SBPC

Figure 1 illustrates an actual organizational red-yellow-green scorecard. This form of scorecard reporting is often appealing to managers in that they can examine the colors in the latest time frame to determine who needs to be working on improving goal-driven metrics. However, does this management approach move organizations toward achievement of the “3 Rs” of business? Let's assess the effectiveness of this practice by examining one of these metrics in detail.

Figure 2 evaluates in further details one of the metrics from the scorecard shown in figure 1. For this real application example, there were red indicators five of the 13 reporting periods. The SBPC that is also shown in this figure indicates that no process improvements were made, even though the color often changed from red to green. The SBPC reporting indicates that there is a common-cause nonconformance rate of about 33 percent.

Figure 1: Red-yellow-green tabular scorecard example. (Click for larger image).

From Figure 3.5 Integrated Enterprise Excellence, Volume II—Business Deployment: A Leader's Guide for Going Beyond Lean Six Sigma and the Balanced Scorecard, Forrest W. Breyfogle III, Bridgeway Books, 2008.

From an examination of the SBPC report-out portion of figure 2, we would need to work on step No. 3 in table 1, assuming that 33 percent level of nonconformance is a current business priority improvement need when considering other organizational measurement performances. A successful implementation of this process improvement option would be demonstrated by the SBPC individuals chart shift to a new, stable improved level of performance.

As noted earlier, when an organization utilizes a red-yellow-green scorecard system, it takes action whenever the color is red; however, the SBPC reporting indicates that in this example, all the red-colored events are common-cause variability. That is, the perceived improvements when changing from red to green were simply common-cause variability occurrences. From the SBPC reporting, we would conclude that no process improvements were, in fact, made, even though the color changed from red to green several times.

When undertaking process improvement effort (table 1, third item), assignable causes that negatively effect process performance from a common-cause point of view can be determined by examining data in the latest region of stability. Collected data in this most recent stable region can be used to test hypotheses that assess differences between such factors as machines, operators, day of the week, raw material lots, and so forth. This investigation can provide insight to where focus should be given to determine what might be done or what further investigations to make to improve the process. This is a more efficient analytical discovery approach than reacting to a red signal and expending effort trying to determine why the negative-to-goal signal occurred.

Figure 2: Comparison of a red-yellow-green scorecard to SBPC predictive measurement reporting. Histogram included for illustrative purposes only. (Click for larger image).

From Figure 3.15 Integrated Enterprise Excellence, Volume II—Business Deployment: A Leader's Guide for Going Beyond Lean Six Sigma and the Balanced Scorecard, Forrest W. Breyfogle III, Bridgeway Books, 2008.

Improvement to the system would be demonstrated by a statistical significant shift of the SBPC report out to a new, improved level of stability.

Conclusions

In many organizational business systems, there is a need for providing predictive measures. SBPC provides a charting system that addresses this need.

With this form of reporting, when there is a recent region of stability, we can consider data from this time frame to be a random sample of the future. With this statistical business performing charting approach, we might be able to report that our process has been stable for the last three days, three weeks, three months, or three years with a prediction that 20 percent of the incoming calls to a call center are on hold longer than 40 seconds.

Organizations gain much when they integrate SBPC metrics within their business functional process map, where the organizational chart is subordinate to this enterprise view process map. When the enterprise is analyzed as a whole with a blending of analytics, goals can be set for SBPC metrics that benefit the business as a whole (i.e., avoiding silo process improvement efforts that might sound good but are not as beneficial as they initially seem relative to the big financial picture). The enterprise execution roadmap to accomplish this is something that can be addressed at another time.

About The Author

Forrest Breyfogle—New Paradigms

CEO and president of Smarter Solutions Inc., Forrest W. Breyfogle III is the creator of the integrated enterprise excellence (IEE) management system, which takes lean Six Sigma and the balanced scorecard to the next level. A professional engineer, he’s an ASQ fellow who serves on the board of advisors for the University of Texas Center for Performing Excellence. He received the 2004 Crosby Medal for his book, Implementing Six Sigma. E-mail him at forrest@smartersolutions.com