Managing the Business Within Your Business

Seven Essentials for Improving Test Environment Performance

Published: Tuesday, July 5, 2005 - 21:00

Brutally competitive markets are driving companies to design, build and improve their products faster and at lower costs. Faced with this economic climate, companies are understandably intent on freeing up resources—capital, engineering time and even plant space—that can be reallocated to high-growth, high-margin endeavors.For companies in high-tech electronics, testing is a mission-critical part of the new product development cycle. In these companies, test-related functions—and the engineers who perform them—consume enormous amounts of resources and company time.

The term “test environment” refers to the people, tools, practices and roles required to execute test-related functions. However, the test environment remains largely overlooked as a potential source of dramatic improvements in profitability, speed and engineering productivity.

In many companies, people either don’t see or don’t put the appropriate attention on this business within a business.” Like any business without strong metrics and rigorous financial statements, test-environment businesses tend to produce poor results at the bottom line. The symptoms of poor performance are easy to see if you know where to look.

Recognizing the symptoms of poor performance
The following are among the most common signs that a test environment is producing suboptimal business results:

• Hoarding test equipment is the norm. Instead of redeploying underutilized equipment already on site, the company purchases, rents or leases additional equipment to support new projects and respond to changing requirements.
• There’s no system for rigorously measuring test asset utilization. This prevents people from seeing that the company is accumulating too many assets, and spending too much time and money supporting these assets. Meanwhile, the hidden costs of procuring, maintaining and tracking unused equipment increase every day—severely limiting investments in better technology.
• Engineering personnel spend countless unproductive hours researching, locating, procuring, setting up and tracking their test infrastructure. This needlessly increases costs and drives up time-to-market on critical projects.
• The absence of equipment standards precludes knowledge sharing, collaboration and innovation—all while driving up maintenance costs.
• Internal silos ensure new test setups and routines are created for every project, meaning the company repeatedly misses opportunities to leverage effective, already-existing setups and methods. Days or weeks in engineering man-hours are needlessly expended. 

Quantifying the costs—and the opportunities
To understand the magnitude of the opportunities for improvement, consider industry standards for test equipment utilization. Based on my company’s work and conversations with hundreds of professionals in the high-tech product industry, we know that test equipment asset utilization typically ranges from 5 to 20 percent. A single piece of test equipment can cost anywhere between $5,000 and $150,000 when purchased new. Low utilization of these assets drives up capital spending 50 to 75 percent higher than necessary and reduces opportunities for investment in other, more strategic areas.

Low utilization is also a symptom of poor coordination in engineering and operations that produces waste in terms of lost time and rework. These inefficiencies are perpetuated by the lack of metrics for measuring internal process quality. The results? Pressures and delays in schedules, which invariably drive companies to throw money and resources at projects to make up for lost time.

The primary internal customers of your company’s testing function are the users of test equipment: namely, your design and development engineers. Manage your test environment effectively, and the potential value-creation for these customers includes access to better technology and many man-years of productivity that are real and measurable.

The enterprise also wins by freeing up cash and engineering time that can be reallocated to pursuits that deliver greater value for external customers (including shareholders).

Missing the big picture
A test environment is at once costly to maintain and critical to the success of the enterprise. In many companies, however, no one group is responsible for the centralized management of the test infrastructure.

In the test group, engineers and technicians are rightly focused on core tasks related to testing such as designing for testability, cutting cycle time in high-volume production areas, designing testing fixtures, etc.

It’s equally unlikely that their colleagues in the calibration lab are managing the test infrastructure in a holistic fashion. Instead, calibration lab team members focus on the work they were hired to do: calibrating and repairing test equipment.

In some cases, test or calibration groups handle tactical test infrastructure management tasks such as equipment acquisition. But the latter function is typically executed without benefit of a broader business plan—meaning the enterprise misses opportunities to cut costs, reduce time-to-market and improve engineering productivity through strategic test environment management.

Start by seeing the business within your business
Companies can’t harness the above opportunities until they’re first willing to treat the test environment as a distinct functional entity.

This “business within a business” perspective creates a powerful framework for 1) understanding your test environment’s value-creation model in financial terms; 2) justifying investment in your test environment; 3) measuring effectiveness and execution over time; and 4) recognizing what roles need to be held by the people on test environment management teams.

Point No. 4 is important because in any business, producing and sustaining results requires people to hold and understand clearly defined roles. In an effective business, people hold promises for sales, marketing, operations, finance, etc. In an effective test environment business, people hold promises for driving program participation (sales), fulfilling on end-user requests (operations), developing more valuable services for internal customers (marketing), measuring value (finance), etc. This makes it easier for people on the test environment management team to understand their roles and see how their performance contributes to the success of the test environment business and the broader enterprise.

Improving test environment performance: seven essentials
When developing an initiative for strategically managing your test environment, you’ll increase likelihood of success by following these seven principals:

1) Use cost savings to make the case for change.

Shifting to strategic test environment management will require new investments of time and money. Leaders in operations, finance and engineering will be the investors for this business within your business. They’ll compare the various investment options available to them and prioritize these options based on expected return.

Like the enterprise’s shareholders, these leaders are generally interested in growing revenues and reducing costs where they can.

Accordingly, we recommend an approach to quantifying value that focuses on increasing profit through cost reduction as the primary value proposition driver. When making your business case for strategic test environment management, start with three elements that are easily understood and analyzed by the aforementioned functional leaders:

• Equipment costs—Including the income statement elements of depreciation, rental/lease expenses, noncapital equipment purchases and the gain/loss on the sale of equipment. This also includes the balance sheet elements of capital acquisition cost and accumulated depreciation.
• Service costs—Including all the costs for calibration recall, calibration and repair, and any third party outsource costs
• Ownership costs—Including property tax, acquisition process costs, inventory carrying costs and the interest expense from asset investment

Down the road, executives can use these same elements to make ongoing assessments of the program’s performance.

2) Don’t use quality improvements as the only justification for change.

In most organizations, there are ample opportunities to use strategic test environment management to support broader quality and customer satisfaction initiatives such as Six Sigma, ISO 9001 or TL9000.

These opportunities become considerably easier once your program has built a history of delivering results that are more easily measured (e.g., hard savings such as reduced test equipment costs). Hence, for purposes of making an initial business case, focus first on enhancing profitability through cost reduction.

Improved quality performance—as measured in requests kept, span and internal customer satisfaction—will follow as a direct result of better coordination between internal groups.

3) Beware of “silver bullet” solutions.

No matter how technically elegant a solution appears to be, it won’t deliver breakthrough results unless its designers account for all three of the components that drive action in any test environment: people, tools and practices.

“Silver bullet” test equipment asset management solutions—new software packages, centralized databases, equipment pools, etc.—have been implemented in scores of companies. Although much time has been devoted to developing the tools component, solution designers invariably neglect to consider how real people use tools, coordinate with each other, and embrace or reject new practices in a test environment. Disappointing ROI has been the most common outcome.

4) Focus on utilization.

Higher utilization levels allow companies to unleash existing, pent-up investment capital capacity. Management then has the flexibility to shift already-planned investments of capital to new areas, or hold on to newly released capital capacity for future programs while dropping the savings directly to the bottom line as a cost reduction.

Increasing utilization is also essential when it comes to maximizing the time of engineering personnel. An effective test environment management program improves utilization of engineering capacity by minimizing or eliminating the need for engineers to be distracted by administrative concerns related to test equipment.

In far too many organizations with test equipment asset management programs, a piece of equipment is counted as “utilized” when it is deployed out of a centralized inventory. But equipment frequently sits idle in labs in between uses. All the while, the company incurs all of the costs associated with the asset without a commensurate return.

A more powerful approach is to consider a piece of test equipment “utilized” when it is turned on and either sourcing or measuring a signal. Equipment that’s turned off and sitting on a lab bench isn’t considered utilized. This provides more accurate and complete data for determining whether capital is being needlessly expended.

5) Build your portfolio of practices.

An effective test environment management business uses an array of practices for assessing and improving business performance, including:

• Technology alignment practices that match test technology with the business’ technology roadmaps. Examples of these practices include technology plans and capability analyses
• Capacity alignment practices that match test technology with current and future demands. Examples include utilization analysis and scheduling
• Inventory management practices for managing the test equipment portfolio for lowest cost/highest ROI mix. Examples include financial ROI analysis, acquisition and disposition of assets.
• Logistics practices for collecting and distributing data and physical assets. Examples include request fulfillment and calibration recall.
• Measurements and reporting practices such as dashboards and customer satisfaction sensing

6) Measure quality improvements via customer-focused operational metrics.

In a strategically managed test environment, quality is expressed in terms that are relevant to internal customers.

• Requests kept (RK) is an operational metric to drive customer satisfaction. RK is measured by tracking your business’ record of meeting internal customers’ requests for test-related assets and support.
• Span is a measure of the amount of time between your earliest delivery and your latest delivery, relative to your promised delivery time. Span is a measure from Six Sigma that recognizes that an early delivery of goods can be as disruptive to your customer as a late delivery.
• Customer satisfaction is a leading indicator metric based on immediate and follow-up surveys designed to uncover problems before they drive project leads and end users to opt out of the test environment management program.

7) Start small.

Earning the trust of users and management is essential to the success of any test-related change initiative. When you introduce a new program at a project or department level, you enable test management team members to focus first on producing satisfaction for a small group of customers. The team can refine its practices while still delivering high levels of performance to users and measurable ROI to the business. This approach is especially important when the solution is unproven.

A gated approach also sets the stage for satisfied users to tout the program to their counterparts in other parts of the organization and renders management more open to a broader implementation in the future.

About The Author