Featured Product
This Week in Quality Digest Live
Six Sigma Features
Mark Rosenthal
The intersection between Toyota kata and VSM
Scott A. Hindle
Part 7 of our series on statistical process control in the digital era
Adam Grant
Wharton’s Adam Grant discusses unlocking hidden potential
Scott A. Hindle
Part 6 of our series on SPC in a digital era
Douglas C. Fair
Part 5 of our series on statistical process control in the digital era

More Features

Six Sigma News
Helps managers integrate statistical insights into daily operations
How to use Minitab statistical functions to improve business processes
Sept. 28–29, 2022, at the MassMutual Center in Springfield, MA
Elsmar Cove is a leading forum for quality and standards compliance
Is the future of quality management actually business management?
Too often process enhancements occur in silos where there is little positive impact on the big picture
Collect measurements, visual defect information, simple Go/No-Go situations from any online device
Good quality is adding an average of 11 percent to organizations’ revenue growth

More News

Tim Leary

Six Sigma

Case Study: Six Sigma in a Software Support Environment

Continuous process improvement.

Published: Tuesday, December 15, 2009 - 06:00

Story update 11/23/2010: A paragraph was added to the end of this case study to reflect the current state of the company's quality initiatives.

Acme Technology Services (not their real name) is a privately-held provider of technology-enabled business solutions. Acme’s retail software division provides software that enables merchants to better manage their supply chain from the suppliers through to the customers. ARS provides customization, sometimes extensive, to satisfy individual company needs. Due to a large sales team, the client list had steadily grown.

The software support structure has many nice features, including a web-based ticket capture system. Customers complete a data-capture screen for incidents and requests-for-service. Tickets are automatically routed to the support team queue via e-mail and are dispersed by a queue coordinator. ARS has some bright, resourceful individuals who extract and report on operations data.

Over time, sales began to slow and become increasingly difficult. The software development cycle time slowed down while customer complaints increased. Profitability continued to fall. There was also a growing trouble-ticket backlog.

Acme’s CEO heard and read about Six Sigma and lean initiatives and how they helped many companies improve operations. He hired a Six Sigma Black Belt, we’ll call him B. Better. Better had a lot of experience with Six Sigma and a proven track record. He knew being brought in by the CEO increased his likelihood of success. Support from the top is essential for any Six Sigma initiative.

Better applied the define, measure, analyze, improve, control (DMAIC) approach to problem solving. He always kept Pareto’s law in mind: “Twenty percent of the business problems result in 80 percent of the time and cost.” Better sought the right balance with an intent to improve profitability without sacrificing timeliness, accuracy, and customer satisfaction by eliminating special cause variations.

He formed a core team consisting of the process owner, some knowledgeable people from line operations, and himself. He arranged online Six Sigma training sessions throughout all management levels and for all subject matter experts involved. 

Define

Better interviewed all stakeholders and subject matter experts for Acme’s retail software division, including the CEO, CFO, vice president of operations, and the vice president of application development who was the process owner. He received a walk-through of the daily routine and demonstrations of all support systems.

In his early observations, Better learned that a survey had never been conducted and that groups and individuals are measured on timeliness, but not accuracy, quality, or customer satisfaction. Tickets weren’t prioritized. Tickets were handled first-in-first-out, or for whoever screamed the loudest. Many tickets had incomplete information and were not actionable, yet still counted in backlog totals.

There is a small, company-owned development group in India that was barely utilized. The person responsible for operational reporting has his performance review and bonus approved by the person who runs the group.

With Six Sigma, it is standard procedure to implement quick-fixes along the way. This sustains interest and also prevents the problem from getting worse. So Better helped the company quickly address some low-hanging fruit:

• Incidents and requests for service were separated.

• Incidents were given priority over requests for service and rated within three categories: critical, urgent, and important.

• Tickets were worked based on their priority. (Note: A customer’s tendency to classify all tickets as critical can be coached).

• Revisions to the ticket document were made. An “incomplete information” field was added, as well as a description field for indicating what was needed. Tickets missing information were flagged and routed back to the customer.

• The customer was given the ability to autosearch a ticket to check its status.

 

To capture the voice of the customer (VOC), Better and the core team developed an 18-question customer survey concerning timeliness, accuracy, and customer satisfaction. The survey was scored on a scale of 1 to 5, with one being the best and five being the worst. One free-form text field was included to give the customers a chance to provide more detail. The survey was delivered via the web and all data was captured into a database. There were 115 companies. Responses came back from 67 companies, or 58 percent. There were 92 individual responses. Overall, this was an excellent response rate.

The following are key observations that Acme’s retail software customers had about their service:

• Courteous and knowledgeable, customer service support reps contribute strongly to the overall satisfaction.

• Initial response is important and some improvement has been noticed.

• Survey confirms areas to improve: response and resolution time, and communication concerning unresolved issues.

• Of the 92 who responded, 70 (76%) believed the customer representatives are courteous.

• 47 (51%) believed their requests for service were not handled in the expected time frame.

• 70 said they had to contact customer service three or more times before their issue was resolved.

• Seven (8%) claimed that most problems were never resolved.

 

For the biggest revenue-generating customers:

• 80 percent believed the reps were courteous.

• 64 percent believed the reps were knowledgeable about the software.

• 44 percent believed ticket status was communicated in a timely manner.

• 40 percent believed the customer reps are responsive.

• 48 percent felt there was a rapid initial response.

• 16 percent felt request for service were responded to in a timely manner.

• 40 percent were dissatisfied overall with customer service.

• 20 percent were satisfied with the quality of software delivery.

 

The majority of those who responded, 69 customers, (75%) took the time to fill out the free-form field. Here is a breakdown of their responses:

• 27 (39%) pointed to a need for improved communication and notification.

• 11 (40%) out of those 27 said “pick up the phone” as the No. 1 facilitator

• 23 (33%) pointed to timeliness and quality as their No. 1 concern.

• 11 (16%) were happy.

• 5 (7%) pointed to more training within Acme’s retail software staff as their best idea.

 

So, to summarize, here is what Acme’s retail software customers are saying:

The good

Better customer service this year. Keep up the good progress.

The staff tries hard to resolve issues. Pleased that the quality of service is being reviewed.

Improvement noticed in the response of the support team. Support team improved a lot since John Smith started managing the team.

The bad

Slow problem resolution. Much knowledge tied up with few people.

Response by support is usually OK until they turn an issue over to a developer, then communication often stops.

Quality assurance testing appears weak.

Occasionally, it takes months to get a fix from development. Customers are billed for each person that reads the question, even when they don’t know the answer.

Lack of testing and documentation for updates.

The ugly

Quality of enhancements is “horrible.” It takes three deliveries to get it right. Estimates are “horrible.” They must use weak developers.

Better if the product worked when presented.

Initial responding is within one day. Thereafter, I get no answer. All my questions unresolved.

Nothing is completely tested or documented. The only update we get is there will be a delay. After about 2 p.m. on the West Coast, support turns into an answering machine.

If it were not for the investment and time commitment to switch to new system, we would have left long ago.

 

Better used a business process analysis (BPA) tool (as illustrated in the workflow figures below) to show the “as is” development process. Each “swim lane” represents a role and each step in the value chain was depicted.

Click for larger image.

 

Click for larger image.

 

Measure and analyze

A process can’t be improved unless it can be measured. Metrics for timeliness, accuracy, and customer satisfaction were gathered to establish a baseline and raise problem flags. Six Sigma identifies the “Big Xs” that cause the problem “Y.”

For new application development problems, most of the needed data was not available. The full cycle averaged three hours with 25 minutes (14%) allocated for quality assurance. The percentage of time spent in testing was low. Benchmarking analysis indicated that quality assurance testing should account for about 50 percent of total time. Of the jobs that went through quality assurance, 31 percent failed. This high failure rate is an obvious indication there were problems upstream in development.

For application maintenance problems, the full cycle averaged eight hours. It took 15 minutes of analysis time to classify and route the incident. Seventy percent of tickets were incomplete and not actionable, yet still counted toward backlog totals. Ninety-five percent of tickets indicated user training was required. Thirty-five percent of tickets that reached programming didn’t require programming, indicating improper diagnosis and wasted review cycles. Fifty-four minutes (15%) spent in QA out of eight hours is too low. Of those tickets that reached QA, 19 percent failed. This percentage is too high, resulting in rework and more wasted cycles.

When looking into testing procedures we learned that unit testing is sometimes done and functional testing is never done. System and integration testing is sometimes done. Stress testing is sometimes done. Usability testing is never done prior to release to the customer, it is done after release. Security testing is never done.

When delving into quality assurance we learned that systems integration and stress testing are sometimes done. Usability and security testing are never done.

Improve

Given the organization’s budget and appetite for change, Better targeted waste elimination and cycle time reduction in his recommendation.

His recommendation was an operations improvement plan aligned with the voice of the customer:

1) Increase training for staff and customers. This aligns with the goal of reducing customer inquiries, a quicker response and improved accuracy when responding to inquiries, which leads to reduced rework and backlog. This reduces cost and improves customer satisfaction.

2) Improve documentation. More complete documentation will improve knowledge inside and outside the organization leading to reduced inquiries and shorter cycle time.

Note: Six Sigma classifies training and documentation costs as preventive costs because they prevent many problems from occurring in the first place.

3) Facilitate document availability. Implement document management and screen capture software to facilitate customer trouble-ticket creation.

4) Adopt function-point-control in all development groups. It will improve work-project estimating and improve work accuracy leading to reduced rework, backlog, costs, and improved customer satisfaction. Metrics include productivity (function points per work-hour), product quality (defects per function point), process quality (time spent beyond unit testing), and unit cost (cost per function point).

5) Quality assurance is the responsibility of all groups. Key drivers are training, management support, and automated tools to assist efforts. Until function-point-control can be made operational, staff should be added downstream in quality assurance. All groups should spend more time testing prior to software release.

6) Governance. Process: Customer-service staff members were directed to proactively pick up the telephone and speak with the customers. Ticket ownership: Customer-service personnel were assigned as “ticket owners” to serve as single points of contact for each trouble ticket and request for service. Individual and group accountability: Fields were added to the workflow software to enable tracking of each handoff through the process to enable capture of more granular metrics, including accuracy and timeliness. Ticket discipline: To enable accurate capture of ticket status changes, all staff members were asked to be more disciplined when filling out tickets. Adhere to priorities: Management was asked to make sure the staff members were permitted to allocate the time required for training. Operational reporting: Operational reporting was moved out of the operation to the finance department.

7) Offshore staff augmentation for development and quality assurance. Skills were built offshore to leverage follow-the-sun advantages and support 24 × 7 contractual obligations.

 

Control

Benchmarking, scorecard implementation, and linking of departmental performance to employees’ performance appraisals help ensure changes will stick.

Results

After one year with most of the changes in place, operations improved. There was a significant increase in customer satisfaction, employee morale, and profitability, as well as reduced cycle time and turnover. All of this was reflected in the balanced scorecard and a follow-up survey.

Final comment

The economy slowed. Rather than take advantage of the time afforded, staff cuts were made. The CEO did not have the appetite to fund improvement initiatives.

Preventive costs always raise operational costs in the short run but pay off big in the long run. The CEO’s short-sighted outlook resulted in increased cycle time and staff turnover and decreases in customer satisfaction and profitability. This further contributed to lost sales and customers leaving, despite the difficulty in doing so. The division went up for sale.

A young woman named Lynnette had first-hand experience on the receiving end when the LSS improvements were made. She was very favorably impressed. Lynnette also had a vision. She saw the possibilities if the division were managed the right way. Lynette wrote a business plan, obtained venture capital funding and bought the division, forming a small, independent company. She called it ACE Retail Software. Upon assuming the reigns, Lynnette immediately brought Mr. Better in to help lay the proper foundation based on lean, Six Sigma and the Theory of Constraints (TOC). The word is spreading. Customers that stayed are delighted! Others that left are coming back and bringing others with them. The sky is the limit for Lynnette now, as ACE Retail Software has a solid foundation. It is lean, mean, and built to scale!

Discuss

About The Author

Tim Leary’s picture

Tim Leary

Tim Leary, LSS Black Belt for Longevity Consulting’s  newly formed Business Transformation division, is a results-oriented change agent with breadth and depth of experience in organizational transformation, LEAN and Theory of Constraints (TOC). He has diverse experience with business problem identification and solution execution. Leary is a skilled project/program manager who engenders trust and support across all entities to accomplish business objectives. He has a record of successfully planning and executing quality programs. He has achieved an MBA, CSSBB, CISSP, PMP and ITIL V2 certifications.