Telling Production What to Do

Breakthroughs in voice-command technology are changing industrial production

Published: Monday, October 14, 2019 - 12:02

How would you like to go hands-free, maintain visual focus, and save time? These are just some of the benefits of using voice command to control machinery. Increasingly sophisticated natural language processing, based on artificial intelligence, also means that it is becoming possible to issue complex commands by simply telling a machine what you want it to do. This removes the need to memorize specific command words or learn complex menu systems and sequences of instructions.

During the past few years, voice-assisted technology has become very popular in the consumer market. Products such as Apple’s Siri, Amazon’s Alexa, the Google Assistant, and Microsoft’s Cortana have become household names—quite literally, as people use them to turn on the lights and control the heating. This technology is also appearing in the industrial market, where things can be considerably more challenging.

When compared to a keyboard or touch screen, voice command has some definite advantages in an industrial setting. First, it doesn’t require the operator’s hands or visual focus. This means the operator can remain engaged in another task while issuing commands to a machine. Outside of industry, these advantages are seen by consumers in common tasks such as driving or cooking. For example, you might ask your digital assistant to change the music or turn on the oven while chopping vegetables. In a car this is even more important, allowing drivers to control their phone, music, navigation, and heating without taking their hands off the wheel or eyes off the road. Later in this article, we will look at how the ability to maintain visual focus and keep your hands on the task can be beneficial in industrial settings.

Voice command is also really fast. Most people can type at up to 60 words per minute but can comfortably speak at twice that speed. Early voice command systems simply replaced the keyboard with speech recognition. This meant that specific command words had to be spoken in the correct sequence. Programming a controller to accept these inputs is relatively straightforward and has the advantage that the speech-recognition engine is only listening for a small number of command words, greatly improving recognition accuracy. However, this approach doesn’t do anything to reduce the training required to operate complex machinery. In fact, without visual feedback, it can be much more difficult to remember the sequence of complex processes. This means that the controller may need to provide spoken prompts, saying things like, “OK, now please choose from one of the following options....” As we’ve all experienced with automated telephone answering systems, this approach ends up actually being a lot slower than choosing a sequence of instructions on a screen.

The current generation of speech command systems is natural language processing (NLP) together with speech recognition. This enables a computer to understand the meaning of sentences rather than simply recognizing standard commands. In theory, complex instructions can be given as a natural conversation between the operator and the controller, saving time and greatly reducing the need for training. It should be possible for operators to simply tell the machine what they want it to do. The machine might need to ask for some clarifications and final confirmation before carrying out irreversible operations, but it should all feel natural and efficient. However, at the moment, NLP isn’t always completely reliable, and the results can be a bit mixed.

Voice command for measurement

Large-scale measurement is a great example of how voice commands can be useful because they keep your hands and vision free for other tasks. Often, this can work really well with just a few simple voice commands, so it is a mature technology that doesn’t rely on NLP. In aerospace manufacturing, laser trackers are the standard tool used to measure jigs and airframes. They are also widely used in ship building, and producing wind turbines and other large, high-value structures. A laser tracker uses a laser to track a reflective target. The operator uses the reflective target as a probe, carrying it around and touching it onto the features to be measured. In the past, a second operator was required to operate the tracker unit. Voice command makes it easy for a single operator to perform these measurements, moving the reflector around while issuing commands such as “measure” and “delete last measurement.”

Voice command for machine tools

Mazak was one of the first machine tool manufacturers to use voice with its Voice Advisor system in 2007. However, it only gave feedback through text-to-speech using a synthesized voice to give notifications instead of displaying them on the screen. It didn’t originally accept voice commands.

More recently, ITSpeeX created ATHENA, a voice-operated assistant for machine tools that has already been adopted by DMG Mori, Makino, and OKK. ATHENA provides a universal interface that will work with any controller. It doesn’t replace the controller. It sits between the controller and the operator, effectively replacing the keyboard and menu system. Incredibly, it allows natural commands to be issued in a common way for any controller. This allows operators to easily move between different machines without learning their way around the different controllers. ATHENA allows the operator to control the machine—e.g., warming up, datuming, and running programs. It can also give reports on machine status, perform calculations such as process capability, and answer questions such as, “How do I change the coolant?”

An example of how simple it can be to perform an operation that might be difficult to find in a conventional menu system is a tool change. Using ATHENA, the interaction might sound like this:
Operator: “ATHENA. Tool change, tool five.”
ATHENA: “Are you sure you want to change to tool five?”
Operator: “ATHENA. Yes.”
ATHENA: “Changing tool five.”

“There are several hundred functions that can be carried out using simple spoken commands,” says Mark Waymouth, ATHENA project manager at Makino. “This dramatically simplifies the operation of the machine. The easiest human-machine interface we could think of is speech.... We’re looking at ATHENA to be able to help set up, run, utilize the tooling, maintain the machine, and learn the machine.... One of the things ATHENA does is help with coaching. What’s one of the easiest ways initially to coach? Providing access to manuals.”

When ATHENA is asked a question that requires a more complex answer than it can give in a simple spoken statement, the instructions for X are displayed, and ATHENA tells you, “X instructions displayed.”

An industrial digital assistant

ICONICS Voice Machine Interface (VMI) is a more general-purpose industrial digital assistant. It can perform a range of functions in manufacturing, energy management, industrial automation, or smart buildings—for example, monitoring the status of systems and processes, controlling equipment and devices, and analyzing key performance indicators (KPIs). VMI uses the speech-recognition capabilities of consumer digital assistants such as Microsoft Cortana, Amazon Alexa, and Google Assistant. While walking around a factory, a production manager could ask a question such as, “Hey, Cortana. What is the OEE for Line 1, and how does it compare to last shift?” The VMI would provide an answer such as, “The OEE for Line 1 is 78.3 percent, which is up 2 percent from yesterday.”

The future is natural conversation

The ability to interact naturally with machines and data using speech is revolutionizing how humans work with machines. In its most basic form, this technology allows hands-free and eyes-forward interaction so that operators can command machines and obtain data while engaged in another task. The latest generation of NLP-based systems means that it is no longer necessary to learn complex menu systems. You just ask for what you want. This will have major implications in terms of reducing training and enabling workers to more flexibly move between different jobs.

Speech command will also feature heavily in collaborative robotics, where humans and robots work side by side in the same space.

First published Sept. 23, 2019, on the engineering.com blog.

About The Author

Jody Muelaner

Jody Muelaner, is a mechanical engineer with expertise in metrology and advanced manufacturing. Muelaner’s website provides information on topics ranging from the basics of metrology and measurement systems analysis to specific guides such as how to perform a gauge R&R study in Excel.