Cruising Toward Self-Driving Cars

Standards, testing help autonomous vehicles drive safely

Published: Thursday, February 9, 2023 - 13:00

(NIST: Gaithersburg, MD) -- Many of us are enthusiastic about the prospect of self-driving cars or automated vehicles. They promise to free us from the stress of driving. Automated cars may also reduce accidents caused by distraction or poor judgment.

Cars now come with more automated features than ever: headlight activation, emergency braking, detection and avoidance, and more. Many vehicles now include artificial intelligence to help vehicles drive safely.

So why aren’t fleets of self-driving cars already for sale and out on the road? There’s a lot we don’t really know about how to:
• Automate driving
• Measure driving performance
• Make sure automated vehicles drive safely

The human aspect of driving is challenging to automate

Experienced drivers make many decisions instinctively, such as stopping quickly when they spot a deer in the road. We still don’t know how to describe that instinct to a machine, even with its artificial intelligence and sensors. Modeling instinct, or teaching a car to drive like a human, is an even bigger engineering and scientific challenge.

Driving is not just an individual skill; it’s a collective effort. Each driver communicates with other drivers, pedestrians, and cyclists to get from point A to point B safely. Additionally, drivers interact with a huge infrastructure that is focused on safe and effective vehicle operations, such as traffic lights, signage, and alerts to changing traffic patterns.

All these aspects of human driving—instinctive decisions, driver collaboration, and interacting with road infrastructure—must be replicated for automated driving.

Many experts believe humans still have an edge over today’s technologies. But technology is rapidly improving.

Industry, government, and other partners are working toward replicating these aspects of human driving for automated vehicles (AVs) and building a supporting infrastructure. The effort will require human-factors engineering (a field that considers humans’ abilities and limitations), further technical development, and local and regional planning. The goal is to deploy automated vehicles that operate safely and effectively.

Automated vehicles require thorough testing to ensure safety

Compared to human-driven vehicles, driverless vehicles can be tested more thoroughly and less expensively before they hit the road because the driver is not a person; it’s software. Their simulations allow more testing in a shorter time, compared to test track or public road testing.

These simulations include models of other vehicles and all critical elements of the driving environment. Such simulations can model scenarios that automated vehicles may encounter, including vehicle failures or cyberattacks. These simulations can increase confidence in AVs.

Manufacturers of traditional vehicles test parts and systems on cars—like steering and brakes. Since these parts are proprietary, they don’t typically share the results of comprehensive testing. Regulators and other organizations responsible for safety can’t confirm the results or even the measurement and testing methods used.

NIST plays a key role in the drive toward autonomous vehicles

NIST develops and standardizes measurement methods that will be vital in the development of autonomous vehicles by doing what it does best: building a consensus among interested parties on the measurements needed to evaluate automated vehicle safety, and developing reliable measurement methods.

NIST scientists will use driverless cars like this one later in the year to develop measurement methods for driverless cars. Credit: Dataspeed Inc.

In a series of workshops, NIST partnered with automated vehicle stakeholders, including the U.S. Department of Transportation, the National Highway Traffic Safety Administration, industry representatives, and other research institutions.

NIST and its partners began by examining industry’s concept of an AV’s “operational design domain (ODD),” a vocabulary that includes names for the various conditions that the vehicle can operate in, such as operating in precipitation. The ODDs distinguish type of precipitation (such as snow and rainfall) and even types of rainfall (heavy or light). Think of it as the vehicle’s resume, describing its capabilities. The ODDs indicate what types of testing should be included for that vehicle.

NIST and its partners proceeded to develop a concept that complements industry’s operational design domain, called the operating envelope specification (OES). It includes all the factors in the vehicle’s ODD and more, along with criteria for assessing the car’s performance.

The OES provides a framework for measuring the automated vehicle’s operating environment. Critical vehicle behaviors, such as lane changing and navigating intersections, are specified as behavioral competencies. AV developers, regulators, and researchers can then use this framework to create testing scenarios. Think of the OES as the job description and associated metrics that all autonomous vehicles should meet.

NIST worked with leading vehicle system developers to create a co-simulation platform for studying automated vehicle safety. The platform is called co-simulation because it brings together best-in-class simulations for braking, engines, sensors, and much more. This platform evaluates multiple safety factors and vehicle models at the same time. Testers swap out vehicle models, level of automation, driving scenarios, and evaluation methods during co-simulation.

Using the platform, stakeholders can address such questions as:

• Will automated braking work in time to avoid a collision?
• Will automated cars obey speed limits and other traffic laws?
• Can an automated vehicle safely pass another vehicle on a busy highway?

The OES will help inform the tester and the vehicle. But what about testing the automated driver? This is the most significant challenge in automated vehicles. We know how to test human drivers and give them a driver’s license. We don’t yet know how to test a software driver. While the OES supports testing of vehicles, in the future it may also help us test automated “drivers.”

Additionally, NIST held a workshop last year, asking AV community stakeholders how NIST can support automated vehicle development. NIST identified the need to study:
• Interactions between the systems on the car
• Sensing and perception, such as recognizing and responding to objects in or around the road
• Cybersecurity research related to adversarial machine learning
• Risk associated with AI components
• Automated vehicle communications

Once facilities, equipment, and expertise are in place, NIST will work with automotive and government experts to identify critical automated vehicle behaviors and how to best measure these behaviors. The goal will be to set the standards for AV testing nationwide so these vehicles can operate safely.

NIST is building consensus on measurements needed to test automated vehicle safety and developing reliable measurement methods for testing driverless cars. Credit: Gorodenkoff/Shutterstock

What will a driverless future look like?

We are exploring simulation tools to help test the safety of automated vehicles. Government, manufacturing and technology companies, researchers, and other stakeholders can contribute to help NIST explore trustworthiness metrics, including safety metrics that will be vital to automated vehicle success.

As a driverless future nears, replacing human drivers with automated systems still raises critical questions. For example, perception by humans or machines will never be perfect, so investment in infrastructure may take up some of the slack. The NIST team will hold a workshop with infrastructure stakeholders this year to gather perspectives on how to measure the impact of these investments.

There are important questions to address regarding automated vehicles and their software drivers. Are they safe, secure, and trustworthy? Should safety be at least as good as the best human drivers? How can we compare the two?

Answering such questions will be a collaborative effort among experts in science, transportation, and human-factors engineering.

Automated vehicles must be safe, with their risks understood and measured. Researchers, developers, and government agencies must share their safety assessments with each other. Just as drivers collaborate to keep everyone safe on the road, many organizations will be involved in keeping the autonomous vehicle future safe for everyone.

Drivers know how to drive—but it’s difficult to describe or teach to others. Building a robotic or a self-driving car that does what we humans do is a challenge.

While fully automated vehicles may still be far down the road, when they do arrive you can be sure NIST will be under the hood.

This article was first published on Jan. 18, 2023, by NIST.

About The Author

Edward Griffor

Edward Driffor is the associate director for cyber-physical systems (CPS) at NIST. As well as being a well-respected scientist in the auto industry with expertise in self-driving cars, he is an adjunct professor of Artificial Intelligence at the Center for Molecular Medicine and Genetics at Wayne State University and the University of Grenoble Alpes. He has authored numerous books and is an expert on topics within the fields of mathematics, physics, biosciences, computing, and engineering.