Featured Product
This Week in Quality Digest Live
Metrology Features
Shobhendu Prabhakar
Remote inspection shouldn’t be a stop-gap tool just for the pandemic
Ron Cowen
What happened when a physicist recalibrated his oven sans manufacturer’s directions
Kristopher Lee
ASM International is a key educational partner in advancing quality control for additive manufacturing
Matthew Martin
The established technology of coordinate measuring machines is being challenged by blue-light scanning

More Features

Metrology News
Scan was used to create 80% to-scale 3D-printed replica
Quickly captured complex geometries resulting in optimally designed roof racks and greatly accelerated completion time
Designed for scientists, engineers, technicians working in medical devices, automotive, aerospace, and other fields
New line of micrometers are easy to use, have enhanced features and performance, competitively priced
REVO 2 features 1-meter cubed capacity and automation options
Hexagon Manufacturing Intelligence launches technologies to transform inspection workflows for enhanced productivity
An efficient and cost-saving way of working with materials like plastic, glass and ceramic
Detects and reports on tool breakage, missing tools, overload, tool wear and fluid flow in real time

More News

Edward Herceg


Three Rs of Analog Position, Sensor-Based Mechanical Measurements

Understanding repeatability, resolution, and response

Published: Monday, July 29, 2019 - 12:02

Those of us old enough to remember the “good old days” recall that grade school focused on learning the three R’s: readin’, ’ritin’, and ’rithmetic. In the world of sensors, there are also three Rs: repeatability, resolution, and response. Despite how important these sensor parameters are, there is often confusion in the minds of users about exactly what they mean, and in what ways they tend to interact with each other. This article explains these three Rs for position sensors to dispel any confusion that exists.


Repeatability is a measure of the variation between outputs of a sensor-based measuring system for repeated trials of an identical mechanical input in a constant environment. Common practice is to use at least three repeated inputs, but five or more identical inputs are considered to be an even better sample for determining this parameter. Repeatability is usually evaluated by applying an averaging process to the variations in output values observed for the multiple trials. It is typically specified as a percentage of full-scale output or full-span output, but sometimes it is specified in absolute terms such as parts per million or fractions of the mechanical units applicable to the actual sensor-based measurement.

A constraint on repeatability measurement is that the trial inputs must be applied in the same way, usually from a lower value to higher value, to eliminate any effects from hysteresis. Hysteresis error is a measure of the difference in system output when the mechanical input is rising up to the desired input value from a lower value, compared to an identical input coming down from a higher input value to the desired value. For most contactless position sensors, hysteresis error is smaller than repeatability error.

An example demonstrating repeatability can be seen in figure 1, which shows a spring-loaded position sensor in a typical gauging stand being calibrated with a precision gauge block of 0.5000 in. dimension. The sensor delivers an output of 0 to 10 volts DC full scale for 0 to 1 in. of probe movement. The sensor’s tip is moved inward to allow the gauge block to be inserted between the tip and a flat base, and then released to contact the block. In five trials, system outputs are: 5.0012, 5.0016, 5.0013, 5.0010, and 5.0015 volts DC. The average value is slightly more 5.0013 volts, and the maximum variance is ±0.0003 volts, which is equal to ±30 ppm of full-span output, or 0.003 percent of full-span output.

Figure 1: Sensor repeatability testing apparatus

Resolution is a measure of the smallest change in the input to a sensor-based measuring system that will produce a measurable change in the electrical output from that system. Although this might seem like a fairly simple concept, it is affected by factors external to the sensor itself, the most significant of which is the signal-to-noise ratio of the system’s analog output. Electrical noise present on the system’s output can reduce the effective resolution of the system by masking any small changes in the sensor’s output. For example, if the sensor’s resolution specification is 0.25 mV, but the system output noise and ripple is 2 mVp-p, clearly sensor output changes smaller than 2 mV will not be discernable within that noise. Thus, the actual system resolution is only about 12 percent of what the sensor resolution specification offers.

Like repeatability, resolution is often specified as a percentage of full scale/full span output but may also be specified in absolute terms like fractions of the units of the actual sensor measurement, or in digitally augmented measurements, in bits, which is just a fractional measure expressed in powers of two, as found in computers. Thus 10-bit resolution is one part in 1,024 (210), 12-bit is one part in 4,096 (212), etc.

Response denotes a sensor-based measuring system’s performance under dynamic input conditions, that is, when the system’s mechanical input is changing rapidly. It is particularly important to recognize that response is a measuring system parameter, not merely a sensor parameter or specification.

In practice, there are several ways to characterize response, typically based on whether the system is a first-order or second-order system. Traditional analog systems have used Bode plots to show frequency response and phase lag for repetitive inputs. For step-function response, three times the system-time constant is a typical measure of dynamic performance. In digital sampling systems, the update rate for a specified number of bits is one of the preferred measures of response.

Regardless of the choice of how to specify response, the ultimate purpose is to understand how well the measuring system can respond to a changing input before the system’s output becomes inaccurate, unusable, or unstable.


From the foregoing definitions, it is easy to see that a system’s repeatability could easily be affected by its resolution. If the measuring system’s resolution is inadequate, it would likely be a significant limiting factor to excellent measurement system repeatability. In practice, sensor repeatability may be excellent, but measuring system repeatability cannot be any better than that permitted by the system’s resolution.

While the interaction of repeatability with resolution in a measuring system is pretty easily understood, the interactions of resolution and response are not so straightforward. When the system’s mechanical input is changing rapidly, the effects of resolution on system output are usually masked by the larger effects of decreased system output due to limitations imposed by the system’s dynamic response. But if the mechanical input to a position measuring system changes slowly or intermittently, especially in a jerky way, then the effects of stiction (static friction) come into play.

Typically, the effects of stiction in position measuring systems can be nonlinear and are often not very repeatable, so determining or characterizing system resolution can be much more complicated, if even possible. And because the system resolution interacts with system repeatability, as was noted above, measurement errors can increase substantially, particularly if the system is providing position feedback for closed-loop control. Of course, any effects caused by stiction will also appear as nonlinearity in the sensor’s output. But because stiction effects are not very repeatable, digital linearization techniques to offset the nonlinear effects will not be practical.

For this reason, efforts to reduce stiction are usually necessary to minimize any measurement errors caused by stiction in very slow-moving or intermittent-motion positioning systems. These efforts can involve applying techniques such as “dither,” a low-amplitude signal of high frequency that is input into the system to supplant stiction with much-reduced dynamic friction; or by decreasing friction on the moving surfaces of the sensor by improving their surface finishes and by coating them with a lubricant.


One conclusion that can be drawn from this article is that specifications for a position sensor are not standalone elements but must take into account system level interactions, such as electrical noise and mechanical constraints like static and dynamic friction. Another is that many sensor specifications are developed for system operation in a constant environment, which may be unrealistic in a large number of applications where temperature, vibration, humidity, and similar environmental factors are highly variable. This exposes the effect of system-level sources of measurement errors that can detract from a sensor’s excellent specifications. The key is to always remember that a sensor is only part of a system, and that repeatability, resolution, and response apply not merely to the sensor but also to the measuring system at large.


About The Author

Edward Herceg’s picture

Edward Herceg

Edward Herceg is the chief technology officer and chief applications engineer at Alliance Sensors Group, manufacturer of position sensors with engineering services in solving measurement challenges. Herceg is one of the most accomplished electromechanical engineers and technical marketers in the sensor industry. His experience also includes discipllines in electrical, chemical, mechanical, structural, and materials sciences. Herceg authored the Schaevitz Handbook of Measurement and Control (Schaevitz Engineering, 1972), the first book written on LVDT position sensors.