Featured Product
This Week in Quality Digest Live
Innovation Features
Melissa Burant
A way to visualize and help prioritize risks, actions
Jennifer Chu
High-speed experiments help identify lightweight, protective ‘metamaterials’
Michael King
Augmenting and empowering life-science professionals
Rich Nobliski
Helping narrow the manufacturing skills gap with 3D tech
NIST
NIST scientists perfect miniaturized technique

More Features

Innovation News
Fluid Board, a compact and modular color dosing and changing system
It’s the backbone of precision measurement. What’s best for you?
Low voltage useful to petrochemical processing, pharmaceutical manufacture, and other processes
Latest in video probe product line features upgraded CPU
New tool presents precise, holistic picture of devices, materials
Enables better imaging in small spaces
Helping mines transform measurement of blast movement
Handles materials as thick as 0.5 in., including steel
For companies using TLS 1.3 while performing required audits on incoming internet traffic

More News

Rupa Mahanti

Innovation

Data Must Be Kept Fresh!

Understanding data decay

Published: Thursday, March 16, 2023 - 11:03

Data decay, or the aging of data, is data degradation over time—which results in bad data quality (Mahanti 2019).

There are some data, such as date of birth and place of birth, which are evergreen and not subject to decay; that is, if you have captured the data values correctly in the first place and the data are untouched, the data will not change. Time has no effect on such data. However, other data are subject to aging and decay, even if left untouched. The time factor and what triggers the decay are the variables. 


Photo by George Firican

The quote in the image above is from Cesar Augusto Lima in response to a LinkedIn post by George Firican in relation to natural decay of both lattes and data. Firican, the founder of LightsOnData and a recognized thought leader, made an interesting comment in his post on Oct. 31, 2022, on LinkedIn with a cup of latte. His statement was: “The design of the latte starts to be ruined after a while, even if left untouched. Same with certain data.”  

Data such as stock market data are extremely volatile and change every few seconds. 

Other data, like passport expiration dates, are comparatively less volatile, with an expiration ranging from five to 10 years. 

Still, in the case of other data such as contact data, decay is event driven, such as: change of address triggered by expiration of rent lease dates or due to movement to a new location for job purposes; change in telephone numbers due to transfer to a new country or due to operator changes; and so on. About 25 to 30 percent of an organization’s contact data can go bad each year under normal circumstances (Neubarth 2013). Hence, if an organization’s customer database has 12 million customer contact records, then approximately 3 million to 4 million customer contact records will be obsolete annually, resulting in significant dollar costs in terms of postage as well as missed opportunities (Mahanti 2019). 

How do you keep contact data relatively fresh? 

Contact data need regular maintenance. To ensure that data are up to date, it’s important to set guidelines for how often each field should be updated. For example, according to Reserve Bank of India guidelines, Indian banks require Know Your Customer (KYC) updates once every three years, and more frequently if a transaction has not occurred. Know Your Customer (KYC) standards are designed to protect financial institutions against fraud, corruption, money laundering, and counterterrorist financing, but because they establish customer identity, the contact details are also updated. 

Concluding thoughts 

Guidelines and processes for updating data should be defined for critical data elements in an organization. This will ensure that data are up to date, of high quality, and fit for usage. “Latte must be consumed fresh; data must be kept fresh!”

This article draws significantly from the research presented in the book Data Quality: Dimensions, Measurement, Strategy, Management, and Governance (ASQ Quality Press, 2019). Future research will focus on how to measure data quality and data quality strategy. 

Discuss

About The Author

Rupa Mahanti’s picture

Rupa Mahanti

Rupa Mahanti is a business and information management consultant and has extensive and diversified consulting experience in different solution environments, industry sectors, and geographies (United States, United Kingdom, India, and Australia). With work experience that spans industry, academics, and research, Mahanti has guided a doctoral dissertation, published a large number of research articles, and is the author of the book Data Quality: Dimensions, Measurement, Strategy, Management and Governance (ASQ Quality Press, 2019). She is a reviewer for several international journals and publisher of “The Data Pub” newsletter on Substack.