Big Data, Meet Small Data

The real revolution isn’t big data, but the increased accessibility to small data

Published: Monday, June 2, 2014 - 13:24

I’ve been following the panting exuberance of big data apostles for the past few years, rolling my eyes at most of it. Sure, it can be interesting, but maybe my age is showing when I say, “So what?” to most of it.

What finally pushed me over the edge enough to comment on it was a tweet I saw from none other than our friends at SAP, saying (I’m paraphrasing) “You have lots of data; let SAP analyze it.” But of course. There’s gold in them thar data… at least to SAP.

Just because we have lots of data, does it mean they have to be analyzed? What is the question, what is the problem, what is the opportunity being addressed? As Einstein said, “Not everything that can be counted, counts.”

It has become easier and easier to create, capture, and store data. Just ask the NSA. Or Target. Sure, data inventory doesn’t take up physical space, but it does take time, equipment, and people to manage it. And perhaps it is inventory... excess, over-produced inventory. What does lean tell us about inventory?

Sure, there is potential value in linking disparate data sets to come up with interesting and perhaps profitable conclusions. Unfortunately, the most common outcome I’ve seen is an increasing number of KPIs, dashboard indicators, and the like tied to data. And once again, an increasing number usually creates a dilution of focus. Just ask those companies that have a 2-in. thick strategic plan.

A year ago Rufus Pollock penned an article for The Guardian that described how the real revolution isn’t big data, but the increased accessibility to small data.

“Just as we now find it ludicrous to talk of ‘big software’—as if size in itself were a measure of value—we should, and will one day, find it equally odd to talk of ‘big data,’” he says. “Size in itself doesn’t matter—what matters is having the data, of whatever size, that helps us solve a problem or address the question we have.”

Bingo. Just like the 5S of lean or micro meditation of mindfulness, data is just a tool. First identify the problem, question, or opportunity. Only then determine what data, and analysis, is necessary.

“Meanwhile we risk overlooking the much more important story here, the real revolution, which is the mass democratization of the means of access, storage, and processing of data,” continues Pollock. “This story isn’t about large organizations running parallel software on tens of thousand of servers, but about more people than ever being able to collaborate effectively around a distributed ecosystem of information, an ecosystem of small data.”

Smaller data, more interconnected and accessible.

Pollock adds, “This next decade belongs to distributed models, not centralized ones; to collaboration, not control; and to small data, not big data.”

What size are your data?

First published May 16, 2014, on Kevin Meyer’s blog.

About The Author

Kevin Meyer

Kevin Meyer has more than 25 years of executive leadership experience, primarily in the medical device industry, and has been active in lean manufacturing for more than 20 years serving as director and manager in operations and advanced engineering, and as CEO of a medical device manufacturing company. He consults and speaks at lean events; operates the online knowledgebase, Lean CEO, and the lean training portal, Lean Presentations; and is a partner in GembaAcademy.com, which provides lean training to more than 5,000 companies. Meyer is co-author of Evolving Excellence–Thoughts on Lean Enterprise Leadership (iUniverse Inc., 2007) and writes weekly on a blog of the same name.