Big Data and Small Data: A Collaborative Affair. Keeping up with the Bezoses

Share this blog

Ask Amazon CEO Jeff Bezos what he has for breakfast, and he would probably reply, “big data.” Yes, big data is the Wheaties of Champions, and businesses are investing in the best tools and expertise to harness it. But not every business behaves like Amazon, and there’s one important tool that’s being bypassed on the “cereal shelf,” small data.

But before we delve into the subject, let's first define big and small data, or at least try to since agreed-upon definitions for either do not exist.  Big data is a term that describes the large volume of data - both structured and unstructured - that inundates a business on a day-to-day basis. These extremely large data sets may be analyzed computationally to reveal patterns, trends, and associations, especially relating to human behavior and interactions. Small data, on the other hand, can be defined as small datasets that can be easily stored and are capable of impacting decisions in the present. Either Big data and small data blog image 300x 200sourced locally or mined from big data, small data "can be organized and packaged, often visually, to be accessible, understandable, and actionable for everyday tasks."1

In the Digital Age, big data receives all the attention. It's big, it's glamourous, and it has massive stage presence. Data analyzers have even assigned it a five-V acronym to categorize the way it behaves and differentiate it from small data: Volume, Velocity, Variety, Veracity and Value. Small data, well, its behavior has yet to be labeled. Many companies consider small data the runt of the litter, but small data can be essential to the more nuanced, strategic decisions your business will make to ensure its success. In this series, we will not only delve into the five V's of big data but also delve into its much older yet its much less talked about counterpart, small data. Our goal? Understanding the importance of both and how to parley that understanding into a strategic and successful data action plan.

But first, let's become acquainted with the five V's of big data's behavior.

No surprise, here. Volume refers to the vast amount of data available to you at any given second. Thanks to IoT and the number of connected devices, in the blink of an eye, we've skyrocketed from megabytes and gigabytes to terabytes and now up to peta/exa/zetta/yottabytes. (Before this blog is finished, we may have to create a new name for the 1,024 yottabytes.) Making sense of these big bytes requires advanced processing techniques and solutions.2

Velocity refers to the speed at which new data is both generated and distributed.2 Big data rarely even makes it to databases before it is analyzed and acted on. Data can also come in waves, depending on the circumstance, moving from a steady predicable wave stream, such as customer feedback, to an unpredictable mega tsunami, like a video gone viral that prompts a product recall.

Variety refers to the types of data available to us from structured data (i.e. date, time GPS location, database), semi-structured (i.e. clickstream), to unstructured data (i.e. text, image, voice, or video),2 and big data can encompass all three. In fact, about 90% of the world's data is now unstructured,3 and many businesses are now focused on turning unstructured into actionable data.

Veracity refers to the trustworthiness of big data, and it is quite possibly the trickiest behavioral aspect.2 Quantity does not necessarily mean quality, and inaccurate or messy data often prevents our ability to create value from it.

Value refers to our ability to change big data into a commodity that adds value to our business.2 Of all the tera and yottabytes flying at us from different sources at different speeds, only a third or so of it contains valuable information that is actionable.2 Luckily, resources exist to mine big data and find the valuable "ore."

If big data tells you the "who," "what," "where," "when," and "how"; small data tells you the "why."

As previously stated, there are no categories nor is there even an agreed-upon definition for small data. Most agree that this type of data is small enough to be "accessible, understandable, and actionable,"1 basically pulling us back into our comfort zones of mega and gigabytes. Unlike big data, small data can often be housed in one database. Although its size is smaller and its speed is slower than big data's, its role in driving your business decisions can be just as vital. As an old German proverb once said, "God is in the detail"; small data allows us to dissect structured or unstructured data to provide us with its hidden meaning.

So, how can you approach big and small data collaboratively, ensuring both data sets are communicating effectively with each other and with you? Ultimately, our recommendation is to acquire the tools to garner both data sets. And partner with a data-savvy firm, like Copley, to ensure your data provides you with the true customer story. Knowledge is power, so stay tuned.
The Copley Consulting Group | Big Data & Small Data: A Collaborative Affair
1. Banafa, A. (2016, July 25). Small Data vs. Big Data: Back to the Basics. OpenMind BBVA.
2. Marr, B. (2015, March 19). Why Only One of the 5 Vs of Big Data Really Matters. IBM Big Data &
Analytics Hub
3. Marr, B. (2019, October 16). What Is Unstructured Data, and Why Is It So Important to Businesses? An Easy Explanation for Anyone. Forbes.

Filed Under: Blog, FDA Validated