There are many new trends shaping the way we manage, specifically the way we manage for quality. One of the most important is the increased access that businesses have to massive amounts of data, and the ever-growing importance of big data management.
What is Big Data?
Big data refers to extremely large sets of structured and unstructured data that can’t be managed with traditional data-processing software or a single computer.
There are countless ways organizations can harness the power of big data, from analyzing consumer behaviour and market trends to identifying growth opportunities and improving product quality. The data can also help companies make faster business decisions or determine the root cause of any product failures or defects.
While businesses have been gathering and storing large amounts of data for decades, the term “big data” has only emerged in recent times – coinciding with the relentless growth of the world wide web and advances in data-processing technology. Companies now using advanced analytics techniques such as machine learning, data mining and predictive analytics to gather new insights from previously untapped data.
The Three Vs of Big Data
The concept of big data gathered momentum in the early 2000s when distinguished data analyst Doug Laney first articulated the “three Vs”, which are widely recognized as the three defining pillars of big data.
Volume
The first of the three Vs relates to the sheer size of data that is now being processed by organizations, as well as the number of sources it can be gathered from. This includes the internet, social media, traditional media, databases, and more recently the internet of things (IoT). An estimated 2.5 quintillion bytes of data is created every day, meaning there will be 40 zettabytes of data created by 2020 – a 300 times increase on 2005.
Velocity
Data is now generated and delivered at incredible speed, meaning organizations need to deal with it in a timely manner. While data was previously delivered in “batches”, giving companies time to process it efficiently, the rise of new data sources such as mobile apps and social media has led to data being created continuously in real-time.
Variety
Traditional data is structured and numeric; it was typically collected from one place and delivered as a database file, such as csv, excel or access. However, data is now being presented in a number of non-traditional, unstructured formats including emails, text documents, videos, audio, social media posts and infographics, to name only a few.
A number of industry analysts go beyond the traditional three Vs, with some stretching the pillars to as many as ten. However, the most common alternative is to add an additional two Vs: veracity and value.
Veracity
This refers to the reliability or trustworthiness of data. It can be difficult to determine the veracity of certain data; consider the rise in ‘fake news’, for example, or the identity of somebody writing a tweet. Even with structured data like statistics, it is important to understand the source of the data, the methodology used to gather it, and the sample size.
Value
The power of big data ultimately lies in the value you draw from it. Despite the volume and variety of data available, and the speed at which it can be harnessed, organizations need to clearly outline the costs and benefits of accessing it. What impact could the data have on the business? What problem could it solve?
How Leaders Can Capitalize on Big Data
With so much data at their fingertips, it is crucial for leaders to put proper procedures in place to take advantage of this information for the benefit of their organization, employees and customers.
Many companies are either retraining their existing employees, bringing in consultants or looking for new hires with the necessary technological and analytical skills to effectively leverage big data. In numerous cases, organizations have added a new role to their C-suites, which typically takes the title of chief information, marketing, strategy, or risk officer, chief data officer, or chief analytics officer.
One of the most important tasks for leaders is establishing a new mindset that emphasizes the importance of big data. If you find yourself in this position, start by asking yourself, “Where do we want to improve?” Potential improvement areas will be what guide strategy, direction and culture in terms of how big data will be utilized.
Leaders need to recognize that dealing with big data is very different to dealing with “small” data. The risks are simply not the same; there is more at stake with big data because of the variation that accompanies larger data sets. It is physically impossible to examine each individual piece of data, so representative samples need to be analyzed.
With massive amounts of data, you will likely need to find new and more powerful analytical techniques. In addition, leaders may also need to organize data teams specifically for this type of data management. This may even mean locating experts from outside the organization, such as business analysts and data scientists.
The Power of Big Data Management
Effective big data management can have several benefits, including improved processes, new business insights, and recognition of sales and marketing opportunities. However, you can only reap these benefits if you are effectively utilizing big data management strategies and tools.
Two possible strategies are to either utilize existing data management platforms or to implement new software specifically for big data management. A combination of these strategies can also work just as well. Big data management can be the responsibility of a number of different work teams, including IT, research groups, or specialized business units.
The first step is to step back and ask yourself whether your company has a strategy in place for managing and leveraging big data. It is then up to leadership to determine the best strategy to pursue.
For tips on how to overcome the challenges associated with managing big data, take a look at our dedicated webinar.
Big Data in Action
It is clear that big data provides many opportunities for businesses, but it can also be leveraged to great effect outside the commercial arena – not least in the medical space.
i2b2, or Informatics for Integrating Biology and the Bedside, is a big data project sponsored by the NIH Roadmap National Centers for Biomedical Computing. Its mission is to allow clinicians to integrate medical records, genomics, and clinical research data; narrowing vast amounts of information down to mini-databases that can be used to guide medical decisions.
The program uses a two-step process which involves making copies of data sets and selecting specific cases in order to avoid the loss of data from the original electronic medical record system. Throughout the process, patient privacy is maintained and the data is compliant with The Health Insurance Portability and Accountability Act of 1996.
Doctors at Massachusetts General Hospital were able to use this software to determine that a widely prescribed antidepressant, Lexapro, had harmful effects on patients before any large scale studies were done by the FDA. A similar drug, Celexa, was issued a warning by the FDA because of its link to changes in electrical activity in the heart. However, despite its chemical similarities to Celexa, Lexapro was not issued the same warning because studies had not yet been conducted on patients taking this drug.
Dr. Shawn Murphy used i2b2 software to examine existing data on over 38,000 patients taking Lexapro who also had heart monitoring data recorded in electronic medical record systems. Based on this data, he was able to conclude that Lexapro actually had worse effects on patients than Celexa did. His findings were published in BMJ and then reviewed by the FDA.
Big data is inherently quite scary; it’s like trying to mentally picture the size of the universe. However, once you get past the scare factor, you can begin to consider ways to look at the data, and you may come across more unconventional ways to leverage its power.
For more information on how Juran can help you leverage big data to drive improvements in your organization, please get in touch with the team.
