Top of Page

My Big Data Resolution for 2015

NewVantage Partners

Figure 1. NewVantage Partners study of Fortune 1000 technology executives.

I generally like precision, and I strive for it in my writing and thinking. It certainly helps in communicating ideas., However, I have been pretty relaxed in how I use the term, Big Data. I've used the term to refer to the size of the data set (e.g., "We have Big Data."). I have even used it to describe a method (e.g., "We are using a Big Data approach."). Still other times, I have used it as a general label (e.g., "Big Data is the future.").

In a survey of Fortune 1000 technology executives, 30% of the executives found the term, Big Data, confusing and misleading and 53% said it was overstated (see Figure 1). With my inconsistent use of the term, perhaps I am merely contributing to their dislike of the term.

So, I am making a New Year's resolution for 2015. I will be more precise when I use the term, Big Data.

bigdataphenomenonSo, how will I use the term, Big Data? A good place to start is to see how others use it and define it. Berekely solicited over 40 data experts for their definition of Big Data. These definitions could be reduced into six broad areas. These six areas represent the problems/challenges we face in the realm of Big Data. When you tackle a data-intensive project, you will likely need to answer questions in each of these six areas:

  1. Processing of 3Vs: The first area is about the nature of the data including the size (i.e., volume), speed (i.e., velocity) and complexity (i.e., variety) of the data and the technology to process it all.
  2. Analytics: Analytics is about the types of analyses we perform on the data. There are two general approaches when analyzing data: the exploratory approach (machine learning) and the hypothesis-driven approach (human judgement, critical). I look at these two approaches as complementary to each other. While the hypothesis-driven approach requires scientists to set up and test pre-determined hypotheses, machine learning can augment that process by quickly analyze millions relationships that might lead to potential areas of interest that can then be reviewed, and perhaps pursued, by scientists with more formal testing. For example IBM Watson helped researchers at Baylor College ingest and review 70,000 research papers on a cancer protein. It identified six new proteins to target for cancer research.
  3. Insights: The analytics leads to insights. When we consider the insights we gain from our analytics, we need to ensure the results are reliable, valid and useful. We often times turn to data scientists to help us make sense of the analytics and to verify its veracity.
  4. Data Integration: Because analytics is all about finding relationships among variables in our data sets, businesses are looking to integrate their data silos to uncover interesting relationships across different metrics. The value of integrated data sets is greater than the sum of its parts.
  5. Communication: We need to be able to share our results to interested parties. Toward this end, technology vendors are creating visualization tools to improve how we communicate our results.
  6. Security/Privacy/Ethics: Finally, the era of the quantification of everything has resulted in issues surrounding the misuse of data as they relate to security breaches, and privacy and ethics concerns. In fact, a recent study of Chief Information Officers found that security is the number one challenge of Big Data projects (51% indicated it as a top challenge). Some major brands that have been the target of security breaches include Target, Home Depot, Neiman Marcus and JP Morgan Chase. Other examples include the US National Security Agency’s monitoring of individuals worldwide as well as Facebook's experiments on attitudes using unsuspecting users.

"Big Data" is a Phenomenon

As these definitions of Big Data show, Big Data is not one thing; it is about a lot of different things. Each of these things is not new or unique to Big Data; we have had to address each of them decades ago. These things are simply more relevant in today's overly-quantified world. So, in 2015 and beyond, when I'm asked to define Big Data, here is how I'm going to answer:

"Big Data is a phenomenon that is the result of the quantification of everything."

, , , ,

7 Responses to My Big Data Resolution for 2015

  1. Ulf Mattsson December 27, 2014 at 8:43 am #

    I agree that "everything has resulted in issues surrounding the misuse of data as they relate to security breaches, and privacy and ethics concerns" and "a recent study of Chief Information Officers found that security is the number one challenge of Big Data projects."

    I recently read the Gartner Report "Big Data Needs a Data-Centric Security Focus" concluding "In order to avoid security chaos, Chief Information Security Officers (CISOs) need to approach big data through a data-centric approach.

    The report suggests that new data-centric audit and protection solutions and management approaches are required.

    I noted that companies are starting to follow these guidelines. For example, Hortonworks recently released these types of features, including data tokenization, advanced HDFS Encryption, key management and auditing.
    I think it is time to re-think our security approach and be more data-centric.

    Ulf Mattsson, CTO Protegrity

Trackbacks/Pingbacks

  1. 23 Reasons to Get Excited about Data: Free e-book from IBM | @bobehayes - July 12, 2016

    […] live in a Big Data world where everything is being quantified. In fact, data already touches many aspects of your lives. IBM just released a free e-book […]

  2. Four Ways Big Data can Improve Customer Surveys | @bobehayes - March 8, 2016

    […] analytics that generate the customer insights to drive the business forward. In this new Big Data world, however, it's clear that businesses are now able to use different sources of data (e.g., buying […]

  3. Why Data Science Needs Subject Matter Expertise: Data Have Meaning | ContentMarketing - February 1, 2016

    […] our Big Data world, we are awash in data. We have a lot of it. It comes at us quickly. And it comes in different […]

  4. Hexanika | Making Sense of Our Big Data World [video] - Hexanika - August 25, 2015

    […] world of Big Data is defined by the quantification of everything. In order to make sense of all the data around us, we need to understand statistics and statistical […]

  5. Making Sense of Our Big Data World: Statistics for the 99% | @bobehayes - August 18, 2015

    […] world of Big Data is defined by the quantification of everything. In order to make sense of all the data around us, we need to understand statistics and statistical […]

  6. 65 new external resources and articles about data science, big data – January 6 | Doclens - January 7, 2015

    […] My Big Data Resolution for 2015 […]

Leave a Reply

 

bob@businessoverbroadway.com | 206.372.5990

UA-23043697-1