Several Big Data vendors have provided their predictions about what we might expect from the field of Big Data in 2014. These predictions come from articles from the following vendors (links provide each vendor’s predictions): IBM, SGI, Think Big Analytics, Xplenty, Pentaho, Alpine Data Labs, MapR Technologies, DataDirect Networks and Concurrent. For each article, I used any statements that were in the form of a prediction.
First, I generated a word cloud using all the predictions (see Figure 1.). Second, I conducted a content analysis by grouping similar predictions into broader categories. Generally speaking, because several data points paint a much more reliable picture than any individual data point, I consolidated all vendors’ predictions to get a more robust picture of the Big Data landscape this upcoming year. While there is considerable overlap among vendors’ predictions, there were some differences across vendors, perhaps the result of each vendor’s unique strength. The following areas represent the big topics for Big Data in 2014:
1. Analytics
Click image to enlarge.
Data can be overwhelming to users and most enterprises are expected to keep drowning in the ever-expanding, quickly moving data they have at their disposal. Consequently, Big Data vendors see a continued focus on analytics as a way to get insight about the data you have. Whether it be machine learning, clickstream analytics or visualization tools, expect to see a push on all things analytics in 2014. Besides, without analytics, data (of any size or shape) are meaningless.
2. Hadoop / Open Source
Click image to enlarge.
Hadoop will gain enterprise credibility as more companies see an increase in production deployment of Hadoop across a variety of industries. Hadoop’s increased adoption is likely due to its ability to tackle ETL (export, transfer, load) processes and its relatively low cost as a central enterprise hub for analytics and processing power for new applications. Vendors think that open source will dominate the Big Data platforms. The ecosystem will take proprietary technologies and make them open source, reflected in the move for platform technologies to become open source. The open source community will be responsible for rapid innovation (perhaps due to massive growth of applications by third party developers – see Applications below).
3. Continued Growth
Click image to enlarge.
According to some Big Data vendors, we should expect that Big Data will continue to see growth by way of funding and investments, resulting in an overcrowded Big Data space. After this growth, we might see a subsequent consolidation of the space through acquisitions. In fact, industry analysts say the Big Data market is expected to grow at a CAGR of 34.17 percent over the period 2013-2013. IDC says the global Big Data Market will grow to $16.1 billion in 2014, seeing growth in infrastructure, services and software.
4. People
Click image to enlarge.
A grab-bag of sorts, this category emphasizes the fact that, in addition to technology-related factors, we need to consider people-related variables if we are to realize the value from Big Data. Before companies pursue Big Data projects, they will need to gain a better understanding of their little data issues.
Simpler, user-friendly software will not be the sole answer to creating data scientists; users of Big Data tools need to get educated on data analytics and the scientific method. Additionally, companies will see the growth of Big Data roles like the Chief Data Officer as well as Big Data project managers. Clearly, Big Data is about more than just technology; you need to consider the people who use and are impacted by the new Big Data tools. In fact, researchers at IBM have identified nine areas (they call levers) necessary to create value from analytics, including such areas as Culture, Trust and Sponsorship.
5. Data Integration
Click image to enlarge.
Companies have access to data that come from a variety of sources, including self-monitoring machines, CRM systems, social media communities and more. Therefore, companies will be need to ensure the systems they use can be integrated with other systems. Researchers at MIT found that analytic innovators are more effective at aggregating/integrating data than analytically challenged companies. Big Data vendors that can integrate different data sources seamlessly will win.
6. Privacy/Security
Click image to enlarge.
With the recent security breaches at Target, Neiman Marcus Group and Michaels, the issue of security and privacy of customer data needs to be at the fore of every Big Data project. One Big Data vendor, however, feels the privacy/security issue (along with related areas of governance, risk management and compliance) has been sucked into a proverbial black hole in the Big Data universe.
7. Applications
Click image to enlarge.
With Big Data vendors adopting an open source approach in their platform (see category 2), we will see an increase in the creation of apps to extract value from the data. For example, AutoGrid, a new company, is opening up their platform on energy grid data to third-party developers and partners to make new applications. The more apps that are developed on this engine, the greater the chance AutoGrid lands on killer app(s) that helps grow its customer base. This approach makes sense to me. Look what happened when the government opened up their GPS platform in 2000 for commercial developers.
8. Data Veracity, Cloud and SQL
Click image to enlarge.
The last category contains three areas, each with a couple of vendor predictions. These themes reflect the following three areas: 1) Data veracity and the role of data errors as a source of optimization. Big Data vendors would do themselves a favor to ensure proper data governance to minimize data errors. Decreasing data errors will necessarily improve predictive power of any algorithm; 2) the growth of the cloud for processing and “as a service” platforms and 3) SQL on Hadoop, perhaps to improve the user-friendliness of their tools.
Summary
Figure 1. The Big Picture of Big Data for 2014. Click image to enlarge.
Big Data vendors’ predictions for 2014 paint a picture of a continually growing industry. The agreement across vendors gives some credibility about what we can expect in the Big Data world in 2014. In the world of Big Data, we should expect to see improvements / advances in a wide variety of areas: all things analytics, organizational best practices around analytics, applications built on open source platforms, data integration and data governance issues driven by privacy / security concerns.
What do you think will be the Big Data trends for 2014?
Beyond the Ultimate Question
Measuring Customer Satisfaction and Loyalty (3rd Ed.)
Great summary, and we’ll thought out article. However, I feel more could have been added to the “people” section. I have found during my career within this “field” one characteristic that I seek out is an overall curiosity in nature. Technical skills can be taught, curiosity, I’m not so sure.
Bob, very informative article. Cloud computing is driving a new wave of innovation in the area of big data. The open source solution from HPCC Systems provides a single platform that is easy to install, manage and code. Designed by data scientists, HPCC Systems is a data intensive supercomputer that has evolved for more than a decade, with enterprise customers who need to process large volumes of data in a 24/7 environment. Its Thor Data Refinery Cluster, which is responsible for ingesting vast amounts of data, transforming, linking and indexing that data, and its Roxie Data Delivery Cluster are now offered on the Amazon Web Services (AWS) platform. Taking advantage of HPCC Systems in the cloud provides a powerful combination designed to make Big Data Analytics computing easier for developers and can be launched and configured with the click of a button through their Instant Cloud solution. More at http://hpccsystems.com