10 Guideposts to Big Data Adoption

Randy Bean, CEO and Managing Partner, NewVantage Partners

Randy Bean, CEO and Managing Partner, NewVantage Partners

NewVantage Partners recently completed its 2016 Big Data Executive Survey, its fourth and final such survey, targeting senior Fortune 1000 business and technology decision-makers such as chief information officers, chief analytics officers, and line-of business executives, all looking to maximize their business opportunity with Big Data.

Big Data has reached a point of mainstream adoption within Fortune 1000 firms, and even the most cautious firms have adopted this strategy in some form

The survey is dominated by large financial service respondents, who continue to be among the heaviest and most mature users of data within the Fortune 1000. However, firms in the life sciences and health care sectors are beginning to develop the kind of robust data management capabilities, which financial services firms have employed for decades.

Survey respondents represented 44 Fortune 1000 and leading firms, broken out by: financial services 73.2 percent, life sciences 17.9 percent, and other sectors 8.9 percent.

It is clear that Big Data has reached a point of mainstream adoption within Fortune 1000 firms, and even the most cautious firms have adopted this strategy in some form.

The percentage of firms reporting that they have a Big Data initiative in production rose sharply from 48.2 percent in 2014 to 62.5 percent in 2015. The  percentage of firms reporting 1 or more Big Data initiatives in production has grown from 19.8  percent in 2013 to 32.1 percent 2015. The  percentage of firms reporting Big Data initiatives operationalized across the enterprise has grown sharply from 11.6 percent 2013 to 30.4 percent in 2015. Only 5.4 percent of firms report that they have no Big Data initiatives planned or underway.

Here are 10 guideposts of the Big Data phenomenon today, as it has changed how we look at data, becoming a competitively essential-part of the business world.

Investment in Big Data Initiatives in Projected to Grow Sharply from 2014 through 2017

The percentage of firms reporting an expected investment in Big Data of greater than $50MM is expected to grow nearly 5x from 5.4 percent to 26.8 percent. Firms investing greater than $100MM is projected to increase from a small 1.8 percent in 2014 to a significant 8.9percent by 2017. Firms investing $50MM-$100MM is projected to increase from 3.6 percent in 2014 to a very significant 17.9 percent by 2017.

While, 41 percent of firms are still investing less than $10MM in Big Data in 2014, this number is projected to drop by more than half to 19.6 percent by 2017. Big Data spending is on the rise.

Big Data is Now Seen as Being of Critical Important to an Overwhelming Majority of Firms

The percentage of firms that now view Big Data as very important or critical to success has grown sharply from a bare majority of 54.5 percent in 2014 to a critical mass of 69.9 percent in 2015. The  percentage of firms that see Big Data initiatives as being mission critical has risen from 23.2 percent in 2014 to 32.1 percent  Only 1.8percent of firms indicated that Big Data was not important to the firm.

Executive Sponsorship: Leadership for Big Data is Coming from the Top of the House

Firms attributing to Big Data are evidenced by sponsorship for Big Data initiatives, with 91.9 percent of organizations reporting that sponsorship starts at the top. Of the executive sponsors, 19.6 percent of firms indicated that sponsorship was the purview of the CIO, 16.1 percent indicated that Big Data was driven by the CEO, and 14.3 percent named the Chief Data Officer as the primary executive sponsor. The Chief Operating Officer and Chief Marketing Officer were each named by 10.7 percent of firms.

The Chief Data Officer is Here to Stay

The percentage of firms’, report that a CDO function has climbed steadily each year from 12percent in 2012 to majority of firms at 54percent.

With 14.3 percent of firms reporting that the CDO is the primary executive sponsor, it is clear that the CDO will play a critical role in the ongoing evolution and adoption of Big Data for most firms.

Keys to Success: Business and Technology Partnership is Critical to Successful Adoption

A clear plurality of 33.9 percent named business and technology partnership, up from 23.4 percent in 2013. Strong business leadership and sponsorship was cited by 23.2 percent of firms. Recognition that data is a shared corporate asset was cited by 10.7 percent of firms

Only 5.4percent of firms cited strong technology leadership, as the most critical factor and 0 percent cited selection and implementation of the right technologies. Successful business adoption is clearly seen as a human and organizational issue.

Investment Drivers 

Firms are expecting greater insights and the ability to act faster as a result of Big Data investments. When asked to name the most important factors driving investment in Big Data, 37 percent cited the need for greater insights into our business and customers, while 29.7 percent cited factor relating to speed–faster time to-answer, decide, and market with new products and services.

Volume, Variety, or Velocity  

Variety continues to trump volume and velocity for Fortune 1000 firms. Firms cite the need to integrate more data–from new sources as well as legacy source. This focus on integrating greater varieties of data is often referred to as the “long tail” of Big Data. While 40 percent of firms cite “variety” (more sources) as the primary driver of Big Data investment, only 14.5 percent name “volume” (more data) or “velocity” (faster data).  

In spite of the interest by firms in integrating more varieties and sources of data, only 3.6 percent of firms cite the need for unstructured data (e.g. documents, text), and only 1.8 percent cite social media data as a priority.

Data Agility  

Firms are rallying to new data management approaches, which enable greater agility. The availability of new data management approaches (e.g. Hadoop) is providing greater agility and nimbleness, with 30.9 percent of firms citing this as a top priority. Only 9.1 percent of firms view the reduction of platform costs for data storage and analysis as being their top Big Data priority.

Data Discovery  

Firms are embracing new environments that facilitate data discovery. Big Data has altered the traditional data management paradigm, which has long been dominated by an enterprise data warehouse approach that has often been perceived as lacking in flexibility when it comes to analytics and data discovery.   

The percentage of firms that now employ an analytical sandbox, Big Data Lab, or Big Data Center of Excellence to facilitate data discovery has grown from 51.9 percent in 2013 to 73.2 percent in 2015. 

Rise of the Data Lake   

Firms are looking to the Data Lake or Data Hub as an alternative approach or complement to the Enterprise Data Warehouse. The  percentage of firms indicating that they plan to invest in a Data Hub or Data Lake approach has grown sharply from 24.1 percent in 2014 to 41.1 percent in 2015.  Another 28.6 percent expect that Big Data will co-exist with the Enterprise Data Warehouse. Only 7.1 percent of firms expect Big Data to replace the data warehouse.   

These guideposts show the path how Big Data has taken towards mainstream adoption, with almost double the number of respondents starting its production since 2013, and planned investments expected to rise sharply. 

Read Also

The "Black Box Paradox" in Big Data Analytics and Data-Driven Modeling

Daniel Lingenfelter, Staff Engineer, Seagate Technology

A Look into the Disrupting Industry of Cyber Space

Jerrod Chong, VP of Solutions, Yubico

The Storm Behind the Cloud: Ushering In the Next Era of Innovation

Merijn te Booij, Chief Marketing Officer, Genesys

Cloud: Enhancing Truly Unified Communications for Businesses

Ken Bisnoff, SVP of Strategic Opportunities, TelePacific