The percentage of firms identifying themselves as being data-driven has declined in each of the past 3 years — from 37.1% in 2017 to 32.4% in 2018 to 31.0% this year. These sobering results and declines come in spite of increasing investment in big data and AI initiatives. Whatever the reasons for the failure to achieve transformational results from data initiatives, the amount of data continues to rise in business and society. Analytical decisions and actions continue to be generally superior to those based on intuition and experience. The companies in the survey are investing heavily in big data and analytics. In short, the need for data-driven organizations and cultures isn’t going away.
Becoming “data-driven” has been a commonly professed objective for many firms over the past decade or so. Whether their larger goal is to achieve digital transformation, “compete on analytics,” or become “AI-first,” embracing and successfully managing data in all its forms is an essential prerequisite. Consistent with these goals, companies have attempted to treat data as an important asset, evolve their cultures in a more data-oriented direction, and adjust their strategies to emphasize data and analytics.
We knew that progress toward these data-oriented goals was painfully slow, but the situation now appears worse. Leading corporations seem to be failing in their efforts to become data-driven. This is a central and alarming finding of NewVantage Partners’ 2019 Big Data and AI Executive Survey, published earlier this month. The survey participants comprised 64 c-level technology and business executives representing very large corporations such as American Express, Ford Motor, General Electric, General Motors, and Johnson & Johnson.
Here are some of the alarming results from the survey:
- 72% of survey participants report that they have yet to forge a data culture
- 69% report that they have not created a data-driven organization
- 53% state that they are not yet treating data as a business asset
- 52% admit that they are not competing on data and analytics.
Further, the percentage of firms identifying themselves as being data-driven has declined in each of the past 3 years – from 37.1% in 2017 to 32.4% in 2018 to 31.0% this year.
These sobering results and declines come in spite of increasing investment in big data and AI initiatives. 92% of survey respondents reported that the pace of their big data and AI investments is accelerating; 88% report a greater urgency to invest in big data and AI; and 75% cite a fear of disruption as a motivating factor for big data/AI investment. In addition, 55% of companies reported that their investments in big data and AI now exceed $50MM, up from 40% just last year. Further, companies are building organizations to manage their big data/AI initiatives, with a rise in the appointment of Chief Data Officers from 12% in 2012 to 68% of organizations having created and staffed this role in the past 7 years.
Yet critical obstacles still must be overcome before companies begin to see meaningful benefits from their big data and AI investments. An eye-opening 77% of executives report that business adoption of Big Data/AI initiatives is a major challenge, up from 65% last year. Executives who responded to the survey say that the challenges to successful business adoption do not appear to stem from technology obstacles; only 7.5% of these executives cite technology as the challenge. Rather, 93% of respondents identify people and process issues as the obstacle. Clearly, the difficulty of cultural change has been dramatically underestimated in these leading companies — 40.3% identify lack of organization alignment and 24% cite cultural resistance as the leading factors contributing to this lack of business adoption.
There are a variety of other possible explanations for the failure of large firms to achieve the goal of data-driven organization. Perhaps the pursuit of short-term financial goals pushes longer-term objectives like data-based cultures to the back burner. It may also be that the failure of some high-profile digital transformations has led company leaders to be wary of transformational initiatives. Many business executives that we speak with have shared their frustrations that they are hoping to see greater agility from the technology organizations that support them. In response, many firms have established hybrid organizations, which include centers or excellence, analytic sandboxes, or innovation labs in efforts to derive benefits more rapidly from their data investments. A number of leading organizations are constructing these new functions with a combined team of business leaders, data scientists, and data engineers/architects, operating as internal “swat” teams to drive rapid results.
At a recent executive breakfast that we organized and hosted to discuss the survey results, chief data and analytics officers from many of the participating companies commented that senior leaders who strongly advocate for data and analytics within their organizations are incredibly valuable, but more the exception than the rule. Several steps to address the issue were mentioned by the executives in attendance. One suggestion was not to focus on overall data-driven transformation in a large enterprise, but rather to identify specific projects and business initiatives that move a company in the right direction. Another executive indicated that he had built a “Data Science University” with 400 students. This executive was undertaking a variety of communication initiatives to promote the successes of the program. Another was trying to implement agile methods in key programs, while avoiding terms like “data governance” that have a negative connotation for many executives. In spite of these efforts, none of the executives at the breakfast expected that these efforts would deliver rapid improvements in their firms’ data cultures.
Whatever the reasons for the failure to achieve transformational results from data initiatives, the amount of data continues to rise in business and society. Analytical decisions and actions continue to be generally superior to those based on intuition and experience. The companies in the survey are investing heavily in big data and analytics. In short, the need for data-driven organizations and cultures isn’t going away. Firms need to take a hard look at why these initiatives are failing to gain business traction, and what actions must be taken to reduce the cultural barriers to business adoption. Many companies have invested heavily in technology as a first step toward becoming data-oriented, but this alone clearly isn’t enough. Firms must become much more serious and creative about addressing the human side of data if they truly expect to derive meaningful business benefits.
