The Gartner Hadoop Survey was conducted in February and March this year among 284 Gartner Research Circle members. The Circle is a Gartner-managed panel composed of IT and business leaders.
The survey found that only 125 respondents who completed the whole survey had already invested in Hadoop or had plans to do so within the next two years.
"Despite considerable hype and reported successes for early adopters, 54 per cent of survey respondents report no plans to invest at this time, while only 18 per cent have plans to invest in Hadoop over the next two years," says Nick Heudecker, research director at Gartner.
"Furthermore, the early adopters don't appear to be championing for substantial Hadoop adoption over the next 24 months; in fact, there are fewer who plan to begin in the next two years than already have."
Hadoop is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.
Respondents find Hadoop was overkill for the problems the business faced, implying the opportunity costs of implementing Hadoop were too high relative to the expected benefit.
Read more: Across the board
Only over a quarter (26 per cent) of respondents claim to be either deploying, piloting or experimenting with Hadoop, while 11 per cent plan to invest within 12 months and seven per cent are planning investment in 24 months.
Gartner says the respondents cited two interesting reasons for the lack of intent.
First, several responded that Hadoop was simply not a priority.
Second, they noted Hadoop was overkill for the problems the business faced. Gartner says this implied the opportunity costs of implementing Hadoop were too high relative to the expected benefit.
"With such large incidence of organisations with no plans or already on their Hadoop journey, future demand for Hadoop looks fairly anaemic over at least the next 24 months. Moreover, the lack of near-term plans for Hadoop adoption suggest that, despite continuing enthusiasm for the big data phenomenon, demand for Hadoop specifically is not accelerating," says Merv Adrian, research vice president at Gartner.
Read more: Career watch: A strategic planning scenario for the CIO role
"The best hope for revenue growth for providers would appear to be in moving to larger deployments within their existing customer base."
Skills gaps continue to be a major adoption inhibitor for 57 per cent of respondents, while figuring out how to get value from Hadoop was cited by 49 per cent of respondents. The absence of skills has long been a key blocker, reports Gartner.
Tooling vendors claim their products also address the skills gap. While tools are improving, they primarily support highly skilled users rather than elevate the skills already available in most enterprises.
Hadoop vendors are responding to this challenge by offering a variety of training options. However, Gartner estimates it will take two to three years for the skills challenge to be addressed. Beyond skills, demonstrating the value of Hadoop is the second-highest challenge.
Respondents who are actively piloting or deploying Hadoop report small numbers of users accessing the cluster with 70 per cent of respondents having between one and 20 users accessing Hadoop and a surprising four per cent reporting zero users.
Read more: The CIO’s bi-modal innovation agenda
"Early Hadoop projects typically involve a small number of users and this no doubt keeps user populations down at this stage of the market; moreover, the Hadoop stack remains unsuitable for simultaneous use by multiple users, also keeping numbers down. Another factor, and possibly the best explanation for the small number of Hadoop users, is the skills shortage," says Heudecker.
"One of the core value propositions of Hadoop is that it is a lower cost option to traditional information infrastructure. However, the low numbers of users relative to the cost of cluster hardware, as well as any software support costs, may mean Hadoop is failing to live up to this promise."
Send news tips and comments to divina_paredes@idg.co.nz
Follow Divina Paredes on Twitter: @divinap
Follow CIO New Zealand on Twitter:@cio_nz
Sign up for CIO newsletters for regular updates on CIO news, views and events.
Join the CIO New Zealand group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.