Gartner may think big data has met disillusionment, but it's barely started

Gartner has a tool called the Hype Cycle it designed to trace where the latest and the greatest technologies are inside companies starting with innovation and ending in the plateau of productivity. Gartner believes we have already entered the trough of disillusionment where Big Data is concerned.

I would maintain we have haven't started yet.


Yes, we've been hearing about Big Data for a couple of years now, but that doesn't mean that many companies actually understand the implications of big data or even what it really means. Nor does it mean we have the tools and technologies in place to really take advantage of the torrent of data that is out there now. And there's only more coming.

If you listen to Gartner itself we are on the way to having a deluge of data from the so-called Internet of Things, sensors connected to buildings, devices, cars, just about everything and Gartner believes there's going to be a ton of data coming from that. in fact, Gartner is predicting that by 2020, there will be 37.3 billion connected devices with 7.3 billion devices and 30 billion sensors. That, my friends is a lot of data. You could even say it's big.

Speaking at the Alfresco Summit last week, MIT professor Andrew McAfee said he runs into a lot of skepticism around big data when he talks to companies about it, but he believes that when companies make decisions based on good data, they will ultimately make better decisions.

He says most companies tend to make decision based on the highest paid person's opinion or the HiPPO method. They make the most money, so they must know what they're talking about, right? But McAfee says organizations need to follow the data and the data will show you the way.

McAfee used Google as an example. Google's hiring practices are legendary. They bring in the best and the brightest and pepper them complex brain teasers to see if they could solve them on the spot. There was a core belief in this methodology and that it lead to hiring the smartest and best employees. But Google, one of the most data-driven organizations on the planet hadn't ever tested or measured this belief. When they checked the data, they found there actually was no correlation between performance on these brain teasers during the interview process and how successful they became as employees. When they followed the data they had to go a different way.

And that's the thing, most organizations have no clue what to do with big data. McAfee says when he convinces people that big data is real, he finds that every executive team he speaks to asks the same three questions. And that so many companies fail to realize the potential of Big Data, it's clear we are just at the very beginning of this and it's going to take better tools and technologies to help us dig through all this data and figure out how to make the best use of it.

And it's going to take smart people to ask the right questions. But just because we haven't quite figured out data-driven decision making doesn't mean we we should be disillusioned about it. We need to find a way to make all this data work for us. Today, we've barely scratched the surface. 

Add new comment

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Enter the characters shown in the image.