March 17, 2016

We're very pleased to present a guest blog post from Matthew Ritter here at the TXO Blog.

In an article at the end of 2014, this post hypothesized that big data was moving out of the hype phase and into a more stable and mature (if less exciting) “Plateau of Productivity”. For reference, here is the canonical Hype Cycle diagram:


One way to test the movement of a technology along this cycle is looking at search volume through Google Trends. When interpreting the Trends data, it’s important to remember that the Hype graph is in units of ‘expectations’, not ‘attention’. For example, a spectacular technical failure will crush expectations while increasing attention.

To drive that point home, let’s take a quick detour over to the Trends data for the term ‘self driving cars’:

Screen Shot 2016-03-16 at 8.04.21 AM.png

That giant spike? Definitely not an improvement in expectations:

Screen Shot 2016-03-16 at 8.06.24 AM.png

Bearing that in mind, let’s take a look at “Big Data” ended up doing in 2015:

Screen Shot 2016-03-16 at 8.07.55 AM.png

As that post predicted over a year ago, Big Data’s attention got a bit smaller in 2015. This could be partially due to a similar slowdown in interest in the Internet of Things, which was expected to be a driver. Instead, the dark horse of 2015 was certainly Virtual Reality, which easily outpaced the other topics, and is still rocketing upwards:

Screen Shot 2016-03-16 at 8.10.05 AM.png

Again, it’s worth stressing that the flat search trends for Big Data do not indicate that the technology’s development is over, just that it’s found its place, and is quietly powering some of industry’s most important applications. Of course, best practices around its implementation hold true:

  • People matter. That includes analysts who make sense of the many correlations that an automated system will spit out, but also executives who are responsible for acting on those insights (particularly when they are counter-intuitive!)

  • Be careful with who you hire (both full time and as a consultant) in these fast-moving spaces, because it can be hard to discern who really knows what they’re talking about, and who’s trying to ride the wave. That said, don’t be afraid to dip your own toe in the water - as I’ve written before, there’s no such thing as a fake data scientist.

  • Big Data implementations are IT projects like any other, with a very real risk of spiraling out of budget and schedule. They need to be planned, implemented in phases, and maintained.

  • Systems need to be stable, and usable for all applicable stakeholders

If those principles are followed, your Big Data initiative will have joined Google’s, Facebook’s, Twitter’s and the rest in the Plateau of Productivity. Congratulations - now, have you thought about your Virtual Reality strategy?

Guest Post by Matthew Ritter, PreInvented Wheel

TextOre, Inc.