20/20 vision—in 2020 we will know a world where we can actually see everything. The first big change I’ll go for is IOT. With the Internet of Things generating tons of real-time data, we will know what everyone does, when and how. Around 30 billion objects are expected to be connected to the IOT by 2020.
Can billions of sensors give us billions of insights?
Everything will be connected, from sensors in soil and oceans, to wearables and even dog collars. This means that the billions of sensors in devices will be generating crazy amounts of data. We will need to process and analyze this data. The possibilities for finding insights are endless. But we have the challenge of finding a way to analyze the massive amount of new data which will be in a big part dark data. We currently estimate that dark data is about 90% of an organization’s data. BI needs to start finding insights on this dark data to actualize the business potential of IOT.
This leads to a second change. Until recently, data analytics has been focused on answering questions. The user asks a question and the program responds with an answer. Data analytics tools can find answers very easily. The real challenge is asking the right questions. Most organizations today analyze and benefit from only 10% of their entire data. We can blame this to an observational bias called the Streetlight Effect. This observational bias happens when we search for what we are looking for in the most obvious places. We do this because it is comfortable for us. Humans don’t usually like to exit their comfort zone. So we can blame it on our subconscious, but the point is we keep asking the same questions over and over again. We keep getting the same results.
Automated Analytics Will Get Insights
So how can we avoid the Streetlight Effect and actually get insights from all the new data coming from IOT? As new data keeps coming in—but is not cleansed or prioritized, the trend will be to rely on Automated Analytics to learn and suggest to us what data would be of interest to us and worth analyzing. Yes, Artificial Intelligence and machine learning, all those things that sound very futuristic, are now a reality in the Smart Data Discovery market.
There are tools like Panorama Necto 16 that are using Automated Analytics to crawl and analyze the data, finding automated insights and any anomaly in the data that might be worth taking a look at. The solution has patented algorithms that learn which types of insights will actually be worth showing to a specific user. In the next couple of years, we will start to see more solutions following this trend. These new solutions will have a challenge: to combine automation with the human intelligence that can never fully be replaced.
Say Hello To The CDO
Human intelligence is not going anywhere, on the contrary, more of it will be involved. As good data analysis will continue to become the key to organizations’ success, more people will be involved in data analytics. Meet the third big change that is predicted for 2020: Chief Data Officers. They are senior executives responsible for a firm’s data and information strategy, governance, control, etc. Data analytics will become critical for the most important business processes, like gaining customer intelligence, improving sales, productivity, and revenue. And enterprises will need an expert leading them through this—their CDO.
We are still a couple of years away from 2020, but as they say—time flies! (Unlike cars…yet) So it’s always good to keep track of how these trends develop to make sure we stay ahead of the game.