How to avoid the data junk drawer: 5 Big Data trends for 2018
With every passing year, data gets bigger. However, without practices in place to guide insight from Big Data, companies are left with a data junk drawer.
Big Data will always be relative to the industry and size of the business it belongs to. Regardless of the scope of data a company is holding onto, there is a massive amount of knowledge to be found in Big Data. In fact, about 2.5 quintillion bytes of data are created daily, and there are no signs the rate will slow.
But 2018 is the year Big Data is expected to mature with easier access to solutions and real-time data stored on the cloud. To understand how to do so, here are the five Big Data trends to expect in 2018:
1. It's time to think of data as an asset
Before companies can embrace a business strategy directly influenced by Big Data analysis, "organizations should start treating information more as an actual enterprise asset rather than just talking about it as one. They should actually apply asset management principles and practices to it," said Doug Laney, VP and distinguished analyst at Gartner. "It should be monetized, measured and managed" as an asset.
When data is combined through sales, marketing or feedback analysis, true business value arises. Within those categories, businesses should focus on descriptive, diagnostic, predictive and prescriptive analytics.
Essentially, businesses can look at the what, why and how of their data. The results can then be sent to various departments in a company.
After all, knowledge is power, and Big Data is knowledge.
2. Companies will continue to use data lakes
Data needs to come from a "trusted place" so it cannot be misused, according to Johnny Thorsen, VP of travel strategy and partnerships at Mezi. This highlights the importance of data lakes, a method of storing troves of data.
Data lakes allow Big Data to become actionable data. The information stored in data lakes are oftentimes considered "raw" because they serve as repositories with the intention of later analysis for further understanding.
In the year ahead, 70.8% of IT professionals are interested in the use of traditional ETL (extract, transform, load) for their data lakes. Predictive analytics and real-time analytics come in at second and third, respectively, according to a Syncsort report.
With proper usage of data lakes, companies are "no longer speculating or guessing, you actually have real, tangible evidence of when things are failing, when people are using, how they're using it, and you might find that they're doing it in ways you didn't even perceive," said Jason McDonald, president of Contino.
In fact, information in data lakes can come from mainframes, streaming data, and relational databases. To fill their data lakes, nearly 70% of IT professionals expect to use relational database management systems, 62.5% expect to use their own enterprise data warehouse and 46.4% expect to use NoSQL databases.
3. Don't plan on decreasing spending on data lakes
The integrity and quality of data is expected to improve with the use of data lakes because it enables companies to store as much as they want and "to be creative from there," according to Thorsen. Companies usually turn to hosts like Hadoop and businesses are projected to spend about $2.3 billion on the company and its services by 2021.
However, while nearly three-quarters of organizations planned to invest in data lake architecture by 2018, it is important to remember that data lakes are typically managed by CIOs and their respective IT teams, according to Forrester.
This means it is easy for IT departments to extract data useful to them and potentially ignore other parts of the business. In doing so, companies may not be benefiting from data lakes at the capacity they could.
Additionally, while pooling data in one place seems efficient and convenient, it also invites security concerns. However, compared to the alternative of data existing in multiple locations, data lakes are still "security neutral."
4. Plan to reduce investments in on-premise Big Data platforms
It should come as no surprise that the continuation of moving Big Data from on-premise solutions to the public cloud will continue in 2018. Nearly one-third of Big Data workloads run in the cloud, according to OVUM ICT Enterprise Insights.
However, companies need to keep in mind that time is of the essence when transitioning where their Big Data is stored. The longer it takes for a company to migrate data to the cloud, the more rivals can gain a competitive edge by already using insights derived from cloud-based Big Data.
This year companies should expect the price of storing Big Data to decline and processing speeds to double, according to Forrester. Though it is a common belief that the total cost of ownership (TCO) and risk for five to 10 years for on-premise solutions are cheaper than being on the cloud, it is inevitable that prices will drop. This is already evident.
Last month, Microsoft announced a 52% decrease for its Big Data offering, Azure HDInsight. Already, the company boasted that customers saw their TCO lower by 63%, which is a big deal for companies resistant to digital transformation.
5. Look for analytical talent, not just managing talent
Hiring managers need to put more focus on analytical talent than management roles right now, according to Laney.
Big Data infrastructures are increasingly being outsourced, so it is more important for hiring managers to look for analysts rather than employees who can "manage a server farm."
In addition to more analysts, data curators will become a larger part of the Big Data landscape.
"We'll see more and more companies looking to external data sources, so a lot of Big Data that's out there isn't data that the company has itself, but data that it's integrating from external sources," said Laney. The data curator's role is to identify those external data sources.
Follow Samantha Ann Schwartz on Twitter