How to solve the IoT's data management challenges

How is the IoT impacting data management?

Most conversations about the Internet of Things revolve around just how many devices will come online over the next few years. Rarely do people discuss how much information these devices will produce or the costs of managing that data.

According to a study from Dell EMC, the IoT is contributing to a growing challenge: The world's available storage capacity is growing at a slower rate than the digital universe. This means the IoT and devices are producing more data than storage solutions can handle.

How is this trend impacting enterprises? How can organizations leverage data management solutions to address this growing problem? 

What the IoT demands of storage 

How the IoT impacts an organization largely depends on how that institution operates.

For example, suppose a pharmaceutical company wants to integrate smart devices into its factories to automate complex processes, gather production data and predict equipment failures. Based on these objectives, it's clear each facility must be able to process and analyze a large portion of this data in real time or near real time. 

The situation described above introduces four challenges: 

  1. The devices produce unstructured data, which makes traditional relational data management schemes obsolete. 
  2. Each factory must have an on-premises storage array capable of handling continuous data streaming. 
  3. All of the data must be re-routed to an off-premises environment, whether a private cloud, public cloud or conventional data center. 
  4. Metadata must denote a datum's device origin, time of creation, level of sensitivity and other factors. 

In addition, the amount of data factories produce may increase over time as devices receive software updates. What data storage and management solutions can satisfy these parameters? 

How does the industrial IoT impact data management? How does the industrial IoT impact data management?

Data management solutions for the IoT 

Given how often applications need to interact with IoT, it's clear that flash arrays are the optimal choice, given that solid-state drives support faster data transfer rates than hard disk drives. However, how does that hardware fit into a holistic solution designed to support IoT?

Not surprisingly, NetApp and Veritas actually have a couple of systems which address the challenges we outlined in the previous section. 

NetApp's solution actually works in conjunction with SAP HANA solutions, which provide real-time analytics. Specifically, SAP HANA TDI operates over SGI UV 300H servers combined with NetApp AFF/FAS system. The latter is a feature-rich, multiprotocol storage platform designed to:

  • Automatically back up data within SAP HANA. 
  • Provide a persistent storage medium for in-memory databases.
  • Utilize NetApp ONTAP to establish scalability. 

With respect to Veritas, our partner offers a software-defined storage solution called Access. Access has a number of of cost-effective features that are pretty useful. 

For instance, Access detects which files are of low use, and relegates them to low-cost disk-based storage. This is a helpful function for enterprises that want to store IoT data for later analysis. Veritas specifically designed Access to handle unstructured data, which addresses many of the issues associated with the IoT. 

Ultimately, if you're integrating the IoT into your operations, speak with a consultant specializing in data management. It's best to develop a strategy on how you can accommodate the technology before scaling storage after the fact.