ARN

Solving today’s data challenges with DataOps

Organisations today face multiple data challenges: data volumes are growing exponentially; important data is spread across ever more systems and locations and is in constant motion across an increasingly varied ecosystem that includes multicloud and the edge.

Because of these challenges organisations are failing to gain full business advantage from the increasingly rich information their data holds.

Anew approach to data management, DataOps, emerged several years ago to help organisations meet these challenges and improve business outcomes. It represents a new way to connect data producers with data consumers. However no matter how data is stored, managed and used, data security must remain paramount.

This brandpost will review these challenges and examine how DataOps gives organisations new ways to manage their data securely and extract full advantage from it.

Finally, it will introduce Seagate Technology and show how its data storage systems address these challenges and enable the channel to help businesses securely gain maximum advantage from their data.

THE DATA CHALLENGES FACING ORGANISATIONS

In late 2019 and early 2020, research firm IDC undertook a survey of 1500 IT executives in businesses ranging from mid-sized to large enterprises around the globe, sponsored by Seagate.

From their responses, IDC projected enterprise data volumes would increase at an annual rate of 42.2 percent over the following two years, and it estimated only 32 percent of the data available to enterprises was being put to work.

IDC also estimated approximately 30 percent of stored data to be held in internal data centres, 20 percent in third-party data centres, 19 percent in edge data centres or remote locations, 22 percent in cloud repositories and 9 percent in other locations.

IDC also found this data to be increasingly in motion. “On average, organisations now periodically transfer about 36 percent of data from edge to core. Within only two years, this number will grow to 57 percent,” it said. This means enterprises will have to manage much greater volumes of data in motion. This data will also include important new data being generated in systems at the edge that handle transactions, customer interactions and, in operational technology, interactions with connected industrial systems.

IDC also predicted the management of data in a multicloud ecosystem would become a top data management challenge over the following two years, with management of data in hybrid cloud a close second.

From the responses, IDC identified the top five barriers to putting data to work as being:

- making collected data usable.

- managing the storage of collected data.

- ensuring that needed data is collected.

- ensuring the security of collected data.

- making the different silos of collected data available.

IDC also found value being extracted from only 32 percent of the data available to organisations: 44 percent of available data is not captured and of the 56 percent captured, only 57 percent is exploited.

DATAOPS – WHAT IT IS AND WHY IT MATTERS

Theconcept of DataOps, short for data operations, is credited to consultant and writer Lenny Liebmann, in a blog post in 2014. It is a discipline designed to ensure data creators are connected with data consumers to enable collaboration and to accelerate innovation. DataOps aims to facilitate a holistic view of data and enable users to access and derive optimal value from it—both data in motion and at rest.

DataOps uses Extract, Load, Transform (ELT) functionality to pull data from multiple sources—core, cloud and edge—and load it into a common structure such as a data lake.

DataOps can be applied to the entire data lifecycle, from data preparation to reporting. It requires the data analytics team be tightly connected with information technology operations.

Correctly implemented DataOps can lead to measurably better business outcomes: boosted customer loyalty, revenue, profit, and a host of other benefits.

Most respondents to the IDC survey said DataOps was “very” or “extremely” important, yet only 10 percent reported having implemented DataOps fully across the enterprise.

However, implementing and gaining full advantage of DataOps can be challenging. It requires these large volumes of data to be fully and correctly correlated. Artificial intelligence (AI) and machine learning (ML) can be deployed to make this challenging task much easier.

The application of DataOps can produce distinct customer, and hence business, benefits. Optimising the governance and flow of data contributes to better quality offerings, which directly affects how customers feel about their purchases. It can also increase speed of delivery, which also increases customer satisfaction.

DATA SECURITY AND MANAGEMENT

All the benefits that can be gained from DataOps can be lost if data security is compromised. So it is hardly surprising that data security was rated as one of the highest concerns and priorities of IT leaders and business leaders in IDC’s survey. Yet, two thirds of respondents said they had insufficient data security.

There are three key components to data security.

Data Classification All data needs to be classified according to its importance to the organisation and the dangers if it falls into the wrong hands. This is a mammoth task that requires close collaboration between all who create, use or have an interest in the data. It requires a comprehensive data classification scheme, and the appropriate classification of all data according to this scheme.

Data Flows Business owners must track all data flows and ensure none of these are illegitimate, either as a result of error or malicious interference. A comprehensive understanding of data flows will do much to reduce an organisation level of data risk.

Access Control Role-Based Access Control (RBAC)—limiting access to data to those who need it—is basic but rarely comprehensively implemented. Initial determination of access rights is a challenge and maintaining these is even more difficult. Rather than striving for the same level of access control across all data, efforts should be focussed on the most important data. Beyond RBAC, other controls that can be considered include encryption of data in flight and at rest, and user education.

Encryption is a vital technique to protect data. All data should be encrypted when in transit: HTTPS for web traffic, StFTP for file transfers, and IPSec or SSL VPN for all remote access. For data at rest, data on mobile devices presents the greatest risk. All data on laptops should be protected with enterprise-managed full-disk encryption, mandated in mobile device management software for mobile devices.

SOLVING DATA CHALLENGES WITH SEAGATE

Seagate has been offering highly innovative and easily integrated storage systems for more than 40 years. These systems are built on trust, affordability and performance, and designed to deliver a seamless experience at the best value.

Seagate backup and recovery solutions deliver the best value per petabyte with leading capacity, proven reliability and multiprotocol support for mixed storage environments.

They use cost-effective, high-throughput, scalable storage technology to deliver optimal support for analytics and machine learning workloads.

Seagate’s cloud solutions improve access, data security, efficiency and compliance, empowering businesses to provide secure and reliable backups of critical data.

In June 2021 Seagate introduced Exos CORVAULT, an intelligent mass-capacity storage technology designed to streamline data management and reduce human intervention for macro edge and data centre environments. Exos CORVAULT incorporates Autonomous Drive Regeneration (ADR), a self-correcting system developed by Seagate. It renews a drive in-situ without the need for a manual drive swap. It enables most drives to be returned to dependable service by reconfiguring the drive to bypass errant components.

Seagate expansion shelves and JBOD models enable initial storage investments to be scaled efficiently into exabyte-class data centres while simplifying operations and optimising costs. Rack space, storage expenditures and administrative workload, can all be optimised, reducing overall data centre costs.

Want to learn more about Seagate’s technologies and how they enable organisations to overcome data challenges and maximise the potential of DataOps?

Check out the on-demand video of Datasphere 2021 and learn how Seagate’s new mass storage solutions offer the critical hardware, software and services needed to activate data for more ease, security, and scalability: from edge-to-cloud and all points in between. We discuss new ways and proven strategies for making your business’s data dynamic, limitless, open and accessible.

Through expert-led panel discussions and tactical deep dives, we explore real-world challenges faced by today’s enterprises and how to overcome them with Seagate’s most innovative solutions.

Want to explore the benefits of becoming a Seagate Partner?

We understand different partners have different needs, so we’ve created specialised partner programs to help you meet your specific goals. Check us out.