Credit: Dreamstime
Big data is set to move out of the spotlight in the near future, with 70 per cent of all organisations to take up small and wide data instead by 2025.
This is according to research firm Gartner, with its distinguished research vice president, Jim Hare, claiming the coronavirus pandemic is creating historical data that reflects past conditions set to “quickly become obsolete”. This factor, as a result, is breaking production artificial intelligence (AI) and machine learning (ML) models.
“In addition, decision making by humans and AI has become more complex and demanding, and overly reliant on data hungry deep learning approaches,” Hare said.
Instead, the VP recommended the take-up of small data, an approach that needs less data but can offer useful insights, and wide data, the analysis and synergy of a variety of small and large, unstructured and structured data sources.
“Taken together they are capable of using available data more effectively, either by reducing the required volume or by extracting more value from unstructured, diverse data sources,” Hare said.
According to the firm, small data is an approach that uses certain time-series analysis techniques, few-shot learning, synthetic data or self-supervised learning.
Meanwhile, wide data applies 'X analytics', in which the 'X' stands for finding links between data sources, as well as for a range of data formats, including tabular, text, image, video, audio, voice, temperature, smell and vibration.
“Both approaches facilitate more robust analytics and AI, reducing an organisation’s dependency on big data and enabling a richer, more complete situational awareness or 360-degree view,” said Hare. “D&A [data and analytics] leaders apply both techniques to address challenges such as low availability of training data or developing more robust models by using a wider variety of data.”
One application of these approaches, according to Gartner, includes customer service, such as demand forecasting in retail, real-time behavioural and emotional intelligence in customer service applied to hyper-personalisation, as well as customer experience improvement.
Meanwhile, physical security, fraud detection and adaptive autonomous systems can also utilise small and wide data, particularly when applied to robots, which learn through constant analysis of correlations and time and space through different sensory channels.
