Data is the new oil but without the sort of refineries to turn it into more useful products, it remains useless for organisations in Australia.
That’s where data science comes into the picture, converting information into intelligence, according to DataSpark chief operating officer, Shaowei Ying.
As a subsidiary of the Singtel Group, which also owns Optus, Ying said in the geospatial era that’s unfolding, data mining is fast becoming an "essential" part of the industry, especially in the telecommunications space.
"We want to play in this field - it is an area that makes sense for us," Ying said. "We have a role to play in this geospatial data.
"Telco data is very rich in terms of location and size. But having the data is not good enough, you need to extract that data and extract intelligence out of it. After extracting that intelligence, you need to apply it in a meaningful way to create a social economic impact.
"But in order for a business to make sense, it needs to be done at a certain scale. That’s where the engineering part of it comes in."
According to Ying, the data flowing through a telco communication network is "highly rich and multi-dimensional", presenting new opportunities for organisations as a result.
"Data analytics is not new to a telco," Ying said. "Telcos have been harvesting insights from data sets to drive their own internal planning and operational efficiencies.
"A lot of this information that telcos have and are using to drive information can also be of relevance to external parties, not just for the purpose of optimising retail footprint, but also to enable other use cases for transportation, or insurance, for example."
As a result, Ying believes such location based insights services could reach $10 billion globally given its ability to not only address different functions, but different verticals and sectors also.
Yet for Ying, such data sets remain complex to process and make sense of due to the "huge number" of location updates happening for millions of subscribers at the same time.
"So, the question is, how do you do that in an ultra low latency manner?" he asked.
"That’s where data science needs to come in to improve the quality of location data and derive useful data features that we can call mobility intelligence."
As explained by Ying, the next step is figuring out how to apply these data features.
"The analysis has to be done on scale - transforming the data, analysing the data, and in the process, visualising, then make a decision out of it," he said.
"In many cases, you have to go back and forth a few times but it is essential that you have the right tools to help you with it. Spend time and resources to build tools to enable data scientists to do work and extract features.
"At the end of the day, we want to be a software enabled algorithm factory that is moving towards the delivery of not just code, but algorithms. Businesses should also look for partners to do it at a much larger scale and accelerated rate to succeed."
Hafizah Osman is attending Cloudera's Strata + Hadoop World conference in Singapore as a guest of Cloudera.
