Credit: Dreamstime
The data challenge
Most AI projects require data. Good data, relevant data, data that’s properly labeled and without biases that would skew the results.
For example, a company looking to keep cats out of a hen house might choose to install a camera and image recognition technology to spot cats coming in. But success hinges on having an adequate training set.
“You’ll need to have lots of pictures, and those pictures will need to have labels on them about whether they have cats in them or not,” says Gartner analyst Whit Andrews, adding that collecting this data is time consuming and expensive. And once it’s all gathered, will the company be able to reuse the same data set for other projects?
But what if it turns out that the business actually needs to know how many cats are coming into the hen house? Then that original data set of pictures will need to be relabelled with the number of cats in each picture as well.
“Maybe one cat is not that expensive, but a herd of cats is a problem,” Andrews says.
Plus, if only a small percentage of images contain multiple cats, then getting an accurate model will be substantially more difficult. This situation comes up frequently in marketing applications, when companies try to segment the market to the point that the data sets become infinitesimally small.
“Almost every company I know of uses segmentation for customer targeting,” says Anand Rao, partner and global AI leader at PricewaterhouseCoopers.
If they collect data expecting it to be used for one purpose, and wind up using it for another, the data sets might not meet the new requirements.
For example, if the data collection is set up so that there’s a balance of data points from each region of the United States, but the business question winds up being about the needs of a very narrow demographic segment, all the inferences will be useless. Say, for example, if the company is interested in the purchasing habits of Asian-American women in a particular age range, and there are only a couple in the sample.
“Be very clear about what decision you want to make with your segmentation,” Rao says. “Try to make sure that the sampling you’re doing is both representative, but also it captures your questions.”
The sample problem occurs in any system trying to predict rare events. For example, if a company is looking for examples of fraudulent behaviour, in a data set of a million transactions, there are a handful of known fraudulent ones — and an equal or larger number of fraudulent transactions that have been missed.
“That’s not very useful for inferencing,” Rao says, adding that this happens a lot with business process automation when a company has many people doing particular tasks each day, but doesn’t capture data about how those tasks are being done, or doesn’t capture the right data necessary to train an AI on how to do it.
“In those cases, you should go and build a system to capture that information,” he says. “Then, a few months later, come back and build the model.”
And for projects that don’t need data, AI is not the right way to go. For example, some business processes, such as insurance and underwriting, are rules based, Rao says. “You can build a rules-based system by interviewing experts and pulling together traditional formulas. But if you can do it with rules and scripts, you don’t need AI. It would be overkill.”
Using an AI for such a project can require more time and the accuracy might be no better, or only slightly better — or you might not need the improved performance.
“So you won’t have the ROI because you’re spending time on a problem that you could have already solved,” he says.
A $300 million AI mistake
In November, real estate company Zillow announced that it was writing down US$304 million worth of homes that it purchased based on the recommendation of its AI-powered Zillow Offers service.
The company may also need to write down another US$240 to US$265 million next quarter — in addition to laying off a quarter of its workforce.
“In our short tenure operating Zillow Offers, we’ve experienced a series of extraordinary events: a global pandemic, a temporary freezing of the housing market, and then a supply-demand imbalance that led to a rise in home prices at a rate that was without precedent,” Zillow CEO Rich Barton said in a conference call with investors.
“We have been unable to accurately forecast future home prices. … We could blame this outsized volatility on exogenous black swan events, tweak our models based on what we’ve learned and press on. But based on our experience to date, it would be naïve to assume unpredictable price forecasting and disruption events will not happen in the future.”
AI learns from the past, says Tim Fountaine, senior partner at McKinsey. “If something hasn’t happened in the past, then it’s impossible for an algorithm to predict it.”
And AIs don’t have common sense, he adds. “An AI algorithm designed to predict the output of a factory that has never seen a fire before, won’t predict that the output would plummet if there’s a fire.”
Predicting property prices is an interesting use of AI, he says. “But you can see everyone becoming a little gun-shy of that type of application.”
