Amazon Web Services (AWS) may be the biggest cloud vendor by far, but it’s not the world’s biggest open source contributor. And some people aren’t ready to let the cloud giant forget it.
For example, Iguaz.io CTO Yaron Haviv said that AWS is just “putting up an open source show,” not “real” open source like Google does—even though AWS actively contributes to Apache MXNet, with more than 40 per cent of contributions from more than 400 developers to the project).
Similarly, analyst Krish Subramanian believes that whatever AWS’s contributions, they’re not enough given how much AWS has benefited from open source.
But such statements are neither fair nor accurate.
No corporation gifts open source code out of charity. It’s always a way to further corporate interest. As Matt Wood, AWS’s general manager of AI and deep learning, described in an interview, the cloud vendor has always existed to take technology that was traditionally only in reach of well-funded organizations and make it accessible to a much broader audience.
Open source is just one more way that Amazon seeks to democratise technology. And for most developers and their employers, it won’t even be the most important way.
The three ways AWS does open source
Last week, AWS teamed up with Microsoft to open-source Gluon, a project that aims to make it dramatically easier for developers to program neural networks. Although some observers made a big deal of these fierce rivals collaborating on code, such joint work is nothing new for either company.
Although Microsoft has been more public about its open source outreach over the last few years, that’s largely because it had so much damage to undue, given Microsoft’s early history of attacks on open source.
AWS doesn’t have such history, so it has less to prove publicly. And criticism that AWS doesn’t have nearly enough history with open source, given how much it has profited from open source, simply misses the mark. Why? Because the criticism confuses ends and means.
Wood pointed out that AWS “routinely hears that there’s a lot of interest in [machine learning], but companies often don’t know how to get started.”
One response might be for AWS to open-source some machine learning software and hope that developers will learn from that code and become proficient in machine learning. It’s a reasonable approach, one with which Google has had some success with TensorFlow.
But such an approach is insufficient by AWS standards.
Wood breaks down AWS’s approach into three levels, with machine learning as the use case:
- The first level includes “black box” offerings like Polly and Lex. These application services are geared toward developers with limited amounts of data to train machine learning models, giving them the ability to build things like voice services. Rather than build out their own sophisticated machine learning services, in other words, this top level of application services rely on AWS to provide the heavy lifting.
- The second level offers a collection of services for developers who have adequate data for training machine learning models but still don’t want to bother with the undifferentiated heavy lifting of building everything themselves. Such developers might simply want to connect an S3 bucket to an engine and start running their machine learning models.
- The third level is where open source comes into play, because it includes open source packages and libraries to help put neural networks into the hands of mere mortals. Here, AWS both contributes to open source projects and optimizes them to run on AWS. As Wood tells it: “We’re incentivized to contribute to projects like Apache MXNet because customers want to use it, and we want it to be as good as possible if they’re going to run it on AWS.”
Open source works when contributors are selfish for customer benefit
It’s the “on AWS” part that gets critics riled up. These people believe AWS should be contributing for the good of humanity, not for its bank balance. Such thinking, however, is somewhat stupid.
First, AWS is a commercial enterprise, and it needs to act in its self-interest. That can be done while still contributing to humanity, but the how it contributes naturally needs to align to its economic interests, which includes its customers’ interests. That’s true for every corporation investing in open source.
It’s wrong to say AWS wholly fixated on pushing its own services. Using databases as an example, Wood notes that in addition to making open source projects like MySQL (as RDS) run better on AWS, AWS also works with Oracle, Microsoft, and others to ensure their databases run well for joint customers.
In the machine learning world, AWS has “teams to work on MXNet, TensorFlow, etc. to make sure it all runs well on AWS to expand freedom, flexibility, and choice” for developers. In this way, said, “we are certainly starting to crawl up onto dry land with Gluon,” as it becomes the “easiest onramp” to machine learning.
Then there’s the early-days status of machine learning. “We haven’t yet grown out of the primordial soup” for machine learning, Wood said. Developers must still be quite technical to adopt machine learning techniques.
So, rather than foist one approach on the market, AWS tries to cater to differing levels of ability: “There’s a real advantage to be able to pick and choose between these options and run those you want,” Wood avows. This early-days openness is common in many areas of open source.
Is AWS selfish? Sure. Does that selfishness translate into greater developer productivity with machine learning and other enterprise software in the process? Yes. And it’s not merely a convenient byproduct: It’s the whole reason AWS exists.
“We listen to our customers. We really do work backward from what our customers are asking for,” Wood added. Those customers aren’t universally begging for more open source; they’re anxious for vendors like AWS and Microsoft to help them access the power of AI more easily, whatever the means.
Those services aren’t open source, but that’s not the point. If they make it easier to run open source, and get involved with machine learning/AI, then it’s a big win for customers and, “selfishly,” for AWS. Everyone wins.
This article originally appeared on InfoWorld.
