Developers react to GitHub Copilot

Microsoft subsidiary GitHub announced the new AI-powered Copilot service as a private beta for developers on June 29, 2021, with the “AI pair programmer” embedded in Microsoft’s massively popular Visual Studio Code editor as an extension for beta users.

As developers write their code, Copilot will step in to make suggestions, much like the autocomplete feature in email applications like Gmail, but for code written in Python, JavaScript, TypeScript, Ruby, Go, and other programming languages.

Copilot was built in collaboration with OpenAI, the artificial intelligence lab founded by Elon Musk, Sam Altman, and others and that Microsoft invested $1 billion in last year. OpenAI adapted its GPT-3 language-prediction model for computer code for this project in a model it calls Codex.

Teaching neural networks to write code is not a new endeavour, with start-ups like TabNine and Kite working on similar projects. But the heft of Microsoft and OpenAI means Copilot is a strong entrant to the market from Day 1.

Philip John Basile, senior front-end developer at cybersecurity company Dragos, has dabbled with these other AI coding assistants, but told InfoWorld that Copilot is already “just on another level.”

The announcement certainly made a splash, generating 1200 comments on Hacker News in just two days and driving hundreds of early users to show off its hits and misses on social media.

In terms of the developer experience today, a few early users—many of whom are GitHub Stars or Microsoft MVPs—have been showing off the capabilities on social media and in livestreams. Here’s what they have to say.

What works well in Copilot

Cassidy Williams, director of developer experience at PaaS company Netlify, said that Copilot “is going to be incredibly useful for things like smaller utilities that I have to often rewrite across various projects. The fact that I could use it to do certain capitalisation patterns, or write certain pattern-matching functions for me that I would normally have to either just hunker down and write or look up how to do for the 100th time,” she told InfoWorld.

“Instead of autocomplete, it gives me ideas for full-fledged functions, random code items, and can even pull together some written paragraphs for when I blog,” Dragos’s Basile said, based on his early experiences with the tool.

“I think it really works extremely well as a very smart autocompletion tool,” prolific open source maintainer and software development consultant Alexey Golub told InfoWorld.

“It can infer the context from your code and finish whatever line you were typing accurately most of the time. It also helps in other scenarios outside of coding too, like writing blog posts, talk abstracts, comments, or documentation. Sometimes, the right word might just barely escape you, but then Copilot suggests it for you.”

Colby Fayock, a developer advocate at test automation company Applitools, stayed up all night playing with the tool once he was given early access.

“I was surprised at how well it was able to provide answers for a lot of the input I tried,” he told InfoWorld. “The first thing that came to mind was that this would be most useful for generating common utility functions like sorting by date that one might use in different projects. Typically, that might be a common Google search for a Stack Overflow answer, but now I can just write the function and have that answer given to me right in Visual Studio Code.”

Vinit Shahdeo, a software engineer at collaboration platform Postman, said “Copilot is definitely going to increase the developer’s efficiency by reducing development time and suggesting better alternatives for the code.”

What works less well in Copilot

Although Copilot looks like a very useful productivity booster, it still has a long way to go to replace actual chunks of human developer work.

“For more complex projects, I don’t think it can just write my code for me,” Netlify’s Williams said. “There were some nuances as I played with it where I could tell it was doing something fairly generic when I wanted something else. But, if it means I can get the tedious code out of the way so I can get my business logic done right, I’m all for it.”

Dragos’s Basile said, “When you are working with it, it will give you 10 items that could be the right fit. Some of them are just flat-out terrible while others are perfect. You really need to sift through the sand to find the diamond.” Similarly, technical blogger Ray Villalobos often struggled to get a useful result, so he resorted to retyping comments until Copilot offered something useful.

While Applitools’s Fayock was pleased with the tool, he didn’t feel as much joy when writing in React. “I tried some examples like generating a user profile, and part of the issue is creating a React component that is useful requires more than just some React, but also styles to go along with it,” he said — which Copilot does not support. “It also seems to struggle when trying to generate something that would ultimately require a library to import.”

Is Copilot ready for the enterprise?

So the AI isn’t coming for your developer jobs just yet. Furthermore, Postman’s Shahdeo is wary of newcomers to the industry becoming reliant on tools like Copilot, “as this might block their learning path if they get into the habit of autocomplete.”

“I’m not sure a computer will ever be as good as a person, but it does give you a good starting point and sometimes the code it returns makes you think about how to approach as solution,” blogger Villalobos said.

Although Copilot shows initial promise, even GitHub is cautious about its enterprise applicability at this early stage. GitHub notes in its FAQ that the code Copilot suggests “may not always work, or even make sense. While we are working hard to make GitHub Copilot better, code suggested by GitHub Copilot should be carefully tested, reviewed, and vetted, like any other code. As the developer, you are always in charge.”

“I think it can already be used at enterprise level,” Golub said. “Worst case, it will just not do anything (if its suggestions are not relevant then you can just ignore them); best case, it will save you time and be more productive.”