Gitpod flaw shows cloud-based development environments need security assessments

Researchers from cloud security firm Snyk recently discovered a vulnerability that would have allowed attackers to perform full account takeover and remote code execution (RCE) in Gitpod, a popular cloud development environment (CDE). Cloud-based development environments are popular because they're easier to deploy and maintain than local ones and promise better security. However, organizations should properly assess security risks CDEs can introduce and are unique to their architectures, especially since they haven't received much scrutiny from the security community.

"Many questions remain unanswered with the adoption of cloud-based development environments: What happens if a cloud IDE workspace is infected with malware? What happens when access controls are insufficient and allow cross-user or even cross-organization access to workspaces? What happens when a rogue developer exfiltrates company intellectual property from a cloud-hosted machine outside the visibility of the organization's data loss prevention or endpoint security software?," the Snyk researchers said in their report, which is part of a larger project to investigate the security of CDEs.

Traditional integrated development environments (IDEs) that are locally deployed on individual developer workstations, can have a plethora of security problems and vulnerabilities, too. In fact, CDEs are in many ways a big improvement over traditional IDEs: They can eliminate the configuration drift that happens over time with developer workstations/laptops, they can eliminate the dependency collisions that occur when developers work on different projects, and can limit the window for attacks because CDE workspaces run as containers and can be short-lived.

When vulnerabilities are found in their software, the CDE provider can likely deploy a fix quicker than an organization would need to deploy security patches to all its developer workstations and laptops running a traditional IDE. Of course, security response times can vary among CDE providers, so organizations should choose their provider carefully if they're entrusting it with their development infrastructure, including code, access tokens, production secrets, and other intellectual property.

The commonly misunderstood cross-site WebSocket hijacking

The vulnerability found by Snyk, which the Gitpod team addressed within a day, is tracked as CVE-2023-0957 and falls into a category of issues known as cross-site WebSocket hijacking.

A core security mechanism built into browsers is known as the Same Origin Policy (SOP). This prevents code running on a site from reading information from another site that a visitor is logged into. Since browser requests to a site -- for example, site A -- usually include a user's session cookies, without SOP, a malicious site B visited by the user could load a resource from site A and be able to steal a user's session cookie with site A.

The problem is that this defense mechanism only exists for HTTP but not WebSocket, a two-way communication technology that allows a browser to exchange data with a server using a persistent connection. "When a WebSocket handshake relies solely upon HTTP cookies for authentication, a malicious website is able to instantiate a new WebSocket connection to the vulnerable application, allowing an attacker to both send and receive data through the connection," the Snyk researchers explained.

In other words, if a user visits a malicious website and that site opens a WebSocket connection on their behalf to another server they're authenticated to, the malicious site can send malicious commands through the connection and receive responses by piggybacking on the user's cookie. This is why WebSocket connections should be implemented with additional authentication.

How researchers exploited the now-fixed Gitpod flaw

The Gitpod architecture consists of multiple microservices deployed in a Kubernetes environment, with user workspaces deployed as ephemeral pods. Gitpod workspaces consist of a server component written in TypeScript and a dashboard web application built with React that communicates over WebSocket with a JSONRPC API exposed by the server. The dashboard is the interface that the developer interacts with and where they can import a repository from a source code management provider like GitHub. Once set up, the workspace is also made accessible via SSH and HTTP using dedicated subdomains under the gitpod.io domain.

The Snyk researchers verified that the Gitpod WebSocket implementation didn't use additional authentication and an attacker could open a WebSocket connection to the workspace from a different origin. However, a different mechanism recently implemented into browsers called SameSite cookies came into play rendering their attack ineffective.

SameSite cookies are meant as a defense against cross-site request forgery (CSRF) attacks, where a site can force a user's browser to issue a request to another site on behalf of the user. However, unlike the SOP, which checks the origin's scheme + host (including subdomains) + port, the SameSite policy only checks for the domain to be the same.

SameSite applies to all cookies, including those sent over WebSocket, meaning to launch a Cross-site WebSocket Hijacking attack against Gitpod, an attacker would have to use a malicious web page also hosted on gitpod.io. Since each workspace is assigned its own subdomain, the Snyk researchers had to find a way to serve a malicious web page from a Gitpod workspace they set up.

The default code editor in Gitpod workspaces is VS Code (Visual Studio Code) and this is exposed through a web-based interface. So, the Snyk researchers tried to kill the VS Code process inside the workspace using the command line interface and bind a Python-based web server to the port previously used by VS Code to serve their malicious HTML file. This didn't work, because a "supervisor' process that monitors the workspace restarted the workspace.

Eventually, the researchers observed that if they only killed the VS Code process but didn't bind the port to another process, the supervisor will only attempt to restart VS Code and not the whole workspace. This gave them the idea of stopping VS Code and replacing quickly it with a patched version they created that served their exploit via the /version API endpoint, which normally just returns the VS Code version number.

"We modified it so that the correct Content-Type of text/html and the contents of an HTML file were returned," the researchers said. "Now, we terminated the vscode process, allowing our newly introduced changes to load into a newly spawned VS Code process instance."

Finally, they had a link with their malicious web page running on a workspace inside Gitpod and on a Gitpod subdomain. Now all they had to do was send this link to a victim Gitpod user who was logged into their own workspace and if visited, it would allow the exploit to open a WebSockets connection to the victim's workspace and issue JSONRPC methods such as getLoggedInUser, getGitpodTokens, getOwnerToken, and addSSHPublicKey. The last method allows an attacker to add their own SSH key into the victim's workspace, ensure persistent remote access via SSH into the workspace.

The Snyk researchers praised Gitpod for its fast response time and patch but added that "as cloud developer workspaces are becoming increasingly popular, it's important to consider the additional risks that are introduced."