Cloud servers hacked via critical SaltStack vulnerabilities

Attackers are exploiting two critical vulnerabilities disclosed late last week in the popular SaltStack infrastructure automation software to take control of servers. Several organizations and open-source projects already had their servers hacked and had to shut down services over the weekend.

The attacks began a couple of days after the vulnerabilities were publicly disclosed without a proof-of-concept exploit being available, highlighting that IT operations teams have very little time to react when flaws become known and should increasingly rely on automated patching.

The Salt vulnerabilities

On April 30, researchers from security firm F-Secure published an advisory about two vulnerabilities -- CVE-2020-11651 and CVE-2020-11652 -- found in Salt, a popular open-source Python-based framework that's used to automate tasks, data collection, configuration and updates for servers in private data centers or in the cloud. The Salt architecture involves the use of a master server where administrators can define tasks and clients called "minions" that execute them.

"The vulnerabilities described in this advisory allow an attacker who can connect to the 'request server' port to bypass all authentication and authorization controls and publish arbitrary control messages, read and write files anywhere on the 'master' server filesystem and steal the secret key used to authenticate to the master as root," the F-Secure researchers said. "The impact is full remote command execution as root on both the master and all minions that connect to it."

F-Secure published its advisory one day after SaltStack, the company that maintains Salt, released versions 3000.2 and 2019.2.4 of the framework to address the issues. Even though they decided to withhold the proof-of-concept exploit code to buy users more time, the F-Secure researchers warned at the time that "any competent hacker will be able to create 100% reliable exploits for these issues in under 24 hours." The company also warned that based on internet scans, over 6,000 Salt master servers were directly exposed to the internet and could be directly targeted.

Salt exploit reports start coming in

Over the weekend, security experts reported on Twitter that they were seeing exploitation attempts for the Salt vulnerabilities. Confirmation of successful compromises then started coming in from different organizations.

The LineageOS Project, which maintains the popular community Android firmware of the same name, had to take all its servers down, including its website, mail server, wiki, gerrit, download servers and mirrors.

Ghost, a blogging platform that maintains an open-source content publishing solution based on Node.js, was also hit and had to take servers offline, which impacted its Ghost(Pro) service and Ghost.org billing, although no payment card information or plaintext credentials were affected.

"Our investigation indicates that a critical vulnerability in our server management infrastructure (Saltstack, CVE-2020-11651 CVE-2020-11652) was used in an attempt to mine cryptocurrency on our servers," the company said on its status page. "The mining attempt spiked CPUs and quickly overloaded most of our systems, which alerted us to the issue immediately."

Some customer sites on the Ghost(Pro) service suffered network instability, partly because of new firewalls introduced in response to the attack. The company also cycled all sessions, passwords and keys and reprovisioned its servers.

Certificate authority DigiCert reported that one of its Certificate Transparency logs was affected after attackers used the Salt exploits to compromise a Salt master server. Certificate Transparency is a standard used by certificate authorities to publicly announce the digital certificates they issue. The logs are digitally signed and are meant to be used by external monitors to detect potentially fraudulent certificates.

"I'm sad to report that we discovered today that [Certificat Transparency] Log 2's key used to sign SCTs was compromised last night at 7 pm via the Salt vulnerability," said Jeremy Rowley, DigiCert's vice president for product development, on an industry mailing list. "Although we don't think the key was used to sign SCTs (the attacker doesn't seem to realize that they gained access to the keys and were running other services on the infrastructure), any SCTs provided from that log after 7pm MST yesterday are suspect. The log should be pulled from the trusted log list."

So far the attacks had the goal of deploying cryptocurrency mining malware on servers, but Salt is a very powerful tool and, as the reported incidents show, attackers could have used the exploits to do much more, including to steal sensitive data.

Patch Salt as soon as possible 

On Monday, SaltStack published a blog post urging all users to update their Salt master servers and to restrict direct access to them from the internet, as is recommended in its Salt hardening guide. "A scan by the security firm that identified the vulnerability found approximately 6,000 Salt Masters exposed to the Internet and vulnerable," said Moe Abdula, senior vice president of engineering at SaltStack, in a blog post. "While this is a very small portion of the Salt installed base, we consider it to be one too many."