Microsoft stirs up server soup in effort to be cool

The thought of plunging a server rack into a vat of boiling liquid is likely to send most IT professionals into a fit of discomfort, yet this is exactly what Microsoft has been doing as it works to find new ways of keeping hardware cool.  

The vendor is in the midst of developing a two-phase immersion cooling system for servers as part of its long-term plan to keep up with demand for faster, more powerful data centre computers at a time when reliable advances in air-cooled computer chip technology have slowed. 

“Air cooling is not enough,” said Christian Belady, distinguished engineer and vice president of Microsoft’s data centre advanced development group in Redmond, Washington. “That’s what’s driving us to immersion cooling, where we can directly boil off the surfaces of the chip.” 

Unlike water, the fluid inside the couch-shaped tank involved in Microsoft’s liquid cooling system is harmless to electronic equipment and engineered to boil at 122 degrees Fahrenheit (50 degrees Celsius), 90 degrees Fahrenheit lower than the boiling point of water. 

The boiling effect, which is generated by the work the servers are doing, carries heat away from computer processors, according to Microsoft. The low-temperature boil enables the servers to operate continuously at full power without risk of failure due to overheating. This is the first phase in the two-phase process. 

Inside the tank, the vapour rising from the boiling fluid contacts a cooled condenser in the tank lid, which causes the vapour to change to liquid and rain back onto the immersed servers, creating a closed loop cooling system. This is the second phase of the process.  

Credit: Photo by Gene Twedt for Microsoft.

While liquid microprocessor cooling technology is nothing new, plunging servers headlong into a vat of liquid to keep them cool takes liquid cooling to a new level – although the approach is not entirely novel, or new. 

Cryptocurrency miners pioneered liquid immersion cooling for computing equipment, using it to cool the chips that log the digital currency’s transactions. Indeed, other players besides Microsoft have already started extending the technology out to enterprise data centre use.  

One example is LiquidStack, a pioneer of two-phase immersion cooling systems, with applications from micro data centres to hyperscale sites. LiquidStack has become one of the world’s largest suppliers of liquid cooling systems. 

Building on more than eight years of data centre cooling innovation, LiquidStack was born from cryptocurrency hardware technology developer Bitfury Group’s liquid cooling subsidiary, previously known as Allied Control Limited (ACL). 

Earlier this year, the company announced a US$10M Series A investment, having restructured as a new commercial operating company headquartered in the Netherlands, with its commercial headquarters in the USA and R&D located in Hong Kong. 

In conjunction with the capital raising, LiquidStack struck a strategic partnership with Wiwynn, a cloud IT infrastructure provider of server and storage system design, manufacturing, and rack integration for data centres. 

The primary benefit of the two-phase immersion cooling technology is the reduction in power needed to keep servers cool. 

As recently reported by Network World, in 2014, LiquidStack (then known as ACL) built a 500kW data center in Hong Kong using its two-phase cooling, saving more than 95 per cent on energy for cooling when compared to air cooling.  

LiquidStack said it was able to run like this continuously for eight months without needing to add more coolant. 

In Microsoft’s case, early investigations by the vendor into liquid immersion as a cooling solution for high-performance computing applications such as artificial intelligence (AI) revealed that two-phase immersion cooling reduced power consumption for any given server by 5 per cent to 15 per cent. 

Microsoft subsequently worked Wiwynn, as did LiquidStack before it, to develop a two-phase immersion cooling solution. The vendor’s first solution based on this technology is now running at Microsoft’s data centre in Quincy, Washington.  

Microsoft’s two-phase immersion cooling tank at the facility is filled with an engineered fluid from 3M, the same class of fluid used by LiquidStack. 3M’s liquid cooling fluids have dielectric properties that make them effective insulators, allowing the servers to operate normally while fully immersed in the fluid. 

The shift to two-phase liquid immersion cooling enables increased flexibility for the efficient management of cloud resources, according to Marcus Fontoura, a technical fellow and corporate vice president at Microsoft and chief architect of Azure compute. 

For now, Microsoft has one tank running workloads in its hyperscale data centre in Quincy. For the next several months, the Microsoft team will perform a series of tests to prove the viability of the tank and the technology. 

“If done right, two-phase immersion cooling will attain all our cost, reliability and performance requirements simultaneously with essentially a fraction of the energy spend compared to air cooling,” said Ioannis Manousakis, a principal software engineer with Azure.