The Benefits of Liquid Cooling Solutions Over Air Conditioning Systems

The datacentre industry is under immense pressure to reduce its energy usage and achieve a smaller carbon footprint. One of the biggest users of non-compute energy are the cooling solutions used within hyperscale down to medium and small-sized datacentres.

The traditional air-based cooling systems have always been the ‘go-to’ solution for datacentres and server rooms with liquid cooling used more for mainframes and academic supercomputers. The issue facing many datacentre operators today is that of power density and more demanding workloads. These are forcing attention to more efficient ways to cool the compute environment than can be achieved with an air-based cooling solution.

The design of new datacentres and server room refurbishment provides an ideal opportunity to challenge assumptions when it comes to cooling. Technology tends to flow down from the hyperscale operators and overtime this will occur with liquid cooling through a miniaturisation of the systems and options available to deploy.

Liquid Cooling and Energy Efficiency

Cooling is the most energy consuming element within a critical infrastructure design and it is system where the adoption of liquid cooling can improve energy efficiency and power usage effectiveness (PUE) ratios.

Several of the leading hyperscale cloud operators are collaborating on an open specification for liquid-cooled server rack cabinets with the CEEEDC (Center of Expertise for Energy Efficiency in Data Centers) including Google and Microsoft.

The group expects that over time power densities will continue to increase and will create greater heat management problems within datacentres and server room environments. As the heat load increases, traditional air conditioners will become less efficient and capable of helping to manage the heat load. Datacentres will need larger air-based cooling systems, larger diameter fans and more space for heat collection and dissipation. This will lead to an increase in air noise pollution within the computing environment and a higher thermal impact, both of which will lead to increased CAPEX and OPEX and larger budgets.

More information: https://datacenters.lbl.gov/development-liquid-cooled-rack-specification

It is not surprising that the hyperscale operators are the first to turn towards liquid-based cooling when you consider the evolution of their machine-learning systems.

These datacentres are at the forefront of the power density battle and run powerful GPUs and TPUs. CPU performance is being boosted by accelerator processors which can have a far higher thermal design point (TPD) than CPUs and will need around 200W of cooling or more per unit. When coupled with a higher performance CPU the cooling load can increase to 1000W or 1kW per server.

This is also leading to higher power rack densities with some rising as high as 30kW or higher per rack. Cryptocurrency racks have been configured up to 100kW of power. For these higher power capacities, liquid cooling can be deployed within a containment system or the rack via a liquid-cooled door arrangement.

As the compute density increases, so does the need for storage. This is one area where greater care must be taken with liquid cooling. Most datacentre storage devices use non-sealed hard drives that require air cooling. Solid State Drives (SSD) can however be cooled using liquid immersion. The ‘liquid immersion’ cooling solution can also help to reduce heat and humidity stress on storage electronics thereby improving reliability and working life.

Aside from the cost of electricity and energy efficiency drivers, Climate change is another key driver for change in the datacentre industry. Electricity generated by non-renewable means has a higher carbon footprint. Traditional air conditioners are less energy efficient than liquid cooling systems and may be more challenging and difficult to maintain.

Liquid Cooling at The Edge

As the 5G network rolls out there will be more demand for Edge computing micro data centres. The early 5G systems being rolled out will simply be extensions of the 4G network with the full potential of 5G limited until there is a far wider network of antennae and the necessary telecommunications and electronics.

The higher power density of micro data centres facilities will be well matched to liquid based cooling. This is even more so in areas with limited power capacity as liquid cooling is more energy efficient than traditional air conditioners. Here we are not just thinking about rural areas but edge of towns and cities which tend to be at the outer limits of the electricity supply infrastructure.

For remote Edge applications liquid cooling may also prove more reliable whether these are housed in micro data centres or modular data centres or some other form of higher-IP rated cabinet suitable for a near roadside type application.

Summary

The ‘liquid’ side of the industry is evolving at a rapid rate and there are now several key market suppliers for liquid-cooling systems. The industry segment is expected to grow by a factor of 2-3 times by 2026 with key drivers being the growth of digital traffic and the adoption of new technologies including Edge computing, AI (artificial intelligence) and machine learning. Liquid cooling technology is a superior alternative to air-based systems and can handle higher heat loads with lower noise, environmental impact and operating costs. For some installations whether a new build or refurbishment, the CAPEX costs may be higher initially but overtime the return on investment (ROI) will be lower than a traditional air conditioning arrangement. In the future liquid cooling solutions could become the standard approach for server racks with as little as 500W of cooling load.