How To Cool Server Racks and Data Centre Cabinets

Heat is a form of ‘waste’ energy that results from the way the CPUs (central processing units) within a typical IT server work. In a server room or datacentre environment, managing ‘heat’ and keeping server racks cool is as critical an issue as ensuring the servers are protected from mains power supply failures. Excessive heat can age components and can lead to early system failures, erratic system operation and presents a potential fire risk both within a server cabinet and its local environment.

Servers draw electrical power from their local supply, whether this is a power distribution unit, UPS system or mains power socket via an internal switch mode power supplies (SMPS). The SMPS converts the AC (alternating current) waveform of the supply, into the levels of DC (direct current) required to power internal circuits including the CPU.

A CPU works by enabling electrical signals to pass through its microscopic transistors or by resisting (blocking) them. As the electrical (DC) energy passes through the CPU, heat builds up. The heat is dissipated into the surrounding environment. The more powerful the CPU, the greater the volume of heat generated. The greater the number of servers, the larger the amount of heat dissipated as the load utilisation increases on the servers. Within a confined space, such as a server rack, the heat build-up can be extensive and damaging if not managed and controlled. Whilst more energy efficient servers will use less energy to produce the same level of output, CPU performance continues to increase as do their power demands and potential heat generation. More powerful processors used for machine learning applications have higher power demands and generate even more heat.

For server room and datacentre managers the amount of heat generated and dissipated within a room is rarely uniform and must be channelled and managed in order for an air conditioning or cooling system to operate as effectively and efficiently as possible. Cooling and humidity management is necessary to protect critical systems and maintain their operational resilience.

There are several aspects to consider when deciding on how best to cool an environment. For a server cabinet, the most appropriate level of cooling and ventilation will be dependent upon several factors:

• The number of servers, switches and routers and their heat output
• The heat output of additional systems within the cabinet (i.e. UPS systems)
• The mechanical construction of the cabinet, perforations, size and space utilisation
• The thermal characteristics of the cabinet in terms of ventilation
• Cabinet arrangement and position within the room
• Cabinet location (which could be outside a server room in an extreme environment)

Server rating plates can be misleading when calculating the level of cooling required. The numbers shown generally the maximum power draws in Amps or Watts or Kilowatts. Use of these numbers can lead to an over-sizing whether it’s a cooling system or an uninterruptible power supply. The numbers can therefore only be used as a guide as to the level of cooling required.

A more realised measurement for sizing a cooling system is to calculate the British Thermal Units (BTUs) which can be converted into Kilowatts as 1kW=3,412BTUs. The BTUs should be calculated for:

• Equipment: the IT servers, switches and routers
• Floor: the total floor area of the room
• Windows: the number and their location
• Occupants: the number of people working within the room
• Lighting: the heat generated from room lighting

Once all the factors have been assessed the total room BTU can be calculated:

Total Heat Gain = Equipment + Floor + Window + Occupant + Lighting

There are individual formulas for each of these types of calculation:

• Equipment BTU = total watts x 3.5
• Floor BTU = width x length
• Windows BTU = 0 if there are no windows or if there are you need to factor in the number and whether they are north or south facing. The Windows BTU = South Windows (BTU) plus North Windows (BTU). For each window the following calculation applies:
• South Window BTU = South Facing windows Length (m) x Width (m) x 870
• North Window BTU = North Facing windows Length (m) x Width (m) x 165
• Occupants = total number of occupants x 400
• Lighting = total watts x 4.25

The calculation will provide a good indication of the size of air conditioner or cooling system required for a small server room. Larger facilities and datacentres may also require thermodynamic modelling and the use of hot-aisle/cold-aisle containment arrangements.

Server Rack Cabinet Cooling Options

Adequate air flow is vital to ensuring that operational reliability if the critical servers inside a cabinet. The cooling and air flow must be good enough to prevent a build-up of heat (hot-spots) and to ensure that all components within the rack are suitably cooled no matter where they are placed (bottom, middle or top of the rack cabinet). This can be even more important consideration if UPS systems with valve regulated lead acid (VRLA) batteries are installed into the rack as the batteries need an ambient temperature of 20-25˚C in order to stand a chance of reaching their optimal working life.

So how can you ensure a good air flow and what do you need to consider?

• Cabinet Front and Back doors: the majority of rack cabinet manufacturers recommend that front and back door with an open-air flow greater than 60%. Air is typically drawn in from the front door and expelled via the rear door. Doors should therefore be perforated or in extreme conditions removed. Solid side panels are recommended to channel the air-flow front to back and prevent heat exchange between adjacent cabinets. In server rooms and data centres with adequately sized air conditioning, the cooling fans within the IT servers and other equipment may be sufficient enough to prevent a heat build-up and provide adequate air flow.
• Cabinet Utilisation: a good rule of thumb is to only use 80% capacity as a maximum whether this loading on a UPS system or the amount of space used within a server cabinet. Where space within the cabinet allows, leave a 1U space between shelves and rows of systems and at least 5-10mm of clearance between the front and back of the cabinets to allow air-flow around the equipment. Blanking panels should be installed on the front to improve cool air-flow efficiency from front to rear.
• Temperature Monitoring: there are several ways to monitor temperature and issue alerts and alarms within a server cabinet. The most common way is to use a thermometer, and this may be an additional accessories that can be connected to an smart or intelligent power distribution unit (PDU) or a standalone temperature, humidity and water leakage monitoring system. Either approach should generate an alarm if the temperature rises above a trigger point which should be audible, visual and electronic in the form of an email, text or SNMP-based alert. It may be necessary to have one or more temperature monitoring points within a larger cabinet. Many IT systems have internal temperature sensors and alarms from these should be captured and acted upon.

Where additional cooling is required the choice then turns to what type of cooling method to use and these range from natural convection, to forced convection, active convection and free cooling.

Natural Convection Cooling

This form of cooling relies on the fact that heat will flow from a warmer environment to a cooler one. If the air around a server rack cabinet is cooler than the internal temperature, the heat within the cabinet will radiate naturally through the sides and doors and the internal temperature will reduce accordingly. Whilst this is a ‘free’ form of cooling it is not efficient, and its effectiveness is dependent upon the external temperature of the air outside the server cabinet. If the differential is not larger enough, little heat will flow from inside to outside the cabinet.

Forced Convection Cooling

A fan or blower fitted insider server rack, can force the air flow and decrease the thermal resistance barrier between the cabinet and its environment. The fans can be installed as top of cabinet assemblies or internal fan trays. The most and effective type of installation are top-mounted fan panels. Internal fan trays are generally used to tackle local hot-spots. Circulation volumes can be increased by adding more fans but a problem with this type of arrangement is air quality and the outside temperature. If the air is contaminated with dust, dirt, oil or moisture, then the fan air flow will lead to these building up on the fans and within the internal rack assembles. Where this is an issue, as in some industrial environments, a closed-loop air-to-air heat exchanger can provide a solution. As with natural convection, the overall flow of heat is dependent on the difference between the internal and external ambient temperatures.

Active Convection Cooling: Air Based

Air conditioning is a form of active convection cooling and the one most turned to when natural or convection cooling proves ineffective. There are several types of air conditioner available for server rooms and data centres including wall mounted and ceiling suspended, in-row units and complete computer room air conditioners referred to as CRACs.

Rather than cool a complete room it is also possible to build an air conditioning unit into a server cabinet. This configuration forms a closed-loop system (with an internal evaporator) and can be ideal for industrial or extreme environments where there is a danger of contamination from dirt and dust or moisture, water and other liquids. Closed-loop systems like this have a cost benefit in that they are more efficient as they are focused on cooling a single cabinet rather than an entire array of rows or a complete server room. For this type of cooling arrangement, it is vital that the air conditioner is accurately sized.

For extreme environments the rack cabinet will also have an IP-rating higher than IP20 making it even more necessary for an air conditioner to be fitted to protect the heat-sensitive equipment inside. The cabinet may also be a custom or bespoke design that is too a high degree sealed to allow operation in ambient temperatures up to 55˚C.

Active Cooling: Liquid Based

Liquid cooling is another form of forced active cooling. This type of system used a chilled liquid to cool the air within a cabinet, hot-aisle/cold-aisle arrangement or a complete data facility. Liquid cooled doors can be fitted to some server rack cabinets to provide an individual and custom solution. Liquid leakages are less of an issue with more modern systems thanks to self-healing systems.

Free Air Cooling Solutions

Free air cooling is a method for cooling large server rooms or data centres and can be a suitable solution for sites with low external air temperatures. For this type of arrangement the low external temperatures are used to chill water which is then stored or used immediately for use with an internal air conditioning system. The solution is more suited to larger facilities rather than individual server racks and cabinets and can rely on adiabatic cooling processes.

There are several options available when it comes to cooling server rack cabinets and data centre facilities. There is no one solution that fits all installations. Some cooling systems and air conditioners can be installed relatively easily, and others may have a more complex installation and even require custom or bespoke elements.

At Server Room Environments we supply, install and maintain a wide range of air conditioning solutions. Please contact us for a free site survey or to discuss your project, whether its an existing system upgrade or new cooling system installation.

For information on recommended ambient temperatures for server rooms and data centres refer to the ASHRAE white paper ‘Data Center Power Equipment Thermal Guidelines and Best Practices’ (http://tc0909.ashraetcs.org/documents/ashrae_tc0909_power_white_paper_22_june_2016_revised.pdf).