How Server Rack Power Densities Impact Critical Power Protection Plans

Server rack densities are increasing and have quadrupled over the last 9 years from 2011 to 2020. From 2.4kW/rack to 8.4kW/rack according to the latest survey by the Uptime Institute. What affect are these increasing demands having on server rack deployment, power protection and cooling system arrangements and what is the primary cause of data centre outages.

What is Server Rack Power Density?

Server rack power density is the total power demand of a server rack and is usually measured in kilowatts (kW). Every item that requires electrical power within the server rack should be listed and its power calculated.

Calculating power demands is not always easy. If you look at the back of a server, it will show the rating of the power supply unit (typically 500W or 0.5kW). This is the total power the PSU can deliver. Most likely it will never be reached. Watts or Kilowatts measures the Real Power drawn by a device. Other devices may only have an Amperage (A) rating, and this means the Apparent Power can only be measured for this item by multiplying the Amps x the electrical supply Voltage (V) to give a VA or kVA rating. For most modern devices we can assume VA=Watts or kVA=kW due to what is known as the Power Factor of the device.

Calculating server rack power densities this way provides an estimation of the total potential power demand. For more applications the figure may be summed and then halved to provide an indicative figure.

For more accurate measurements, a server room power audit can be undertaken. Here a device is used to measure the exact power draw of each device to provide a more accurate calculation. If a server rack is fitted with metered or intelligent PDUs, their power and energy usage measurements (either as a total PDU or per outlet) can be used.

More information on server rack densities and power outages:
https://journal.uptimeinstitute.com/uptime-institute-10th-annual-data-center/

Server Cabinets

The Uptime Institute survey provides evidence that higher rack power densities are becoming the norm, with 8.4kW/rack being the average reported for 2020. At this power rating, the input supply to the server cabinet will typically be single phase (230Vac, 50Hz in the UK). The increase in power rack density, also impacts the size of cabinets generally deployed in terms of their U height.

All server racks have a height measured in ‘U’ where 1U = 44.45mm. This standardised way of measuring heights in server cabinets, make it relatively easy to calculate the space required for equipment to be installed into the cabinet.
Rack mount uninterruptible power supplies are no exception and almost all UPS system manufacturers offer UPS form factors specifically for use in 19inch rack mount server cabinets. 19inch rack mount is an alternative form factor to tower format, typically of a free-standing UPS.

Rack power density increases affect UPS systems in two ways. Firstly, the kVA or kW of the UPS to be installed and the amount of U-height they take up in a server cabinet.

At the 2.4kW/rack average in 2011, the average UPS size will have been up to 3kVA/3kW. At 8.4kW/rack the average UPS will now be around 10kVA/10kW. Whilst there may be little difference in the front-to-back measurements, a larger 10kVA UPS will require additional U-height.

A typical 3kVA UPS would require a U-height of around 3U and its batteries could be internal to the UPS cabinet. A 10kVA rack mount UPS could take up to 5-6U in height and would require an additional space for its battery tray. There is generally no room in a 10kVA for the batteries.

With the increase in UPS size also comes a greater weight. A typical server cabinet can take from 150-300Kg of equipment depending on whether it is a standard or heavy-duty cabinet. The larger the UPS, the larger the battery and the greater the weight load on the server cabinet.

The Uptime institute survey shows that 8.4kW/rack is an average reported by up to 46% of respondents. 25% of respondents were in the 1 to 4kW range, 13% in the 10-19kW and 16% in the 20kW and above category (20-29kW, 30-39kW and 40kW and above).

Whilst a 1-4kW rack power density will require a single-phase UPS solution, at 10kW and above a three phase may be required, either installed as a 3/1 or 3/3 UPS configuration i.e. three phase input/single phase output (3/1) or three phase input and output (3/3). A further complication for planning power within a server cabinet is how best to arrange power distribution units (PDUs), and especially if the servers have A and B supplies.

Cooling Demands

The higher kW demands of the equipment within server cabinets, increases the cooling load and the need for resilience. Cooling is as critical as power in a server room or data centre. The greater the IT kW load, the greater the cooling load.

Most server rooms and data centres will deploy an N+X cooling system with two (N+1) or more air conditioners operating in a cyclical manner. Should one of the AC units fail, the other can support the load.
This level of resilience is important if sudden temperature rises are to be avoid. It does not take long, without cooling for an 8.4kW rack power density to approach a level of heat build-up which can lead to a meltdown and fire.

Environmental Monitoring

In addition to the critical power and cooling systems installed in a server room or data centre environment, the minimum environmental factor to monitor is temperature. Humidity and water leakage can be other factors to monitor.
Temperature monitoring is relatively easy to deploy within an IT environment. The system relies on a base unit to which a temperature sensor (and others) may be connected. The environment monitor may be mains or PoE powered and installed inside a server cabinet as a rack mount or shelf mounted device. Alarm notifications can be set up for analog readings that go outside of a safe range, and email and text alerts issued when this happens to distribution lists.

Server Room and Data Centre Outages

The report by the Uptime Institute also covers power outages. 78% of server room and data centres suffered an IT outage over the last 3 years and this is 50% up on previous surveys. The principle causes of outages being on-site power problems (37%), software or IT systems errors (22%), network issues (17%) and cooling problems (13%).

Power problems are the biggest cause of data centre power outages and for this reason it is important to make sure that there is resilience in the complete critical power path and that every device is powered from an uninterruptible power source. The Tier-rating by the Uptime Institute for data centres provides a way to classify the resilience and maintainability of a data centre with Tier IV offering the highest and Tier 1 the lowest. It is more than likely that the data centres who experienced power related outages were probably only Tier-1 or Tier-II rated.

More information on the Uptime Institute’s Tier-rating system:
https://journal.uptimeinstitute.com/explaining-uptime-institutes-tier-classification-system/

More information on a recent IT related internet outage caused by a software upgrade:
https://www.bbc.co.uk/news/technology-57929544

Summary

The continued increase in rack power densities and significance of power related data centre outages, highlights the need for 19inch rack mount UPS systems. Every device within a server cabinet should be powered from an uninterruptible power source to prevent downtime and where possible, N+X power sources should be considered.

As with any project, it is important to list all the devices to be powered and the power connections they require. This will help to size the UPS and plan cable routes and outlet connections, whether this is via the UPS sockets or power distribution units.

Adding environment monitoring to data centre server cabinets provides additional protect. If there is a sudden rise in the rack temperatures, due to a cooling system failure, corrective action can be taken to reduce the risk of a fire.

Where it is not easy to determine the size of the UPS required, a server room or server rack power audit can be arranged to accurately assess the loads in use. This can be easily arranged by contacting our projects team.