How to Design a Secure and Resilient Server Room Power Supply

Power supply design is one of the key decisions that must be taken whenever building a new server room or refurbishing and existing one. There are several elements to consider even for a single stack rack cabinet if you are to ensure that your server installation can run through power outages.

When we refer to the critical power path in a datacentre, we are looking at the building electrical supply running from the building incomer to the LV switchboard and sub-distribution panels, to the uninterruptible power supplies and power distribution units (PDUs) to which critical IT loads are connected.

Designing for Resilience

Resilience is the ability of an installation to be available and running regardless of a fault condition. When designing a power supply system it is important to consider any single points of failure and to remove these through adding in some form of redundancy. In a perfect world every system and component within the critical power path would have at least N+1 redundancy but this is not always possible due to several reasons including initial capital expenditure (CAPEX) and operating expenditure (OPEX) costs including ongoing service and maintenance.

The building incomer is supplied power from a local substation. The building incomer is connected to an LV switchboard from the which the circuits within a building are supplied. In high-uptime datacentres, there is often reference to A and B supplies. A Tier-4 type datacentre facility as defined by the Uptime Institute will include several layers of redundancy including A and B mains power supplies drawn from separate substation transformers.

For most server rooms this is not an option. Resilience starts with the UPS system and how this is supported.

UPS System Installation

An uninterruptible power supply for a server room environment can be installed as a centralised or decentralised configuration. In a centralised installation the UPS system supplies the complete server room via a sub-distribution panel to which all the critical electrical circuits are connected. In a decentralised or distributed power protection plan, separate UPS systems are used to protect individual circuits or systems within the server room space.

There are two ways to incorporate additional resilience into the design. The first is to use a parallel N+X redundant configuration. This can be achieved by having two or more UPS systems installed and run in parallel. In the configuration ‘x’ refers to the number of additional UPS systems installed. In a two system or module installation we have an N+1 parallel/redundant configuration.

The two mono bock UPS systems share a communications path via parallel cards installed into each. The UPS firmware co-ordinates their operation so that they either share the load via common output or one acts as a master and the other a slave arrangement. If the master UPS supports the load the slave system is ready to take over should the primary UPS experience an overload or fault condition.

Modular UPS systems

Modular uninterruptible power supplies are designed to be installed as an N+X configuration by default. Mono block UPS systems can be operated as single standalone UPS. Modular UPS systems incorporate a frame into which the UPS modules are placed to parallel-up the output capacity and/or provide N+X resilience. Most UPS manufacturers offer three-phase modules around 25-50kW and this may be enough for medium to large sized server rooms. An additional feature of this type of arrangement is that the modular UPS frame is similar in size to that of a server rack cabinet. The UPS scales vertically to reduce the demand on local floor space.

Automatic Transfers

There are three types of UPS topologies available including off-line (standby), line interactive and on-line. On-line UPS are the most appropriate type of uninterruptible power supply for a server room as they incorporate an automatic transfer switch as part of their output circuit. This may be a static transfer switch or a relay-based switch.

The automatic bypass switch offers some level of resilience to the installation. A circuit monitors the output waveform of the uninterruptible power supply. If the voltage and current waveforms start to collapse this indicates either an overload on the output or an internal fault. In either of these instances the automatic bypass transfers the output load (typically PDU connected IT servers) to the mains power supply until the overload is removed or the UPS fault cleared.

Automatic Transfer switches (ATS) or Static Transfer Switches (STS) can also be installed before a critical load. An ATS is relay based which means a small break of several milliseconds when switching between one of two power sources (A and B). The two power sources can be a single UPS system and the mains power supply, the outputs from two separate UPS systems or a mixture of the two. An STS is more expensive as it includes solid state switching electronics, but this can be offset by the fact that this type of device has no break when switching between outputs.

Battery Backup and Standby Power Generators

The battery installed with a UPS system will have been installed to deliver a set amount of runtime power for a set time period. UPS batteries will never fully discharge when there is a mains power failure. This is to prevent a ‘deep discharge’ state from which the batteries may not recover. If the mains power supply is restored before the battery cut-off limit is reached, the connected loads will suffer a power outage.

The installation of a local standby power generator can overcome this issue. The generator adds a layer of resilience to the critical power path and redundancy to the battery set. A typical generator has a built-in day tank that can provide up to 8 hours of standby power. When installed with a UPS system, the UPS battery is used only to cover the switch on period of the generator which can take several seconds to speed up to a stable electrical output in voltage and frequency terms. The UPS system synchronises to the generator supply which is then used to power the inverter. The battery is recharged.

Summary

It is possible with a relatively small budget to build resilience into a server room power supply design. The final single point of failure to consider is the PDUs to which the critical servers and IT loads are connected. The PDUs should be smart or intelligent PDUs with built-in overload protection in the form of circuit breaker protection and with individual outlet socket control. For additional security and resilience, the IT servers themselves may be installed with A and B dual power supplies drawn from A and B UPS protected supplies within the critical power path.