How to Install Server Room and Rack Level Monitoring Systems

There are several good reasons for monitoring a server room or datacentre for temperature and humidity, but should you place sensors and detectors at the server rack or room level and just how many sensors is the right number?

Room Level Monitoring

Almost everyone is familiar with how a room thermostat works; you set a temperature and the heating or cooling system adjusts to the pre-set threshold. This type of arrangement uses a single control loop and is classed as one that is cybernetic having self-veto and self-control. The sensor in this case is thermostat.

At the room level, we are concerned with monitoring the overall ambient temperature and humidity levels. Within an IT facility, temperature and humidity are interrelated as an air conditioner will remove moisture from the surrounding air as it cools. If the humidity is too low, the potential for electrostatic discharge can build around electronic devices and electrical systems. If the humidity is too high, then condensation can form leading to corrosion and potential short-circuit.

Server Room or Datacentre Environment Temperatures

Within any IT facility the amount of heat placed onto the cooling system will be a direct result of the number of servers and their power rating, and the energy efficiency of critical infrastructure systems within the room i.e. air conditioners and uninterruptible power supplies.

The commonly accepted temperature range for a functional IT space, server room or data hall is 18-27degC (64-80degF) but even this can be too high. Whilst most electronic devices, including servers and network peripherals can work up to 40degC, and often without derating, the space must also accommodate people within it, whether IT engineers, contractors or visitors and be suitable for all the components within the critical infrastructure systems.

The most used battery within UPS systems are of the sealed lead acid maintenance free type. This battery has a 5-year or 10-year design life when operated within an ambient temperature of 20-25degC with replacement in years 3-4 or 7-8 respectively. If the ambient is higher the design life and therefore the working life reduces. The sensitivity to temperature of lead acid batteries is that for every 1 degree above 30, the design life halves. Lithium-ion batteries are far less temperature sensitive but have a price premium over lead acid.

If UPS systems and their batteries installed within the IT/white space, then the recommended ambient range is 18-25degC. The recommended relative humidity level for this temperature range is 40-60%.

Sensor and Detector Deployment

Continuous operation of the critical cooling system is of paramount importance to most server rooms and datacentre environments. If the cooling system fails, then server rack temperatures can quickly rise leading to a potential fire risk, and especially if the server rack has a high-power consumption, resulting from high density servers and/or the amount of kit within the rack.

The most obvious point to install a room level temperature sensor is therefore near to the local air conditioner(s). The sensor will provide temperature readings to the monitoring device, which will issue an alert warning if one or more of the readings are above a threshold level. Any sudden rise in temperature will therefore provide a ‘proxy’ indicator of a potential air conditioning unit failure.

Airflow sensors can also be used where there are more than two or more air conditioners in operation. Again any sudden change in airflow measurement can trigger an alarm to be raised.

Temperature sensors can also be placed within known ‘hot-zones’. These can occur in larger rooms due to the room construction and server rack and equipment layout. If there is a large UPS battery set, a sensor can be placed close to this to help to prevent heat damage to the batteries. Some UPS system manufacturers also supply additional battery temperature probes for their systems.

As discussed, temperature and humidity within an air-conditioned environment are related. Whilst a temperature sensor may be enough for room level monitoring, a combined temperature and humidity sensor will provide additional measurements and potentially control of the environment.

If a hot/cold aisle containment is used then a combination of temperature, humidity and airflow could be used to ensure that each ‘zone’ maintains levels within accepted specification.

Grey Spaces and Plant Rooms

Aside form the IT or white space whether the servers are operated, there may also be other rooms that require monitoring within a datacentre or around a server room. These can include administration and test areas (for technicians) or plant rooms housing cooling chillers and pumps or centralised uninterruptible power supplies.

Rack Level Monitoring

At the server rack or data cabinet level we are concerned with the temperature into, within and outside the cabinets. Airflow and humidity may also be important but rapidly rising temperatures are a sure sign that there is a potential problem.

Most servers and other systems within a server rack will rely on fans and heatsinks for their cooling. Their design is based on internal fans drawing in cooled air via their front panels and this passing through the units and exhausted through rear panel grills and case perforations. The harder the fans must work; the more energy is being used to increase the volume of air passing through. Some units have fans fitted to support ‘A’ and ‘B’ power supplies and to offer redundancy in case of fan failure.

Rack Level Temperatures

Temperatures within racks and cabinets will also vary. Warm are rises; cool air falls. A top of rack temperature can be 5-10degC or more, higher than the bottom. Front and rear temperatures will also be different and especially so in hot/cold aisle arrangements. Rack layout configuration is therefore an important decision as kit is loaded into the rack. UPS systems and their batteries ideally need to be located near to the bottom and not just because of their weight but the potential sensitivity of their batteries to rising heat.

If a room has two or more sever racks or data cabinets, then rack level monitoring should be considered. The ASHRAE recommendation is to install six temperature sensors (3×2) within a server rack at the bottom, middle and rear (3) and front and rear (2).

The front intake of the rack should be within the room level temperature range of 18-25degC and this will rise as the air passes through the rack and at the different levels the sensors are fitted and should be no more than 20degC higher at the outlet from the rack.

If there is a room level air conditioning failure, then the air-intake temperature into the server rack could rise relatively quickly. Monitoring at the three rack levels helps to identify potential ‘hot-spots’ and ensure that there is good flow of cool air within the server rack or data cabinet, which could be up to 47U (2m) in height.

More information: https://www.ashrae.org/ or https://searchdatacenter.techtarget.com/tip/Use-ASHRAE-guidelines-to-get-server-temperature-right

Responding to Temperature Alarms

Sudden, high temperature rises can indicate a cooling system failure and lead to a potential fire risk within a server rack or cabinet. In this instance an immediate response is required to protect the rack and facility. The air conditioning or cooling system may also be remotely monitored via the local network or a third-party data centre infrastructure management (DCIM) package or monitoring service.

However, higher than normal temperatures or ‘hot-spots’ within a server rack can be missed and lead to erratic operation, CPU level processing errors and server shutdowns. Most server power supplies will have a thermal cut-out to prevent them operating for a long period outside of their temperature specifications. For this reason even a short-term rise in temperature readings should be investigated. Corrective or preventative actions may be required including replacement of a failed cooling fan or rearrangement of servers, peripherals and even the UPS system within the rack.

Water Leakage and Flood Monitoring

Water leakage and flood monitoring can provide additional insights into the operation of a server room or datacentre and provide additional peace of mind. Here we are looking at using water leakage sensors, either ‘spot’ or ‘ribbon’ probes to detect water leaks, floods and even condensation. Ribbons use a conductive material to sense liquid levels and can be sensitive to even small amounts of moisture. After an alarm condition the leakage sensors can typically be cleaned with water and dried for re-use.

The degree to which you want to use this type of room or rack level monitoring depends on the location of your server room or datacentre and the type of cooling system installed. Water can build-up as condensation as cool air with a high-water content meets warmer materials including metal cabinets can ceiling tiles.

Air conditioners and their pipework can leak water or whatever cooling fluid is used. This can be catastrophic if overhead air conditioners and/or supporting pipework is installed directly above a row of server racks. Any spillage or drops immediately falls onto the racks and into the contained devices.

Ceiling voids and even raised access floors can mask condensation build-up or a slow leak into these areas. Eventually the suspended ceiling ruptures due to the weight of the water/liquid or a short-circuit occurs in the plenum area.

The ideal place to install water leakage sensors is again under air conditioners and their pipework, ceiling voids and under floor plenums (where raised access floors are used) and around perimeter walls.

Additional Sensors and Detectors

Environment monitors tend to allow for a fixed number of sensors for them to be installed with. Temperature sensors may be built-in or available as additional plug-in sensors or probes. The plug-in sensor model may also be used for humidity and water leakage and the device may also be compatible with several other plug-in sensors and detectors that can extend its monitoring functions. Additions can allow for smoke and fire detection, motion sensor and rack or room level security. Custom build sensor options may also be offered for extreme and harsh environments. In addition there may be several digital inputs and outputs to further customise the monitoring device for the site installation.

Fixed Ethernet Wiring and WiFi Installations

Most environment monitor ranges include devices with standard Ethernet connection to a local IP network or a WiFi facility or both. WiFi installation can reduce cabling installation costs but may not be suitable for some datacentre and server room installations.

Alert and Alarm Messaging

A final consideration is how alerts and alarm messages are to be transmitted and to whom. Choices can include email, SNMP, phone number calls and SMS messaging and the use of local setup and monitoring or Cloud-based software. Each environment monitor range should offer a common set of software solutions and support for usage.

Summary

Most critical infrastructure systems within a server room or datacentre will offer temperature and potentially humidity level monitoring. This may be built-in as in the case of an air conditioner or offered an optional sensor or probe that can be installed alongside an uninterruptible power supply or power distribution unit.

Whilst these will provide some indication as to the temperature and humidity in their vicinity they do not provide as comprehensive a view as can be achieved with a dedicated environment monitor system. Not only can the latter approach better protect an installation, but it may be used to help improve energy efficiency and air flow within server room and datacentre environments.