A tour of our cooling systems

The Data Center Secondary Chilled Water Loop

The data center has a pipe loop containing chilled water that flows through a Plate Frame Heat Exchanger (PFHX) as well as via thermostatically controlled butterfly valves to dry coolers on a nearby roof.  This is called the secondary loop as the primary loop is one of several campus-wide chilled water loops that provides approximately 43-45 degree water to the PFHX.  The size of our PFHX supports about 100-120 tons of heat transfer capacity.

The campus chilled water system is over 1000(?) tons and provides chilled water to most buildings on campus.  It uses evaporative cooling towers and can operate either electrically or under steam power.  Steam power is used when electrical demand load reductions are needed.

CRACs

We have 6 Liebert Computer Room Air Conditioners (CRAC) of a recent vintage (2006) which are connected to the secondary chilled water loop.  Warm air is sucked in the top, flows over cooling coils, and is pumped out the bottom, under the raised floor.  These units have two modes of operation which is accomplished with a three-way water valve in each unit.  The valve allows automated switching between water-side economy mode and direct expansion (DX) cooling.

Water-side economizer

The water-side economizer mode flows room air over a coil that is directly connected to the secondary loop, which nominally operates at about 45 degrees in and maybe 55 degrees out.  This is the most efficient operating mode as no compressors are used.  Each CRAC has about 15 tons of cooling capacity when in this mode.  If this was the only way these units worked, they would technically be called CRAHs (for Computer Room Air Handlers).

DX

Each CRAC unit also has a refrigerant-based direct expansion (DX) cooling coil and a compressor.  This has a coil that exchanges heat with air flowing over it and in turn compresses the refrigerant.  Exhaust heat is exchanged via a heat exchanger to the secondary chilled water loop.  The DX mode only operates when the available cooling from the water-side economizer is insufficient, either because there is more than 15 tons of load from the warm air entering the CRAC or because the chilled water temperate is too high to remove sufficient heat.  When the DX mode operates, a significant electrical load is incurred to run the compressors.  The DX mode gets the Lieberts up to their full rated capacity of about 20 tons.  When operating in DX mode, the chilled water temperature can potentially be much higher than the usual 45 degrees — as much as 100 or more degrees.

Backup Cooling with Dry Coolers

In the event that the water temperature of the primary chilled water plant rises above about 70(?) degrees (e.g. due to a central chilled water plant shutdown), thermostatically controlled butterfly valves divert the secondary chilled water around the PFHX and up to the rooftoop dry coolers, which are essentially large radiators with fans that blow outdoor ambient air across them.  In this operating mode, the secondary chilled water can get to 100 degrees or more and is cooled down to near the outdoor ambient air temperature.  When this happens, the CRACs must run in DX mode since their cooling water is too hot for water-side economizing.

One Chilled Water Loop.  Two Modes of Operation.

Due to the use of backup dry coolers vs. the central chilled water plant, the secondary loop temperature which normally operates at 45 degrees can sometimes operate at 100 degrees.  This dichotomy makes it difficult to employ a number of water cooling technologies for in-row or rack-based cooling which generally want to “see” water in the 45-55 degree range (some vendors can push the limit to about 70 degrees).  None of the products we are currently aware of have similar capacity to the CRACs to operate in these two temperature ranges.  This makes it difficult to employ in-row or in-rack cooling for critical services that need to keep running when the central chilled water plant is shut down.

Air Handlers

In addition to the secondary loop feeding the 6 CRACs on the data center floor (and three additional CRACs when the floor was larger before part of it was turned into office space), there are two Air Handling Units (AHUs) in the basement mechanical room.  These units are directly connect to the primary chilled water loop and have large fans and ductwork that pump air to- and from the data center floor and the adjacent offices in the Computer Center building.  Each of the two AHUs has about 10 tons capacity.  The AHUs also bring in outside fresh air.  Because these air handlers lack compressors (and they are on the primary chilled water loop) they do not have an emergency backup when the central chilled water plant is down.

Air-side economizer

From the late Fall through early Spring, Facilities HVAC mechanics switch these AHUs into air-side economizer mode in which they essentially pump cold outdoor air (through filters, of course) into the data center.  This is sometimes referred to as free cooling, although there is of course a fan energy cost.