Liquid Cooling Technology for Data Centres

Cooling technology is evolving to meet the growing demands of high-performance computing in data centres and HVAC Systems. With the increasing demand for data processing and cooling driven by the expansion of large-scale facilities, liquid cooling technology is emerging as the solution to meet precise thermal demand for existing and future facilities.

Traditional air-cooling methods struggle to keep up with the intense heat generated by these cutting-edge applications, leading to a shift towards more efficient liquid cooling systems, offering improved thermal conductivity.

Traditional air-cooling strategies are struggling to keep up with increased thermal loads caused by AI, HPC, and crypto mining.

Traditional data centre design involves removing heat using cool air. Server racks and computer room air handler (CRAH) units are positioned to create “hot aisles” and “cold aisles.” The CRAH units provide cool air to the cold aisle by utilising a raised floor supply air plenum. Fans draw the air across the servers, which transfers the heat from the servers to the air and then dumps this hot air into the hot aisle.

This method has been successfully implemented for decades, even as heat densities have consistently grown. But now, with the exponential rise of generative AI, HPC, and other hyper-intensive computing applications, the industry has reached a critical threshold. As recently as a few years ago, the average heat output from a rack of typical servers was somewhere between 5-10 KW and as high as 12 KW in some edge cases. However, as the tidal wave of new applications has rolled in, heat densities have skyrocketed to as much as 50-100 KW per rack. The tried-and-true cooling techniques of the past cannot keep up with these extreme loads. Liquid cooling is widely believed to be the gold standard of the future.

Liquid cooling can take several forms, including, but not limited to, rear door heat exchangers, immersion cooling, and direct-to-chip cooling (sometimes called cold plate cooling). Each of these applications presents its unique flow control challenges.

Belimo has shown a keen interest in the data centre space, developing innovative solutions that simplify and optimize the transition to liquid cooling and support its ongoing operations. By integrating measurement technologies and advanced computational capabilities into standard mechanical valves, Belimo has spearheaded the advancement towards smarter valves, offering functionalities far beyond basic position control.

Traditionally, control valves have relied on position control technology, which correlates the control signal with the valve’s opening geometry. This basic technology determines how much of the cross-sectional area is exposed to allow flow through the valve. Analogous to controlling airflow by adjusting the opening of a window, this method can be challenging. Since flow depends on the exposed cross-sectional area and pressure differentials, precise control can be difficult.

Flow control, a more advanced technology, refers to the behaviour of pressure-independent valves. Unlike traditional methods that control the valve’s opening and rely on proper pressure to achieve the desired flow, flow control valves directly regulate the flow rate, often disregarding pressure and valve position altogether. Electronic pressure-independent (PI) valves are well-suited for the liquid cooling industry. An upcoming blog post will discuss further details on electronic PI valve solutions for liquid cooling applications.

Power control builds upon flow control by measuring flow and the temperature differential (delta T) across a heat transfer device. Using this data, the valve can interpret a control signal from the Building Automation System (BAS) as a power requirement. Based on the measured delta T of the water, it can modulate the flow to exchange the specified amount of thermal energy, whether expressed in BTUs per hour (BTUh), tons, kilowatts (KW), or other units.

Differential pressure control is a fundamentally different approach. According to physics, flow capacity (Cv) and flow rate are directly proportional, rising and falling simultaneously. By maintaining a fixed differential pressure (DP) across a circuit, a valve can adjust the flow rate to match changes in flow requirements. If components are removed from a circuit, such as when a server is removed from the rack for maintenance, the valve will reduce the flow accordingly to maintain the fixed DP. This type of control is particularly relevant for liquid cold plate applications, which will be explored in detail in a future post.

Effects of proper flow across cold plates

Maintaining proper flow across cold plates is essential to ensuring efficient cooling, system reliability, and the longevity of both the cold plates and the components they cool.

Why is proper flow across a cold plate important?

• Server Damage: Too little flow can lead to inadequate cooling on the server, risking potential damage that, if severe enough, could necessitate a replacement costing between $100,000 and $250,000 for high-performance models.
• Unplanned Downtime: Server freezes or unexpected shutdowns can result in significant financial losses. According to Uptime Institute’s 2022 survey, 70% of outages cost $100,000 or more, with 25% exceeding 1 million USD*.
• Excessive Flow Risks: Too much flow can erode cold plate channels, altering heat transfer properties and necessitating replacement. Additionally, excessive flow increases pumping energy consumption.

Challenges to achieving accurate flow across cold plates

• Dynamic Environment: Regular server removals for maintenance change the total flow requirement for the rack, potentially causing overflow to the remaining servers.
• Flow Distribution: Even with managed total flow, ensuring equal distribution to each cold plate proves challenging.
• Variable Server Replacements: Server upgrades or replacements may alter flow requirements, complicating overall flow management.
•  

Using differential pressure to maintain constant flow

To determine the total flow required to cool the entire rack, sum the flow requirements of each cold plate. If the total flow to the rack is controlled and a server is removed for maintenance, this would cause an overflow to the remaining cold plates.

In contrast, maintaining a constant differential pressure across the rack ensures a consistent flow to each cold plate, even if a server is removed.

Comparison of Flow Control (Left) and Pressure control (Right) schematics.

*Uptime Institute. 2022. Uptime Institute Global Data Center Survey 2022. https://uptimeinstitute.com/resources/research-and-reports/uptime-institute-global-data-center-survey-results-2022

Expertise shared by-
Vishal Jadhav
BDM, Data Centre
Belimo Automation India Pvt. Ltd.