When AI raises the heat: The impact on data centre cooling

The capability and uptake of generative AI is growing fast and data centre cooling is becoming a critical risk as AI workloads drive much greater heat loads. Find out more about the key issues and practical risk controls.

AI data processing workloads are far more power-dense than traditional IT workloads and much of that power ends up as heat. If that heat is not removed quickly and reliably, server performance is affected. Data centre cooling is therefore no longer a secondary consideration. It is as critical to operational resilience as power and fire protection, and is changing the risk profile associated with these facilities. We are pleased to share some technical insights that we hope are useful.

GPU dense hyperscale computing environments mean that traditional air-cooling systems are approaching their thermal limits. In the early 2000s, rack densities were typically around 3kW per rack. These are now approaching the 50-100 kW range in some hyperscale environments with some reports indicating that next-generation AI deployments may push beyond 250 kW per rack. That's a fundamentally different environment and one where a cooling failure can trigger thermal shutdowns, GPU throttling, and significant business interruption within minutes.

We've outlined the four main cooling system architectures we're seeing in the market today, along with the key risks we consider.

This is still the most common cooling approach in datacentre facilities, using either direct expansion Computer Room Air Conditioning (CRAC) or chilled water-based Computer Room Air Handling Units (CRAH). In either case cool air is directed toward rack air intake areas. This cooled air provides server cooling with the hot server exhaust air then returning to the unit to be cooled again.

Key risks we consider:

• Compressor and pump bearing failures
• Cooling tower fan motor failures and heat exchanger fouling
• Pipe leakage near energised equipment
• Refrigerant leakage
• Condensation from inadequate insulation
• Joint and gasket failures
   

Recommended good practice includes: 

• N+1 or 2N cooling equipment redundancy
• Air-handling equipment and chilled water piping that are independent from those connected to other building spaces.
• Leak detection systems
• Predictive vibration monitoring
• Water quality management
• Thermal imaging inspections
• Emergency power for HVAC systems
• Physical segregation of pipework from critical electrical infrastructure
• Preventive maintenance of mechanical components

This is a fast growing cooling technology. It is designed to eliminate the inefficiencies associated with traditional air cooling. Cooled liquid is circulated directly to cold plates mounted on CPUs and GPUs. Heat from the CPUs/GPUs is absorbed through these cold plates and transferred to the coolant liquid which is then circulated to heat exchangers where heat is removed from the coolant before the coolant is returned to the cold plates. Coolant Distribution Units (CDUs) are used to distribute and regulate the flow of coolant throughout the system. These systems provide much greater cooling capacity compared to air cooling as the liquid cooling fluids have a much greater thermal conductivity and are much more effective at transferring heat. This cooling technology is now common design practice for AI-centric datacentre environments with a number of vendors now shipping liquid-cooled AI systems at scale. However, unlike traditional air cooling, liquid is piped directly into the server racks which results in additional risks to consider.

Key risks we consider:

• Micro-leaks, quick-connect failures, and flexible hose degradation as liquid is now physically routed inside server racks
• Loss of coolant flow can cause rapid overheating within minutes, with no air-cooling fallback
• Galvanic corrosion, fouling, and reduced cooling efficiency resulting from poor coolant chemistry
• Increased maintenance complexity means that more specialised technicians are required as improper servicing can result in serious coolant leakage
   

Recommended good practice includes: 

• Dual containment (or pipe within a pipe) pipework
• Dripless quick disconnects
• Extensive leak detection, including below racks
• Redundant Cooling Distribution Units (CDUs)
• Comprehensive fluid chemistry monitoring
• Formal Management of Change (MOC) procedures
• Staff training and contractor qualification.

These are typically rack mounted heat exchangers located at the rear (exhaust side) of the server rack. A liquid coolant filled water coil is incorporated into the rear of a server rack. Hot exhaust air passes through the coil, transferring heat into the coolant allowing cooler air to exit into the data hall. RDHx cooling can be both passive (relies on existing rack airflow) or active which incorporates additional fans to boost exhaust airflow through the heat exchanger.

Key risks we consider:

• Water near energised rack equipment
• Condensation risks
• Structural loading – particularly for heavier assemblies
• Maintenance accessibility limitations
   

Recommended good practice includes: 

• Dew point monitoring and automatic condensation shutoff controls
• Rack anchoring verification
• Secondary containment strategies

This technology involves the servers being directly submerged in a dielectric heat transfer fluid. Heat from the servers is transferred to this fluid which is circulated through heat exchangers and cooled to maintain fluid temperature and ensure effective cooling. Immersion cooling can be both single-phase (the fluid remains in the liquid state as heat is absorbed from the servers) or two-phase where heat absorbed from the servers causes the fluid to boil into vapour. This vapour is then condensed back into the liquid state via heat exchanger coils located at the top of the cooling chamber. Two-phase immersion cooling is more expensive and complex than single-phase cooling. It requires a low boiling point fluid and a sealed immersion chamber.

Key risks we consider:

• Heat transfer fluid combustibility
• Component degradation
• Fluid leakage, contamination, seal degradation, and environmental/OH&S exposures associated with the dielectric fluid.
• Maintenance of submerged hardware is more complex and requires specialised procedures
• Conventional fire suppression strategies may need modification depending on fluid characteristics and tank configuration
   

Recommended good practice includes: 

• A comprehensive fluid testing program
• OEM-approved maintenance procedures
• Documented and tested emergency spill response plans and procedures
• A detailed review of fire protection requirements and systems
• Redundancy for critical fluid circulation equipment

For all cooling technologies, it is important that:

• Temperatures are continuously monitored with alarms upon high temperature or excessive temperature rate-of-change
• The facility fire protection system design considers the type of cooling system in use
• Inspection, testing, and maintenance is in accordance with manufacturers requirements.
• Key risks noted are considered, analysed and mitigated
• The system has a high level of resilience given the critical nature of cooling for modern datacentre operations.

Many new AI-focused facilities are combining multiple cooling methods within a single campus. This creates complex interdependencies across electrical infrastructure, cooling systems, water systems and control/automation infrastructure, all of which need to be understood and managed holistically, not in isolation.

Historically risk engineering reviews of datacentre facilities have focused on electrical infrastructure, UPS redundancy, emergency power generation, fire protection and facility construction. For underwriting and risk engineering reviews, we're now also routinely evaluating:

• Cooling system redundancy and failover capability
• Fluid leak detection and mitigation
• Maintenance systems, standards, experience and staff qualifications
• Control system integration and facility monitoring sophistication
• AI rack density and thermal management
• Emergency response procedures (particularly in relation to cooling system failures)

Our risk engineering team works with data centre clients across the full facility life cycle, from design review to construction and through to operational site assessments.

Our risk engineering team is able to:

• Arrange a risk engineering session for your team or a specific client
• Walk through our data centre capability from construction to operational cover
• Provide a technical briefing tailored to a client's cooling architecture