AWS Launches Advanced Cooling for Nvidia GPUs

Published Date : 10/07/2025 

Amazon Web Services (AWS) has developed new hardware to cool Nvidia's powerful GPUs, which are crucial for artificial intelligence workloads. This innovation aims to address the growing energy and cooling demands of advanced AI systems. 

The letters AI, which stands for 'artificial intelligence,' stand at the Amazon Web Services booth at the Hannover Messe industrial trade fair in Hannover, Germany, on March 31, 2025. Julian Stratenschulte | Picture Alliance | Getty Images

Amazon announced on Wednesday that its cloud division has developed hardware to cool down next-generation Nvidia graphics processing units (GPUs) used for artificial intelligence (AI) workloads. Nvidia's GPUs, which have powered the generative AI boom, require massive amounts of energy, meaning companies using these processors need additional equipment to cool them down.

Amazon considered erecting data centers that could accommodate widespread liquid cooling to make the most of these power-hungry Nvidia GPUs. However, this process would have taken too long, and commercially available equipment wouldn't have been sufficient. Dave Brown, vice president of compute and machine learning services at Amazon Web Services, explained in a video posted to YouTube, 'They would take up too much data center floor space or increase water usage substantially. And while some of these solutions could work for lower volumes at other providers, they simply wouldn't be enough liquid-cooling capacity to support our scale.'

Instead, Amazon engineers conceived of the In-Row Heat Exchanger (IRHX), which can be plugged into existing and new data centers. More traditional air cooling was sufficient for previous generations of Nvidia chips. Customers can now access the AWS service as computing instances that go by the name P6e. Brown wrote in a blog post that the new systems accompany Nvidia's design for dense computing power. Nvidia's GB200 NVL72 packs a single rack with 72 Nvidia Blackwell GPUs that are wired together to train and run large AI models.

Computing clusters based on Nvidia's GB200 NVL72 have previously been available through Microsoft or CoreWeave. AWS is the world's largest supplier of cloud infrastructure. Amazon has rolled out its own infrastructure hardware in the past. The company has custom chips for general-purpose computing and AI, and designed its own storage servers and networking routers. By running homegrown hardware, Amazon depends less on third-party suppliers, which can benefit the company's bottom line. In the first quarter, AWS delivered the widest operating margin since at least 2014, and the unit is responsible for most of Amazon's net income.

Microsoft, the second largest cloud provider, has followed Amazon's lead and made strides in chip development. In 2023, the company designed its own systems called Sidekicks to cool the Maia AI chips it developed. 

Frequently Asked Questions (FAQS): 

Q: What is the main purpose of the In-Row Heat Exchanger (IRHX) developed by AWS?

A: The In-Row Heat Exchanger (IRHX) is designed to cool down next-generation Nvidia GPUs used for artificial intelligence workloads, which require significant amounts of energy and cooling.

Q: Why did AWS develop its own cooling solution instead of using commercially available equipment?

A: AWS developed its own cooling solution because commercially available equipment would take up too much data center floor space, increase water usage substantially, and lack the necessary liquid-cooling capacity to support AWS's scale.

Q: What are the new computing instances called that customers can now access on AWS?

A: Customers can now access the new computing instances on AWS as P6e, which are designed to support the dense computing power of Nvidia's GB200 NVL72.

Q: What is the significance of Nvidia's GB200 NVL72 in the context of AI workloads?

A: Nvidia's GB200 NVL72 is a single rack that packs 72 Nvidia Blackwell GPUs, which are wired together to train and run large AI models, making it a powerful solution for AI workloads.

Q: How does AWS's development of its own infrastructure hardware benefit the company?

A: By developing its own infrastructure hardware, AWS reduces dependency on third-party suppliers, which can lead to cost savings and improved performance, contributing to the company's bottom line. 

More Related Topics :