The Road to the 1MW Rack: Why Power and Cooling Are the New Bottleneck in AI Infrastructure

Notes from a panel discussion featuring Bill Kleyman (Apolo.us), Jasmeet Singh (AWS), and Jason Agee (Delta Electronics).

The AI boom isn't just a software story; it's a physics problem. That was the throughline of a recent industry panel on data center infrastructure, where the conversation kept circling back to one question: how do you fit more and more power into the same footprint without melting the building down?

The Density Curve Is Bending Upward, Fast

The numbers tell the story on their own:

The industry is compressing roughly 200 years of infrastructure evolution into the next 24–36 months. That pace is the whole challenge in a nutshell.

Why 800V DC Is the Answer

The core physics problem is current, not power. In AC systems, current, not power, is what paces the system, and pushing more current through existing infrastructure runs into hard physical limits. That's the case for 800V DC architecture: it needs less current for the same power delivery, which means the same copper bus can carry roughly 80% more power density while cutting required copper by about 40%.

The efficiency gains compound at scale. Panelists estimated 800V DC and solid-state transformers could unlock up to 5% efficiency gains. At gigawatt scale, even a single percentage point of efficiency is worth roughly $60M annually in power savings and $40M in cooling savings for a 1 GW AI facility, before even counting the real estate it frees up in the data hall.

Two Workloads, One Campus

A recurring tension in the discussion: AI inference and training are like an F-250 truck — powerful, but not efficient for everyday tasks. Meanwhile, the "boring" workloads (email, spreadsheets, video apps) still need their Toyota Prius — traditional colocation running H100-class GPUs and general compute.

Neither is going away, and increasingly they have to share the same physical campus. That means:

• High-density computing (HDC) and colocation must coexist, often side by side.
• Infrastructure needs to be "modular and prefabricated", with one cooling platform flexible enough to serve both.
• Systems need to be "plug-and-play and swappable" without growing in physical size as density increases, providing drop-in compatibility across different workloads.

Cooling Has Hit a Wall

The stats are stark: 39% of data centers report they're running out of active cooling capacity, and that number jumps to 68% for high-density deployments specifically.

Air cooling has reached its physical limits. It isn't disappearing, but it can no longer carry the load alone; liquid cooling has to supplement it. Among liquid strategies, water, single- and two-phase refrigerant, and rear-door heat exchangers (RDHx) are gaining the most traction. Immersion cooling, while efficient, hasn't scaled at the same rate. For most operators, the upfront cost and scalability challenges haven't justified the efficiency gains.

One underappreciated design trap: the physical overlap between air-cooled and liquid-cooled infrastructure, where electrical, mechanical, and cooling systems get tangled together in the building structure. This is a mistake that gets discovered too late and costs data centers hundreds of millions of dollars a year.

There's also a "retrofit vs. purpose-built" debate raging in the industry right now. The panel's answer: it's not either/or - it has to be both.

The Holy Trinity

The panel framed the whole challenge around three pillars:

Without all three moving together, the jump from AC to 800V DC architecture and the AI infrastructure it's meant to support simply can't happen.

And there's a reality check baked into all of this: announced, installed, electrified, and operational megawatts are not the same thing. Plenty of ambitious development plans get cancelled before they ever reach production, which means roadmap numbers should be read with some skepticism.

The Bottom Line

Today's operational maturity doesn't yet support the 600 kW racks already on the roadmap, let alone the 1 MW rack looming beyond that.

As one panelist summarized it: "Vision without execution is nothing more than hallucination." Or, more optimistically: "Necessity is the mother of innovation."

The demand for AI compute isn't slowing down, but the infrastructure needs to catch up. And as cooling requirements grow, attention must extend beyond cooling technologies to the components that make them work. Reliable coolant circulation, precise temperature control and efficient heat transfer are becoming increasingly important to high-density data center operations.

At Tark Thermal Solutions, we support this evolution with centrifugal pumps for coolant distribution units (CDUs) and thermoelectric coolers for high-speed optical transceivers, addressing thermal management challenges both within the rack and across the networks connecting AI infrastructure.