Nvidia's CEO reckons that millions of AI GPUs will reduce power consumption, not increase it

Jensen Huang
(Image credit: Nvidia)

The benefits of AI can be debated. But one thing we're all sure about is that massive server farms packed full of hundreds or thousands of high-end AI GPUs, each consuming hundreds of watts, soak up a lot of power. Right?

Not Jensen Huang, the CEO of Nvidia whose catchphrase has become "the more you buy, the more you save", in reference to his company's stratospherically expensive AI chips. Perhaps inevitably, Huang has a similar take when it comes to the power consumption associated with the latest AI models, which mostly run on Nvidia hardware.

Speaking in a Q&A following his Computex keynote, Huang's point is firstly that Nvidia's GPUs do computations much faster and more efficiently than any alternative. As he puts it, you want to "accelerate everything". That saves you money, but it also saves you time and power.

Next, he distinguished between training AI models and inferencing them and how the latter can offer dramatically more efficient ways of getting certain computational tasks done.

"Generative AI is not about training," he says, "it's about inference. The goal is not the training, the goal is to inference. When you inference, the amount of energy used versus the alternative way of doing computing is much much lower. For example, I showed you the climate simulation in Taiwan— 3,000 times less power. Not 30% less, 3,000 times less. This happens in one application after another application."

Huang also pointed out that AI training can be done anywhere, it's doesn't need to be geographically proximate. To put it his way, "AI doesn't care where it goes to school."

"The world doesn't have enough power near population. But the world has a lot of excess energy. Just the amount of energy coming from the sun is incredible. But it's in the wrong place. We should set up power plants and data centres for training where we don't have population, train the model somewhere else. Move the model for inference everyone's closer to the people—in their pocket on phones, PCs, data centres and cloud data centres," Huang says.

You can see his points, and yet... Let's put it this way, Huang's take is somewhat selective. It's hard to pin down exactly how much energy is currently being used to train and inference various AI models. The World Economic Forum recently estimated that AI-related power consumption is growing at a rate of between 26% and 36% annually and that by 2028, it will match that of a small country like Iceland.

An Nvidia power-saving device, yesterday. (Image credit: Nvidia)

Now, Huang's argument would be that this is or will be offset by lower consumption elsewhere. But that's only true for things that would otherwise be done without AI. The problem is that it's often tricky to tell the difference.

Whether you're using AI to create code, write a business plan, or create content, it's hard to pick apart the impact it has on the production process. Maybe using AI meant you edited those photos much more quickly and efficiently. Or maybe it just added an extra layer of polish to a video production at the end.

Or maybe you're just doing something with AI that would have been totally impossible without it, and every joule of energy you use is a net gain in the AI energy usage ledger. Equally, maybe AI lets you do things faster and therefore do more things. Each task is more efficient, but you move on to the next thing much more quickly and use much more energy as a consequence.

Computex 2024

The Taipei 101 building and Taipei skyline in Taiwan.

(Image credit: Jacob Ridley)

Catch up with Computex 2024: We're on the ground at Taiwan's biggest tech show to see what Nvidia, AMD, Intel, Asus, Gigabyte, MSI and more have to show.

Huang's comments about energy availability are pretty specious, too. Sure, there may be more than enough solar power to do everything we want, many times over in a totally sustainable fashion. But, really, so what? That's not, predominantly, where our power currently comes from.  

Just like his favourite "the more you buy, the more you save" quip there's obviously a strong element of "he would say that". But, broadly, it's hard to dismiss the notion that the energy footprint of AI is problematic. It's equally hard to imagine that AI is going to push net energy usage down any time soon, whatever Huang claims.

That day may come. Ironically, it may be with the help of AI to design a more efficient way of doing things that we get there. Who knows, maybe AI will help us work out how to make fusion energy production genuinely practical. But for now, just as it doesn't feel like saving money when an AI start-up has to pay $40,000 a pop for one of Nvidia's AI GPUs, it's not easy to square the race to build all those massive AI farms with claims of reduced power consumption.

TOPICS
Jeremy Laird
Hardware writer

Jeremy has been writing about technology and PCs since the 90nm Netburst era (Google it!) and enjoys nothing more than a serious dissertation on the finer points of monitor input lag and overshoot followed by a forensic examination of advanced lithography. Or maybe he just likes machines that go “ping!” He also has a thing for tennis and cars.

Read more
OpenAI logo displayed on a phone screen and ChatGPT website displayed on a laptop screen are seen in this illustration photo taken in Krakow, Poland on December 5, 2022.
New research says ChatGPT likely consumes '10 times less' energy than we initially thought, making it about the same as Google search
SUQIAN, CHINA - JANUARY 27, 2025 - An illustration photo shows the logo of DeepSeek and ChatGPT in Suqian, Jiangsu province, China, January 27, 2025. (Photo credit should read CFOTO/Future Publishing via Getty Images)
China's DeepSeek chatbot reportedly gets much more done with fewer GPUs but Nvidia still thinks it's 'excellent' news
Asus ROG Zephyrus G16 with an AMD Ryzen AI 9 HX 370 chip inside it.
Just in case you've forgotten all about them, AMD posts a less-than-convincing argument as to why AI PCs are better than any other type of PC
The Asus ROG Astral GeForce RTX 5090 on an LED-lit table at CES 2025
Jen-Hsun reckons Nvidia has driven the 'cost of computing down by 1,000,000 times'
Nvidia RTX 5090 Founders Edition rendered on a green background.
It's time for me to admit that AI-accelerated frame generation might actually be the way of the future and that's a good thing
TAIPEI, TAIWAN - 2023/06/01: Jensen Huang, President of NVIDIA holding the Grace hopper superchip CPU used for generative AI at supermicro keynote presentation during the COMPUTEX 2023. The COMPUTEX 2023 runs from 30 May to 02 June 2023 and gathers over 1,000 exhibitors from 26 different countries with 3000 booths to display their latest products and to sign orders with foreign buyers.
Microsoft is Nvidia's biggest AI chip buyer of the year, and it's not even close. With ByteDance and Tencent coming out ahead of Zuck, Bezos, and Musk's outfits, too
Latest in Hardware
Fallout New Vegas Key Art
The Fallout season 2 leaks continue with videos of the New Vegas set, including a sign for Mr. House's casino
The PCIe slot on an Asus ROG Strix B850-F Gaming WiFi motherboard, showing the Q-release latch for GPUs.
Gigabyte seemingly mocks Asus' recent Q-release debacle with a video swapping out an RTX 5070 Ti 100 times
Cyberpunk upscaling
New modder tool makes it easier than ever to swap AMD's FSR 4 scaling for Nvidia's DLSS or Intel's XeSS and vice versa
SteelSeries QcK Performance mouse pads overlapping on a desk
The SteelSeries QcK Performance series has reignited my excitement over the simple pleasure of a quality mouse pad… and trying to click skulls with pinpoint accuracy
NZXT H7 Flow PC case being built into
NZXT H7 Flow review (2024 Edition)
Xbox handheld
Microsoft is reportedly prepping a handheld Xbox for later this year with new consoles coming in 2027
Latest in News
A masked man with an axe in the woods
Rebellion CEO seems kind of awed by major studios making massive videogames: 'How do you organize a game that has 2,000 people working on it?'
A young witch watering a smiling mushroom in a magic garden
Here's a roguelite dungeon crawler Steam reviewers call 'a botanical Diablo' and 'like Cult of the Lamb' except you manage a mystical garden
Destiny 2 Rite of the Nine: The Emissary, massive, ominously standing at the edge of a water basin.
Oops! Bungie rolled out Destiny 2's Rite of the Nine event three weeks early, and new loot is already dropping
Chatacabra from Monster Hunter Wilds
The latest Monster Hunter Wilds event quest gives piles of Armor Spheres for hunting a Chatacabra, making this a very bad week to be a frog in the Forbidden Lands
No Rest for the Wicked Steam early access screenshots
No Rest for the Wicked developer Moon Studios is now 'fully independent' after acquiring the rights to the game from Take-Two
A hunter posing with an absurd Blangonga outfit in Monster Hunter Wilds.
Attention, fashion hunters: There's a Monster Hunter Wilds mod to disable all those obnoxious glowing buff effects that distract from your fits