Graphics upscaling using AI is so important for gaming Sony decided to do its own hardware rather than use AMD technology. That's one of the most important take-homes from the signature deep dive Sony's lead architect for its Playstation consoles, Mark Cerny, has just given.
AMD, of course, is responsible for both the CPU cores and the bulk of the GPU technology in PS5 Pro. But when it comes to AI acceleration for upscaling, Sony decided to very much do its own thing.
For clarity, what we're talking about here when we refer to "AI acceleration" is the matrix math hardware used to power neural networks, otherwise known as machine learning or ML. Currently, both Nvidia and Intel have dedicated matrix math acceleration hardware in their graphics chips for PCs. But AMD does not. Indeed, the existing AMD-based PS5 non-Pro has no dedicated matrix math hardware.
In Nvidia and Intel GPUs, that matrix math hardware is primarily used for various upscaling technologies, including DLSS and XeSS. The lack of matrix math hardware in AMD GPUs is why AMD's FSR upscaling is based on hand-coded algorithms rather than ML. And that, in turn, is a big part of the reason why FSR isn't as good even as XeSS by many measures and is certainly behind DLSS.
According to Cerny, Sony had two critical decisions to make for PS5 Pro and ML acceleration. First, it had to decide whether to have a dedicated NPU or Neural Processing Unit or to use an enhanced GPU. Sony chose the latter.
Then it had to decide whether to license that technology or build its own. Cerny says the PS5 Pro project started way back in 2020 and from the get go it was decided that Sony didn't want generic ML hardware, but something specific to PS5 Pro's workloads.
"Once you're licensing technology, that's what you're doing forever. So, in 2020, despite the degree of effort required, we decided to build our own hardware and software technology," Cerny says.
The target was 300 TOPS of ML performance. That's 300 trillion operations per second, or roughly on par with the raw ML performance of an Nvidia RTX 4060 Ti.
To achieve that, Sony modified AMD's RDNA 2 GPU architecture. "We made a set of targeted enhancements to the RDNA shader core and the surrounding memory systems. We're calling it custom RDNA," Cerny says.
The problem for Sony was that adding matrix math capability to the RDNA 2 shader core as used in the plain old PS5 wasn't going to be enough. Matrix math is incredibly bandwidth hungry to the extent that, without enhancements to the memory system, the new ML hardware would only achieve 3% of its full potential in a worst-case scenario. Yikes.
So, Sony came up with a custom on-chip memory solution, plus tiling algorithms, that allow the matrix math to be done entirely on the GPU and without having to access system memory.
Designing all this was a "four-year journey" according to Cerny. The result uses the vector registers in the RDNA 2 shaders as RAM. That provides 15 MB of memory at a combined bandwidth of 200 TB/s. That compares with the "mere" 576 GB/s of bandwidth of the PS5 Pros GDDR6 memory. Meanwhile, the tiling algorithms allows the matrix math to be split up into pieces small enough to fit inside that 15 MB.
Anyway, it's that new ML hardware that powers Sony's new Playstation Spectral Super Resolution or PSSR upscaling software, which is essentially Sony's answer to Nvidia DLSS and Intel XeSS. Oh and AMD FSR, albeit as explained above FSR is not based on machine learning.
"Looking back at the four years since we started this project, I'm so glad that we made the time-intensive decision to build our own technology. Results are good and just as importantly, we've learned so much about how AI can improve game graphics. It can only make our future brighter, say Cerny."
So, the big question that follows out of this is why Sony felt the need to do its own ML hardware. Cerny says Sony didn't want to be tied into a permanent licensing arrangement, which implies that Sony views ML upscaling as even more important than GPU architecture. After all, Sony is willing to license AMD's broader GPU architectures.
Then again, back in 2020 did AMD even have ML hardware to offer? It's worth noting that AMD's current RDNA 3 based GPUs still lack ML hardware. And some rumors suggest its upcoming RDNA 4 GPUs may not have ML hardware, either. So, maybe the only way Sony could get ML hardware into the PS5 Pro was to do its own thing.
If true, it would also mean that we likely won't see an AMD graphics chip with ML hardware until about 2027 when the upcoming RDAN4 chips are replaced with something based on AMD's new CDNA unified architecture.
Given even Intel's first-gen Arc GPUs had ML hardware and Nvidia is about to launch its fourth generation of GPUs with ML hardware, that really is a damning indictment of AMD's GPU technology and leaves the company nowhere in the AI upscaling race.
Here's hoping that rumors are incorrect and AMD has some GPUs with ML hardware coming soon.
