Sony decided four years ago to develop its own AI hardware for the PS5 Pro instead of using AMD technology, and the big question is why

On December 25, 2024By Littum

Graphics upscaling using AI is so important to gaming that Sony decided to make its own hardware instead of using AMD technology. That’s one of the main take-homes from signature deep dive Sony’s leading architect for its Playstation consoles, Mark Cerny, has just given.

AMD is of course responsible for both the CPU cores and the bulk of the GPU technology in the PS5 Pro. But when it comes to AI acceleration for upscaling, Sony largely decided to do its own thing.

For clarity, what we’re talking about here when we refer to “AI acceleration” is the matrix math hardware used to power neural networks, otherwise known as machine learning or ML. Currently, both Nvidia and Intel have dedicated matrix math acceleration hardware in their graphics chips for PCs. But AMD doesn’t. In fact, the existing AMD-based PS5 non-Pro has no dedicated matrix math hardware.

In Nvidia and Intel GPUs, the matrix math hardware is primarily used for various upscaling technologies, including DLSS and XeSS. The lack of matrix math hardware in AMD GPUs is why AMD’s FSR upscaling is based on hand-coded algorithms rather than ML. And that, in turn, is a large part of the reason why FSR is not as good even as XeSS by many measures and is certainly behind DLSS.

According to Cerny, Sony had two critical decisions to make for the PS5 Pro and ML acceleration. First, it had to decide whether to have a dedicated NPU or Neural Processing Unit or use an enhanced GPU. Sony chose the latter.

Then it had to decide whether to license that technology or build its own. Cerny says the PS5 Pro project started way back in 2020, and from the start it was decided that Sony didn’t want generic ML hardware, but something specific to PS5 Pro workloads.

PS5 Pro ML technology

Sony decided to go its own way with machine learning hardware. (Image credit: Sony)

“Once you license technology, that’s what you do forever. So in 2020, despite the effort required, we decided to build our own hardware and software technology,” says Cerny.

The goal was 300 TOPS of ML performance. That’s 300 trillion operations per second, or roughly on par with the raw ML performance of an Nvidia RTX 4060 Ti.

To achieve that, Sony modified AMD’s RDNA 2 GPU architecture. “We made a set of targeted improvements to the RDNA shader core and the surrounding memory systems. We call it custom RDNA,” says Cerny.

The problem for Sony was that adding matrix math capability to the RDNA 2 shader core, as used in the regular old PS5, wouldn’t be enough. Matrix math is incredibly bandwidth hungry to the extent that, without improvements to the memory system, the new ML hardware would only achieve 3% of its full potential in a worst-case scenario. Yikes.

So Sony came up with a custom on-chip memory solution plus tiling algorithms that allow the matrix math to be done solely on the GPU and without having to access system memory.

PS5 Pro ML technology

(Image credit: Sony)

Designing all of this was a “four-year journey,” according to Cerny. The result uses the vector registers in the RDNA 2 shaders as RAM. It provides 15 MB of memory at a combined bandwidth of 200 TB/s. That can be compared to the “mere” 576 GB/s bandwidth of the PS5 Pro’s GDDR6 memory. Meanwhile, the tiling algorithms allow the matrix math to be broken up into chunks small enough to fit within the 15MB.

Anyway, it’s the new ML hardware that powers Sony’s new Playstation Spectral Super Resolution or PSSR upscaling software, which is essentially Sony’s answer to Nvidia DLSS and Intel XeSS. Oh and AMD FSR, although as explained above FSR is not based on machine learning.

“Looking back on the four years since we started this project, I’m so glad we took the time-consuming decision to build our own technology. The results are great, and just as importantly, we’ve learned so much about how AI can improve the game’s graphics It can only make our future brighter, says Cerny.”

So the big question that follows from this is why Sony felt the need to make its own ML hardware. Cerny says that Sony did not want to be tied to a permanent licensing arrangement, implying that Sony sees ML scaling as even more important than GPU architecture. After all, Sony is willing to license AMD’s broader GPU architectures.

PS5 Pro ML technology

The PS5 Pro’s ML hardware has about the same matrix math power as an Nvidia RTX 4060 Ti. (Image credit: Sony)

Then again, back in 2020 did AMD even have ML hardware to offer? It’s worth noting that AMD’s current RDNA 3-based GPUs still lack ML hardware. And some rumors suggest that its upcoming RDNA 4 GPUs may not have ML hardware either. So maybe the only way Sony could get ML hardware into the PS5 Pro was by doing its own thing.

If true, it would also mean we probably won’t see an AMD graphics chip with ML hardware until around 2027, when the upcoming RDAN4 chips are replaced by something based on AMD’s new CDNA unified architecture.

Since even Intel’s first generation of Arc GPUs had ML hardware and Nvidia is about to launch its fourth generation of GPUs with ML hardware, this is really a damning indictment of AMD’s GPU technology and leaves the company nowhere in the AI scaling race.

We hope that the rumors are wrong and that AMD has some GPUs with ML hardware soon.