Every month, the developers of the GameCube/Wii emulator Dolphin write an interesting progress report that digs into the emulator's new features, bug fixes, and recent changes. These updates offer some great insight into complex emulation topics and the inner workings of an open source project—you'll always learn something, even if you're not too technically minded. But a lengthy blog updated published on Sunday goes far beyond the usual progress report, offering a detailed look at how Dolphin's developers solved a problem that's been bugging the community for years. It's called "Ubershaders: a ridiculous solution to an impossible problem."
It's a fascinating read, whether or not you use Dolphin yourself. If you have any interest in how GPUs and graphics cards work, or have always wanted to know more about shaders and 3D game programming, read it. I can't recommend it enough. This is a huge advancement for emulating a 3D console, and I wouldn't be surprised if it sees ripple effects in the emulation community over the next few years. Anyway, I'll give you the short version, and why this is such a big deal for Dolphin in particular.
Here's the super, super short version: you can now play some problem games, like the Metroid Prime trilogy, with zero stuttering, thanks to a new "ubershader" mode. What the hell is an ubershader? To understand that, you have to understand the problem Dolphin's dev team was trying to solve, and how many possible solutions they ruled out before landing on this one.
As the blog post explains:
"Modern GPUs are incredibly flexible, but this flexibility comes at a cost - they are insanely complicated. To unlock this power, developers use shaders - programs that the GPU runs just like a CPU runs an application - to program the GPU to perform effects and complex rendering techniques. Devs write code in a shader language from an API (such as OpenGL) and a shader compiler in the video driver translates that code into binaries that your PC's GPU can run. This compiling takes processing power and time to complete, so modern PC games usually get around this by compiling shaders during periods in which framerate doesn't matter, such as loadtimes. Due to the number of different PC GPUs out there, it's impossible for PC games to pre-compile their shaders for a specific GPU, and the only way to get shaders to run on specific PC hardware is for the video drivers to compile at some point in the game."
Understand so far? Okay, so here's the problem with emulating shaders. Consoles have fixed hardware, unlike PCs, so their shaders aren't written in the same way. They can be preconfigured, since the hardware will never change. In the GameCube/Wii GPU's case, configurations aren't stored in memory, and they can be called instantly. Dolphin has to translate those effects into shaders, and shaders have to be compiled, which takes time. "To deal with this disparity, Dolphin's only option is to delay the CPU thread while the GPU thread and the video driver perform the compilation - essentially pausing the emulated GC/Wii. Usually the compilation will take place in under a frame and users will be none the wiser, but when it takes longer than a frame, the game will visibly stop until the compilation is complete. This is shader compilation stuttering. Typically a stutter only lasts a couple of frames, but on really demanding scenes with multiple compiling shaders, stutters of over a second are possible."
Dolphin can cache shaders, so once a game has compiled a shader, it shouldn't cause stuttering again. That's why as you play games like Metroid Prime in Dolphin, the stuttering becomes less common. But you're still going to experience stuttering as you encounter new effects, and upgrading builds, changing drivers, etc. will wipe out that shader cache. So the developers tried to figure out a solution.
There were a lot of bad options. One amusing example is generating all the shaders beforehand, but the blog points out there are more possible graphical effects on the GameCube/Wii than there are grains of sand on Earth, so that seems impractical. Predicting shaders was considered, but the developers decided it would be too performance intensive, and due to the way games are programmed, likely wouldn't be effective, anyway. What about sharing shader configurations for games between users? That was possible, but like shader caches, those configurations would have to be tossed out frequently with new Dolphin builds. More critically, this would only work on a game-by-game basis, a big no-no for emulating a console, rather than specific games.
That left one popular solution: asynchronous shader compilation, which is akin to "pop in" in modern graphics engines. Essentially, if a shader is supposed to show up, instead of a stutter while it compiles, the game simply skips rendering it. Good: no stutter. Bad: briefly missing graphics. Worse: this actually permanently screws up some games.
That last problem is why, even though there's a popular branch of Dolphin that implements asynchronous shaders, the developers decided not to pursue it as a universal solution. Finally, one of Dolphin's major contributors, phire, came up with a crazy alternative.
"What if we don't have to rely on specialized shaders?" asks the blog post. "The crazy idea was born to emulate the rendering pipeline itself with an interpreter that runs directly on the GPU as a set of monsterous flexible shaders. If we compile these massive shaders on game start, whenever a game configures [GameCube/Wii GPUs] Flipper/Hollywood to render something, these "uber shaders" would configure themselves and render it without needing any new shaders. Theoretically, this would solve shader compilation stuttering by avoiding compilation altogether."
The post continues: "To put it into perspective, even among all the developers that work on Dolphin, only two or three people at most have the necessary knowledge on not only the GameCube/Wii hardware, but also modern GPUs, APIs, and the drivers to write, debug, and optimize the shaders. Not to mention running an interpreter as huge shaders is not exactly easy on the GPU, and many were afraid that all that work might not even run full speed on current video cards."
Phire spent months working on ubershaders, and the solution seemed to be working, eliminating shader compile stuttering, but with just as many bugs and problems as you'd expect from a brand new form of GPU emulation. The project got most of the way there and stalled for more than a year due to burnout.
Finally, another Dolphin dev, Stenzek, took up the task and helped see it through. Incredibly, ubershaders worked, "with stuttering completely eliminated in D3D, and only a few strange stutters early on in runs within OpenGL and Vulkan." And yet that wasn't the end of it, because with this power came a great cost: performance. Because ubershaders are basically emulating the entire GPU rendering pipeline, they're much more demanding.
"While each game's requirements will vary, your graphics card will greatly affect how high of a resolution you can run. At 1x Internal Resolution (480p) most dedicated GPUs should be able to get the job done, with higher end cards still able to push 1080p or higher even exclusively using Ubershaders. Unfortunately, many of our users don't have the necessary hardware to run Ubershaders at the resolution they'd prefer, which would put them in the unfortunate position of choosing between resolution and smoothness."
The developers figured out how to solve that problem, too. They created a new technique, Hybrid Mode Ubershaders, that " is a marriage of Ubershaders and Asynchronous Shader Generation into a beautiful solution that takes the best parts of both with none of their flaws." Basically, Dolphin will only use the pre-compiled ubershaders when a new rendering pipeline configuration is detected, while compiling that specific specialized shader in the background. Once it's compiled, the specialized shader will take over, essentially functioning as the emulator normally would.
"Assuming that drivers and APIs behave the way we want, this is the perfect solution. Because Ubershaders are only running on a fraction of the objects on a scene and only for frames at a time, the performance hit is almost entirely negated and stuttering is completely eliminated."
Because emulation is infinitely complex, there are exceptions and issues that prevent this from working as it should 100% of the time. Drivers and APIs don't always do exactly what they're expected to.
But for most Dolphin users, thanks to a herculean effort from some incredibly smart people, one of the emulator's longest standing issues has been solved. Some of the GameCube and Wii's best games, the Metroid Prime trilogy, are now playable without stuttering. And hopefully this helped you understand just what an impressive accomplishment that is.
