UL Benchmarks (formerly Futuremark) just released the latest update to its 3DMark graphics testing suite, Port Royal. It's the first DirectX Raytracing (DXR) benchmark from the company, and it's designed to give us a look at what graphics in games could become over the next several years. That's because unlike Battlefield 5, it implements DXR reflections, refractions, lighting and shadows, and—in the demo mode—even "dynamic global illumination with ray traced photon mapping." (I asked, and this is actually only used in the first bit of the demo with the drone flying around in the dark, but damn if it doesn't sound impressive!) Needless to say, that's enough to push some GeForce RTX cards to their breaking point.
I ran the benchmark on the four GeForce RTX cards to see how they performed under the intense workloads. The standard Port Royal run uses a 2560x1440 resolution, so already a step above the 'typical' 1080p display. It requires DXR compatible hardware and drivers, and lists 6GB VRAM as the minimum as well.
In theory AMD and Nvidia could release updated drivers for RX-series and GTX model cards, where DXR calculations would happen using compute shaders or something similar, but for now all non-RTX GPUs simply refuse to run the benchmark. There are a few scores in 3DMark's database for non-DXR cards, but if you look closely, you'll find they're using an RX or GTX card as the primary GPU with an RTX card as a secondary GPU, with the test running on the latter.
3DMark reports a score as well as a framerate for the graphics test, while the demo in theory is just for looks. You could record the demo framerate as well, though it runs at your desktop resolution by default—so on a 4K display, it's going to be significantly choppier than the graphics benchmark. If you use the same settings and resolution as the benchmark, it runs about 15-20 percent slower, probably due to the aforementioned dynamic global illumination.
Running the demo (or the main benchmark) at 4K does give a nice view of what happens when you exceed the VRAM capacity for a lower spec card like the RTX 2060, which chunks along at around 1fps. I also noticed that using ShadowPlay to capture a video of the benchmark didn't go so well on the RTX 2060, with performance dropping about 75 percent. ShadowPlay wasn't a problem on the other GPUs, which showed the usual 3-5 percent drop, but the 2060 is clearly running near the limit of its resources.
Besides testing in the full DXR-enabled mode, Port Royal has a custom run option where you can change the various settings. One setting is reflection mode, which can be set to 'ray traced' (the default), 'traditional' (Screen Space Reflections (SSR)), or 'off' (no reflections at all). However, the traditional mode isn't quite the same as what you'd normally see in games, as the benchmark uses a series of complex post-processing filters to help remove noise generated by using one ray per pixel. These filters are only disabled when setting the reflection mode to off. In Battlefield 5 as a point of contrast, running without DXR reflections (but with SSR and other filtering still in place) results in about a 50-75 percent boost in performance.
Regardless, I tested the three reflection modes and captured framerates to compare performance. The traditional mode gives a modest boost to performance, while no reflections nearly doubles performance. Don't get too excited about the jump, however, as the change in visuals is very significant. Turning reflections off almost makes it look like a completely different test, with the shiny ship replaced by a black silhouette, and SSR mode also has major changes in appearance. For example, many of the SSR generated reflections are somewhat low resolution, and you can see some of the worst offenders in the videos I've uploaded showing the different modes running on an RTX 2080 Ti (see above and at top).
Finally, note that the custom mode overrides the demo's default settings (eg, the aforementioned "dynamic global illumination with ray traced photon mapping" is supposed to always be on for the demo, though a bug in the initial release appears to turn it off if you use the custom run). There's no way to manually enable/disable that feature, but we're not really worried about the demo performance. Here are the results, broken down into 3DMark score, graphics framerate, and 4K demo framerate.
The 3DMark graphics score and fps are closely related—specifically, the score is the framerate multiplied by 216. Why 216? Because that keeps the scoring roughly "in line with traditional 3DMark scores" according to the 3DMark Technical Guide. And since there's only the one graphics test in Port Royal, it means the 3DMark score is the same as the graphics score.
I've provided separate charts, with the official scoring in one, and framerates in the others. Since a custom run doesn't generate a score, I've only included my own framerate captures for the second chart, which also includes the 97 percentile fps (minimum average fps for the bottom three percent of frametimes), and I'll focus on framerates rather than 3DMark scores.
The short summary is that there's good scaling going from the RTX 2060 to the 2070, with the 2070 being 33 percent faster. The RTX 2080 on the other hand is only 20 percent faster than the 2070, while the 2080 Ti is 37 percent faster than the 2080.
Interestingly, the scaling isn't quite what you'd expect based on computational performance, where the 2070 is theoretically only about 22 percent faster than the 2060. That suggests the memory bandwidth is a bigger factor, where the 2070 has exactly 33 percent more bandwidth. Similarly, the 2080 has about 34 percent more computational power than the 2070 but the same bandwidth. The 2080 Ti on the other hand has 37 percent more bandwidth and 34 percent more computational power, and ends up scaling right in line with those figures.
Switching to SSR for reflection mode (aka 'traditional') boosts performance 10-20 percent, depending on the GPU. The RTX 2060 sees the biggest gains, likely because it's struggling the most with the ray tracing load. Switching reflections to off boosts performance by 115-160 percent, again with the 2060 seeing the largest improvement.
I didn't show results for testing at other resolutions (unless you count the demo chart), but I did test at 1080p and 4K as well as the default 1440p. Running at 1080p boosts performance by 60-80 percent, while 4K drops performance around 50-60 percent on the 2070, 2080, and 2080 Ti. 4K on the 2060 however basically tanks, dropping to less than a tenth of its baseline performance.
Port Royal doesn't technically run at maximum quality either, particularly with regards to ray tracing. Keeping the resolution at 1440p but bumping the 'max reflection sample count' from the default 1 to 4 along with setting the 'reflection filtering quality' to high (instead of medium) represents a bit more of a forward-looking approach to ray tracing. Running on the RTX 2080 Ti, that dropped performance from 37.8 fps to just 23.0 fps, which is a pretty big drop. I suspect that in the second or third generation of ray tracing hardware, we'll probably have GPUs that can handle such a workload at reasonable performance, but right now it's clearly out of reach.
3DMark has never really been about perfectly simulating games, of course. Ultimately it's just a benchmark that gives one view of potential gaming performance. Framerates are much lower than what we see in most games, and they're even lower than Battlefield 5 with its DXR reflections, but that's expected. 3DMark's Fire Strike Extreme/Ultra and Time Spy Extreme/Ultra tests also run at lower framerates compared to most games. The workloads are often intentionally heavy, in order to better simulate future games.
I don't want to draw any conclusions about the various RTX cards and ray tracing performance other than what the tests show. Basically, real-time ray tracing is expensive, even on RTX hardware. The good news is we now have a foot in the door and things will only get better over time. I think back to the first Direct3D games and early examples of pixel shaders, and we've come a long way. Ten years from now, even budget GPUs will likely run ray tracing in games at reasonable framerates.
