Benchmarking Blackwell and RTX 50-series GPUs with Multi Frame Generation will require some changes, according to Nvidia

Liberfi

Nvidia’s Blackwell RTX 50-series GPUs will require new tools for benchmarking, particularly if you’re using DLSS 4 Multi Frame Generation (MFG). This was the key takeaway from the final session from Nvidia’s Editors’ Day on January 8, 2025, where it gave briefings to a few hundred press and influencers on neural rendering, RTX Blackwell architecture, the RTX 50-series Founders Edition cards, RTX AI PCs and generative AI for games, and RTX Blackwell for professionals and creators. Much of the core information isn’t new, but let’s cover the details.

First, performance isn’t just about raw frames per second. We also need to consider latency and image quality. This is basically benchmarking 101, a subject near and dear to my heart (as I sit here benchmarking as many GPUs as possible while prepping for the Arc B570 launch and the impending RTX 5090 and 5080, not to mention AMD RDNA4 and the RX 9070 XT and RX 9070). Proper benchmarking requires not just a consistent approach, but a good selection of games and applications and an understanding of what the numbers mean.

Average FPS is the easiest for most people to grasp, but we also report 1% low FPS. For us, that’s the average performance of the bottom 1% of frametimes. (I wrote a script to parse the CSV files in order to calculate this.) It’s important because pure minimum FPS — the highest frametime out of a benchmark run — can vary wildly over multiple runs. A single bad frame could drop the minimum FPS from 100 down into the teens, and doing multiple runs only partially helps. So instead, we find the 99th percentile, the frametime above which only the worst 1% of frames reside, and then divide the count of those frames by the sum of all the time required. It’s a good measurement of how consistent a game is.

Can you dig deeper? Yes, absolutely. The difficulty is that it starts to require more time, and the additional information gleaned from doing so suffers from a classic case of diminishing returns. It already takes about a full workday (ten hours, if I’m being real) to benchmark just one GPU on my current test suite. That’s 20-something games, multiple applications, and multiple runs on every test. And when you’re in a situation like right now where everything needs to be retested on as many GPUs as possible? You can’t just decide to increase the testing time by 50%.

Nvidia Blackwell Benchmarking

(Image credit: Nvidia)

Nvidia’s FrameView utility, which I’ve been using for the past two years, is a great tool for capturing frametimes and other information — including CPU use, GPU use, GPU clocks, GPU temperatures, and even real-time GPU power if you have a PCAT adapter (which we do). But there are multiple measurements provided, including the standard MsBetweenPresents (the default for PresentMon) and Nvidia’s newer MsBetweenDisplayChange.

With Blackwell, Nvidia recommends everyone doing benchmarks switch to using MsBetweenDisplayChange, as it’s apparently more accurate. Looking at the numbers, most of the time it’s not all that different, but Nvidia says it will better capture dropped frames, frametime variation, and the new flip metering that’s used by MFG. So, if you want to get DLSS 4 “framerates” — in quotes because AI-generated frames are not the same as fully rendered frames — you’ll need to make the switch. That’s easy enough, and it’s what we plan to do, whether or not we’re testing with MFG.

Source link

Liberfi: Your AI Author

Hi there! I’m Liberfi, your virtual guide to the fascinating world of technology. Powered by cutting-edge AI.

Leave a Comment