This weekend at TwitchCon, Nvidia is showing off its next generation streaming enhancements, dubbed the RTX Broadcast Engine. It turns out those GeForce RTX graphics cards aren't just about ray tracing in games. Leveraging the Tensor cores on the RTX series Turing GPUs, Nvidia is able to provide real-time effects like virtual greenscreen, AR avatars, and style transfer filters. And thanks to the improvements in NVENC (Nvidia's hardware accelerated video encoding engine), you can also get improved stream quality without tanking framerates—all running from a single PC without a bunch of extra dedicated hardware.
For professional streamers, the new additions may not matter that much. If you have a dedicated streaming studio, multiple cameras, sound mixing, and high-end 4K video capture hardware running in a second PC dedicated to encoding your livestream, you're not going to give all that up. But for amateurs and people just getting into streaming, the RTX Broadcast Engine helps bring down the cost of entry while improving your overall stream quality.
As the name implies, you'll need a GeForce RTX graphics card to make use of the new features. That's because several of the enhancements rely on the Tensor cores. However, the GTX 1660 and GTX 1660 Ti do have an improved video encoding engine that can still offload that complex task from your CPU, and NVENC is supported in the most popular streaming solutions, including OBS, XSplit, Streamlabs, Twitch Studio, Discord, and more.
Moving over to the new features, RTX Greenscreen is perhaps the most interesting. It uses AI trained models to detect your face, body, and hair, and then remove all of the background content like your messy office. Doing that effectively, and in real time, is a difficult task. The results shown in Nvidia's sizzle real (above) don't always look perfect, but the feature is free to use (or will be in the near future) on any RTX card. Plus, it's a lot easier to set up than trying to use a real greenscreen with other fancy hardware.
I'm less excited about the other two additions. RTX AR basically uses face tracking and recognition to animate your own digital avatar. Some streamers might like the level of anonymity that provides, but I think part of the appeal of watching the best streamers is that you can actually see a real person on the other side. RTX Style Filters meanwhile use an AI technique called style transfer to apply different artistic styles to an image or video. It can look pretty cool in short bursts, but again I don't think viewers are clamoring to see charcoal sketch renditions of their favorite streamers.
If you're headed to TwitchCon, you can stop by Nvidia's booth (booth 1823) to see RTX Greenscreen in action. Developers can apply for early access, while the rest of us will have to wait for OBS and other software to release public versions that support the new features. It's not clear how far out that is, but since it's being demonstrated live at TwitchCon, I'd wager we'll see release software support before the end of the year.
