Earlier this year Nvidia announced DLSS 5, a new version of their DLSS technology: a layer of graphical magic that uses their GPUs to do improbable things to the resolution and framerate of your favourite PC games (unless your favourite PC game is Slay the Spire). Nvidia are in a strange space at the moment – now one of the most valuable companies to ever have existed, they’re also one of the drivers of the AI boom and all of the horror and chaos that comes with it. PC gamers, once their core market, are now finding it harder than ever to afford their products, as Nvidia spent most of the last decade overselling first to crypto miners, and then to AI startups. On paper, DLSS 5 is a chance for a simple PR win, a positive example of using AI to improve the performance of modern videogames. Instead, they offered an opportunity to make every game you own slightly uglier, and only if you can afford the dual-GPUs it requires to run.
DLSS 5 is fascinating not just as an extension of Nvidia’s AI-first business philosophy, and not just as a sign of the slowdown in hardware advancement, and not just because of what it says about our desire for ‘more real than real’ graphics. DLSS 5 also reflects the world we find ourselves in today, perhaps the most neatly-packed microcosm I’ve seen to date. But for those wondering, yes, you do need to have read DLSS 1-4 to understand the plot of DLSS 5, so let’s start with a quick recap.
DLSS’ original purpose was to improve the performance of games by rendering each frame at a lower resolution and using AI to blow that image up to a bigger size. Upsampling images is something we’ve been doing with computers for decades – if you’ve ever increased the size of an image in Photoshop, you were upsampling it. All upsampling algorithms are guessing what the bigger image should look like. Older algorithms used consistent mathematical processes for this, but DLSS 1 used a neural network to do this faster and more adapted to the context. This required specific training for each game, but with DLSS 2 Nvidia were able to train a more general tool that could do this on games it hadn’t seen before. Rendering a smaller frame and then upscaling it is a lot more efficient than rendering a larger frame, which meant that DLSS 2 could double a game’s framerate, in some cases.
As DLSS processes got faster, Nvidia became more ambitious with their goals. DLSS 3 added Frame Generation, which would generate an entirely new frame to blend between two real, rendered frames, further increasing the potential framerate (but also increasing the demands on the GPU). DLSS 4 increased this to four generated frames per real rendered frame, up to six generated frames for DLSS 4.5.
DLSS 5 is unlike any previous version of DLSS. Playing a game with DLSS 4.5 turned on, around 95% of what the player is seeing on the screen is not being rendered in the historical sense of the word ‘rendering’. However, the aim of the system is to make that 95% as close as possible to the remaining, traditionally rendered game frames. DLSS 5, like its predecessors, generates frames – but it now generates 100% of what you see on the screen. Rendered frames are no longer upscaled but instead are used as the input for new regenerated frames, which can change anything and everything about the image, from textures to lighting. Nvidia want you to be fully aware that DLSS 5 is changing things, it wants you to notice the “infused” pixels, and “photoreal” lighting. DLSS 5 is not quietly working behind the scenes, it is striding confidently into the center of the stage, overdramatically lit with its hands on its hips.
Part of the initial backlash to DLSS 5 was a concern that developers would have no control over what DLSS 5 did to their work, but Nvidia maintain that they see this as something developers would customise for individual games. They even have some breathless quotes from Bethesda and Capcom saying how eager they are to use it. But even if we take this at face value, I’m still left with plenty of questions. How can Nvidia guarantee consistency between versions and retrainings of the DLSS model? How do I test the effect of DLSS 5 on every single possible frame that my game could generate? The advantage of conventional 3D game engines is that if you ask it to draw a circle, it draws the same circle pretty much every time, and it’s probably going to draw circles the same way tomorrow too. It’s not clear to me how you guarantee any of this with a neural model, even a really expensive and expansively trained one.
There’s also a very clear gap in the messaging from Nvidia around what they view as the future of this technology. Currently, DLSS has to be advertised as optional, because it requires extremely specific technology to run and, in the case of DLSS 5, extremely expensive technology too (the demos shown required two RTX 5090s, one to run the game and a second just to do the DLSS 5 work). However, Jensen Huang, Nvidia’s CEO, suggested that developers might want to use DLSS 5 to create different shader effects, like making the game look like a cartoon or as if it were made of glass – fairly huge stylistic changes that would define an entire game.
This, to me, implies that the aim is to make DLSS 5 the default way of rendering games in the future, which is a very different proposition. This would make DLSS 5 the canon, ground truth version of the game, which has massive implications for Nvidia’s control over the games industry, games preservation, independent hardware manufacturers and more. Viewed this way, DLSS 5 is not simply a perk of owning an Nvidia graphics card, it is a power grab to seize control of how all games are experienced, forever.
The history of 3D graphics is a history of fakes and trickery. We can mathematically describe a 3D model of a cube or a plane or a shotgun in incredible or sometimes mathematically infinite levels of detail, but to render it we can’t draw an infinitely straight line or paint an infinitely smooth colour gradient on a screen that is only a few thousand pixels wide. Over the past decades we’ve developed lots of techniques to help 3D graphics get closer to how we imagine our games should look, like anti-aliasing, which tricks our eyes into seeing things that aren’t quite there. All of these techniques represent tradeoffs of different kinds, that offer gains in some areas and losses in others. So the issue with DLSS 5 isn’t exactly that it’s showing us something that “isn’t really there”, because that’s what most game development is about. It’s the scale at which it does this, the technology pipeline it seeks to create, and how Nvidia has sought to wedge itself directly between game developers and the people who play their games.
The DLSS 5 saga is a microcosm of the situation we find ourselves in with the AI industry today. A large technology company, that tripped and fell into a monopoly position in a tech bubble, announces a new product that puts technical achievement over aesthetic or artistic work. The public reaction was somewhere between indifferent and negative, largely because the product was not solving a problem anyone had. Instead of listening to their critics, or engaging with their audience, the company’s response was to double down and criticise anyone expressing disinterest or concern. To cap it off, this company is a load-bearing part of this enormous creative industry, and most of us rely on them every day in order to continue to work on, write about or enjoy playing videogames.
Videogames are a powerful driver of technological change. Game developers solve incredibly hard problems in innovative ways, and games can popularise new technology and accelerate its adoption. A lot of tech leaders have taken this for granted as a natural process that happens to any new idea, but the reason this has happened in the past is usually because those new ideas have received grassroots enthusiasm and support, or emerged through addressing the genuine needs of creative people trying to do silly, complicated and wonderful things. The games industry is not a machine that you feed new technology into and get money out of, and the DLSS 5 response shows what happens when you take this for granted.
But the fact it exists is also a reflection on us, and our desire for a constant drive towards better graphics and higher framerates. This is unlikely to be the last we hear about neural rendering, and so perhaps it’s a sign for developers and players to think about what good graphics (and good art) means to them, and what they’re willing to sacrifice to make it happen.
