In context: Games at this time use highly-detailed textures that may shortly fill the body buffer on many graphics playing cards, resulting in stuttering and recreation crashes in latest AAA titles for a lot of avid gamers. With GPU producers being stingy with VRAM even on the most recent mid-range fashions, the onus is on software program engineers to discover a strategy to squeeze extra from the {hardware} accessible at this time. Perhaps paradoxically, probably the most promising growth on this route to this point comes from Nvidia – neural texture compression might cut back system necessities for future AAA titles, at the very least in relation to VRAM and storage.
One of the most well liked subjects in the intervening time is centered round fashionable AAA video games and their system necessities. Both the minimal and really useful specs have elevated, and as we have seen with titles like The Last of Us Part I, Forspoken, The Callisto Protocol, and Hogwarts Legacy, operating them even at 1080p utilizing the Ultra preset is now posing points for graphics playing cards geared up with 8GB of VRAM.
When trying on the newest Steam survey, we see that 8GB is the most typical VRAM dimension for PCs with devoted graphics options. That most likely will not change for some time, particularly since graphics card upgrades are nonetheless costly and GPU makers are not taken with providing greater than 8GB of graphics reminiscence on most mainstream fashions.
Also learn: Why Are Modern PC Games Using So Much VRAM?
The excellent news is Nvidia has been engaged on an answer that might cut back VRAM utilization. In a analysis paper printed this week, the corporate particulars a brand new algorithm for texture compression that’s supposedly higher than each conventional block compression (BC) strategies in addition to different superior compression methods similar to AVIF and JPEG-XL.
The new algorithm is solely known as neural texture compression (NTC), and because the title suggests it makes use of a neural community designed particularly for materials textures. To make this quick sufficient for sensible use, Nvidia researchers constructed a number of small neural networks optimized for every materials. As you’ll be able to see from the picture above, textures compressed with NTC protect much more element whereas additionally being considerably smaller than even these identical textures compressed with BC methods to 1 / 4 of the unique decision.
In fashionable video games, the visible properties of a cloth are saved in separate maps that describe the way it absorbs and displays mild, and the assortment used varies from one recreation engine to a different. Every map often packs further, scaled-down variations of the unique map into the identical file. These so-called “mipmaps” are used to optimize graphics reminiscence utilization when the complete decision of the feel is not wanted, similar to when a selected object is much away out of your standpoint.
Also learn: How 3D Game Rendering Works: Texturing
Researchers clarify the concept behind their method is to compress all these maps together with their mipmap chain right into a single file, after which have them be decompressed in actual time with the identical random entry as conventional block texture compression.
Compressing tens of 1000’s of distinctive textures for a recreation additionally takes time, however Nvidia says the brand new algorithm is supposedly ten occasions sooner than typical PyTorch implementations. For occasion, a 9-channel 4K materials texture will be compressed in a single to fifteen minutes utilizing an Nvidia RTX 4090, relying on the standard stage you set. The researchers word NTC helps textures with resolutions as much as 8K (8,192 by 8,192) however did not provide efficiency figures for such a situation.
An apparent benefit is that recreation builders will be capable of make the most of NTC to hopefully cut back storage and VRAM necessities for future video games, or, on the very least, cut back stuttering by becoming extra textures in the identical body buffer and thus lowering the necessity to swap them out and in when shifting throughout an in depth setting. Another benefit is that NTC depends on matrix multiplication, which is quick on fashionable GPUs and even sooner when utilizing Tensor Cores on GeForce RTX GPUs.
However, NTC does have some limitations which will restrict its enchantment. First, as with all lossy compression, it may introduce visible degradation at low bitrates. Researchers noticed gentle blurring, the removing of wonderful particulars, shade banding, shade shifts, and options leaking between texture channels.
Furthermore, recreation artists will not be capable of optimize textures in all the identical methods they do at this time, as an illustration, by reducing the decision of sure texture maps for much less vital objects or NPCs. Nvidia says all maps must be the identical dimension earlier than compression, which is sure to complicate workflows. This sounds even worse when you think about that the advantages of NTC do not apply at bigger digital camera distances.
Perhaps the most important disadvantages of NTC must do with texture filtering. As we have seen with applied sciences like DLSS, there may be potential for picture flickering and different visible artifacts when utilizing textures compressed by way of NTC. And whereas video games can make the most of anisotropic filtering to enhance the looks of textures within the distance at a minimal efficiency price, the identical is not doable with Nvidia’s NTC at this level.
Graphics geeks and recreation builders who need a deep dive into NTC can discover the paper right here – it is properly price a learn. Nvidia may also current the brand new algorithm at SIGGRAPH 2023, which kicks off on August 6.
