If you learn this web site, likelihood is fairly good that you have most likely performed round with a neural picture upscaler in some unspecified time in the future. These applications, like the favored “waifu2x”, use a pre-trained neural community to upscale a picture with out introducing blurring, pixelated artifacts, or extra noise, supplying you with a clear upscaled picture.
This new methodology is certainly a texture compression methodology. The researchers say that it “exploits redundancies spatially throughout mipmap ranges and totally different materials channels.” They go on to say that it’s superior to even fashionable picture compression codecs like AVIF and JPEG XL whereas remaining appropriate for real-time decompression with random entry. They needed to create their very own decoder structure and a highly-optimized GPU compressor.
One of the main issues with neural picture upscalers is that whereas they regularly have a excessive diploma of perceptual high quality—in different phrases, their output “appears to be like good”—it’s usually really fairly removed from the unique picture by way of goal particulars. This “goal distortion” was a serious concern of the authors, and lowering this high quality in order that the ultimate output picture could be very near the supply was a precedence.
So how does “NTC” obtain such huge high quality enhancements and/or price financial savings? It’s that “exploiting redundancies spatially” factor. Typically, a online game materials lately could have at the least three if not 4 or much more texture recordsdata. These recordsdata will embody a diffuse map, a traditional map, a roughness-and-metalness map, and infrequently a displacement map. NVIDIA’s new methodology combines all of those right into a single file format.
There are some limitations of the format, after all. The largest one is that it requires the feel knowledge to be very rigorously authored, as a result of any alignment issues (which could go unnoticed with conventional compression) will quickly scale back the standard of the ultimate neural compression. You can also’t use conventional texture filtering with this methodology, which incorporates anisotropic filtering.
It’s nonetheless early days for Neural Texture Compression, and the authors did not embody any examples of what it could seem like inside an entire 3D scene, so sadly we actually do not know whether or not this methodology is actually applicable for video games. Still, it has the potential to drastically enhance the look of video games as video reminiscence calls for proceed to increase.
