Tensorrt for RTX: Automatically optimized AI inference for every Nvidia GPU 0 comments
Image: nvidia
It’s easy in games: with 3D APIs such as DirectX or Vulkan, new hardware is supported directly and generally runs faster without further adjustments. In the case of AI software, however, this is not the case, as tests with consumer AI software on desktops have shown with NVIDIA Blackwell graphics cards.
There are currently problems without separate optimizations.
The DirectML API, which works everywhere, doesn’t seem to be affected by this problem as much, but Nvidia, which specializes and optimizes it for GeForce cards, seems even more so. Without a single adjustment, probably noticed in the best case for each GPU, performance with new hardware doesn’t increase—and the program doesn’t even run so rarely. This should now change, as Nvidia announced at Microsoft’s Build developer event. Under the name “Tensorrt for RTX,” Nvidia is providing a revised version of its own software tools, which will eliminate the need for developers to optimize their programs for each GPU, and sometimes even each graphics card model, in the future.
The ruler is automatically optimized for your own GPU.
This will be done automatically by Tensorrt for RTX in the future. In the future, Tensorrt optimization will take place in real time on your own PC and should take a few seconds. Once the process is complete, Tensorrt performance should be the same as if the developer had manually run the program for their own graphics card. Furthermore, the new version of Tensorrt should be only an eighth as large as the old one—presumably because optimizations for all GPUs no longer need to be included.
The new version of Tensorrt, like the previous versions, will run on all GeForce RTX graphics cards.
Tensorrt for RTX will be available for download from Nvidia’s developer platform during the Junis conference and can be integrated into your own software from there. Some developers also have preliminary access to the feature; Nvidia expects the first versions of the program here during the Junis conference.
Techastuce received the information from NVIDIA in advance. These were made available under an NDA. The only requirement was the earliest possible publication date. The manufacturer had no influence on the report, and there was no obligation to publish it.
Topics: Calcutex 2025 GeForce Graphics Cards nvidia
An engineer by training, Alexandre shares his knowledge on GPU performance for gaming and creation.
