Nvidia Touts Software Advantage in Beating Rivals Like AMD

Despite Nvidia’s reputation as a semiconductor company, the power of its software prowess is often underestimated. (Akio Kon/Bloomberg)

Key Points

• Nvidia demonstrated superior performance and efficiency in the new SemiAnalysis InferenceMAX benchmarks for AI inference.
• More than half of Nvidia’s engineers focus on software, highlighting its importance in the company’s AI strategy.
• AMD’s first rack-scale 72 GPU AI server product, the MI400 series, won’t be available until late 2026 at the earliest.

The most important component of winning the battle for AI hardware supremacy may not be chips, but software.

Despite Nvidia’s reputation as a semiconductor company, the power of its software prowess is often underestimated. More than half of the company’s engineers work on software.

On Thursday, Nvidia claimed victory for the best performance and efficiency in the newly-released SemiAnalysis InferenceMAX benchmarks, the first independent benchmark to measure inference performance across different models and workloads. Inference is the process of generating answers from AI models.

SemiAnalysis benchmarked the Nvidia GB200 NVL72, Nvidia B200, Advanced Micro Devices MI355X, Nvidia H200, AMD MI325X, Nvidia H100, and AMD MI300X in the initial release. The research firm plans to add chips from Google and Amazon in the future.

Nvidia Senior Director of AI Infrastructure Dion Harris said the foundation of the company’s approach is codesigning the entire computing system from networking to GPUs and then constantly improving the software to give customers better performance even on older hardware.

“I don’t think most people fully appreciate how complex these inferencing workloads are, when you’re deploying them at data center scale,” he said in an interview Thursday. “Introducing these algorithmic improvements translates into more revenue for that exact same infrastructure.”

The fact that Nvidia handily outperformed AMD’s current offering isn’t a big surprise. Nvidia’s GB200 NVL72 is a “full stack” rack-scale AI server, incorporating 72 GPUs linked together inside one server rack, along with networking and software. AMD’s first rack-scale 72 GPU AI server product, the MI400 series, won’t be available until late 2026 at the earliest.

Even then, it seems like Nvidia believes it will maintain market leadership. When asked about future rack-scale competition from AMD next year, Harris emphasized Nvidia’s track record of launching AI hardware faster than anyone else.

“We are moving literally at the speed of light. Our one-year cadence is about us leapfrogging ourselves,” he said, noting Nvidia’s next AI server called Vera Rubin is slated for next year.

He added: “Vera Rubin will enable more performance and efficiencies. We’re continuing to innovate. As long as we add value to our customers, I think we’ll be fine.”

When asked for comment about the benchmark results and Nvidia’s remarks, AMD sent a statement from the company’s GPU software executive, Anush Elangovan.

“InferenceMAX results are a live view of progress across the AI ecosystem. AMD Instinct GPUs are delivering strong, efficient inference performance today—and our open ROCm software is improving nightly.”

Write to Tae Kim at tae.kim@barrons.com