OpenAI is reassessing parts of its chip strategy as it looks to improve how quickly its AI models respond to users. According to people familiar with the matter, the company has grown dissatisfied with the performance of some of Nvidia’s latest AI chips for specific inference tasks. Inference is the stage where AI models generate answers for users and it is becoming increasingly critical as products like ChatGPT scale to millions of daily requests.
While Nvidia continues to dominate chips used for training large AI models, inference has emerged as a new competitive battleground. Talks between OpenAI and Nvidia on a major investment deal, which could be worth up to $100 billion, have stretched on for months instead of closing quickly as earlier expected. During this period, OpenAI has signed deals with AMD and other chip makers as its product roadmap evolved and its computing needs changed. Nvidia has publicly denied any strain, with its CEO calling reports of tension “nonsense” and reaffirming plans for a large investment. Nvidia also said, “Customers continue to choose NVIDIA for inference because we deliver the best performance and total cost of ownership at scale.” OpenAI, for its part, said it still relies on Nvidia for most of its inference fleet and its CEO later posted that Nvidia makes “the best AI chips in the world”.
Several sources said OpenAI wants faster response times for tasks such as software development and AI to software communication. It is seeking new hardware that could eventually handle about 10% of its inference computing needs. Internally, concerns became clear during work on Codex, OpenAI’s coding focused product, where staff linked some performance issues to GPU based hardware. In a January 30 media call, the CEO said customers using coding models will “put a big premium on speed for coding work.” He added that a recent deal with Cerebras would help meet that demand, while speed is less critical for casual users.
OpenAI’s search has focused on chips with large amounts of SRAM built directly into the silicon, which can reduce memory delays during inference. In contrast, Nvidia and AMD GPUs rely on external memory, which can slow response times. Competing AI systems from other companies benefit from in house chips designed specifically for inference. As OpenAI explored these options, Nvidia also approached startups like Cerebras and Groq about potential acquisitions. Cerebras chose a commercial deal with OpenAI, while Nvidia later signed a $20 billion non exclusive licensing agreement with Groq and hired key talent, signaling a push to strengthen its position in a fast changing AI chip market.
Also read: Viksit Workforce for a Viksit Bharat
Do Follow: The Mainstream formerly known as CIO News LinkedIn Account | The Mainstream formerly known as CIO News Facebook | The Mainstream formerly known as CIO News Youtube | The Mainstream formerly known as CIO News Twitter
About us:
The Mainstream is a premier platform delivering the latest updates and informed perspectives across the technology business and cyber landscape. Built on research-driven, thought leadership and original intellectual property, The Mainstream also curates summits & conferences that convene decision makers to explore how technology reshapes industries and leadership. With a growing presence in India and globally across the Middle East, Africa, ASEAN, the USA, the UK and Australia, The Mainstream carries a vision to bring the latest happenings and insights to 8.2 billion people and to place technology at the centre of conversation for leaders navigating the future.
