As demand for AI computing continues to grow, Microsoft is expanding its in-house chip programme to improve performance and power efficiency across its cloud services. The company has introduced the 3rd-generation Maia 300 AI inference chip, which it says can deliver 40% more tokens per megawatt than comparable GPU-based inference systems.
Microsoft launched its 1st AI inference chip in 2023, followed by Maia 200 in January 2026. The Maia 200 chip is already being deployed in select markets, including the US, Europe and Australia, and is claimed to deliver up to 1.7x more tokens per watt.
Maia 200 supports AI workloads across Microsoft Teams, Copilot and Cowork, running models from OpenAI and Microsoft’s MAI family. The company plans to extend support to GitHub Copilot and Microsoft Foundry by the end of 2026, allowing the chip to handle more of its production workloads.
The newly announced Maia 300 is set to become Microsoft’s primary AI inference platform. It will power the company’s in-house AI products and services and is designed for long-context, memory-intensive and decode-heavy workloads.
Microsoft said the chip balances computing power, memory bandwidth, networking and energy efficiency for large-scale cloud operations. Its software-defined dataflow architecture, combined with AI-assisted kernel optimisation, is designed to improve data movement and processing efficiency.
Maia 300 hardware specifications
The Maia 300 features 8 HBM4 memory stacks, delivering up to 18TB/s of peak memory bandwidth within a 950W thermal design power (TDP). It offers 288GB of HBM capacity and is built using TSMC’s N2P process node.
The chip also includes a 2-tier Ethernet scale-up architecture with AI Transport and supports 96 Maia chips per rack pod. Microsoft says these features are designed to improve memory performance and power efficiency for demanding AI inference workloads.
With Maia 200 deployments expanding and Maia 300 positioned as its primary inference system, Microsoft is increasing its reliance on in-house silicon to support its growing AI infrastructure.
Also read: Viksit Workforce for a Viksit Bharat
Do Follow: The Mainstream LinkedIn | The Mainstream Facebook | The Mainstream Youtube | The Mainstream Twitter
About us:
The Mainstream is a premier platform delivering the latest updates and informed perspectives across the technology business and cyber landscape. Built on research-driven, thought leadership and original intellectual property, The Mainstream also curates summits & conferences that convene decision makers to explore how technology reshapes industries and leadership. With a growing presence in India and globally across the Middle East, Africa, ASEAN, the USA, the UK and Australia, The Mainstream carries a vision to bring the latest happenings and insights to 8.2 billion people and to place technology at the centre of conversation for leaders navigating the future.