Microsoft has announced the launch of Azure Maia 300, the latest generation of its in-house accelerator chips designed to power AI inference across Microsoft Azure.

Maia 300 is designed for AI demands such as multi-agent systems and low-latency conversational interactions.

It uses TSMC’s multi-chiplet, two nanometre process node architecture for performance and power improvements and uses High Bandwidth Memory 4 for increased data throughput.

Agentic systems require additional memory, to maintain context and continuity over long-horizon, multi-step tasks. Microsoft said Maia 300 has a peak memory bandwidth of 18 TB/s, more than double that of Nvidia’s widely used GB300 compute platform.

Microsoft said these improvements mean Maia 300 delivers 40 per cent more tokens per megawatt than Maia 200, and up to seven times the token throughput.

The firm already uses its Maia chips already run production AI workloads across the Azure stack, including Microsoft Copilot, GitHub Copilot and its in-house AI models. Maia 300 will sit alongside Maia 200 in Microsoft data centres, with the chips

Microsoft has committed to scaling its Maia chips to multi-gigawatt deployments in 2027, at which point the Maia 300 will become the primary system it uses for AI inference.

Rani Borkar, president, Azure Hardware Systems and Infrastructure, said: “The economics of AI are increasingly determined not by peak compute performance, but how efficiently an entire system can generate tokens at scale.

“So we designed Maia 300 end to end to be our most efficient, scalable and reliable inference platform yet. The opportunity ahead is not simply to build more compute. It is to make advanced intelligence more accessible, affordable and useful.”

In-house chips have become a target for many AI firms, as they target cost optimisation through power efficiency and seek to reduce their reliance on third-party GPUs from the likes of Nvidia or AMD.

Google has been using its in-house tensor processing units for AI training and inference for over a decade, and Amazon has likewise been using its Trainium and Inferentia chips for several years.

OpenAI and Broadcom worked together on the Jalapeño chip, which the AI developer said delivers faster inference than its previous workhorse hardware, and both Meta and DeepSeek are reportedly working on their own chips.

Austin Lyons, founder and principal analyst at Chipstrat, said: “The advantage of custom silicon is that Microsoft can optimize for the latency its own workloads actually require.

“A meeting summary and a coding agent don't need the same response time. Maia targets more throughput at each of those latency points, translates to more useful tokens served per megawatt and lower cost passed for customers.”

---