At its annual Ignite Conference in Seattle on Wednesday, Microsoft announced a duo of custom-designed in-house and data centre-bound AI chips amid the shortage of high-performance chips on the market. According to rumours, NVIDIA has already sold out their best-performing AI Chips until 2024, with the CEO of chipmaker TSMC claiming the shortage could extend until 2025.
Therefore, to combat the shortage of AI chips to power generative AI models such as ChatGPT, Microsoft has joined tech giants such as Meta and OpenAI in developing its own custom version. After months of prolonged research, Microsoft unveiled its Azure Maia AI chip and Azure Cobalt CPU at its Ignite Conference yesterday.
Ignite Conference Highlights
Azure Maia AI Chip
Microsoft unveiled the Azure Maia AI chip’s first, adeptly named after a bright blue star, at the Ignite Conference as Rani Borkar stated: “With 105 billion transistors, it is one of the largest chips on 5-nanometer process technology. The company specifically engineered the Azure hardware stack to achieve the absolute maximum utilisation of the hardware. It supports our first implementation of the sub-8-bit data types, MX data types, in order to co-design hardware and software and also helps us sustain faster model training and inference times.”
For example, Microsoft highlighted that the company will use the Maia chip to run cloud AI workloads, like large language model training and inference. Furthermore, they will also utilise the chip to power some of its largest AI workloads on Azure, including parts of the multibillion-dollar partnership with OpenAI, where Microsoft powers all of OpenAI’s workloads. In addition, the software giant has revealed they have been collaborating with OpenAI on Maia’s designing and testing phases.
When asked about the partnership, Sam Altman, CEO of OpenAI, responded: “We were excited when Microsoft first shared their designs for the Maia chip, and we have worked together to refine and test it with our models. Azure’s end-to-end AI architecture, now optimised down to the silicon with Maia, paves the way for training more capable models and making those models cheaper for our customers.”
Lastly, at the Ignite Conference, Borkar revealed: “Maia is the first complete liquid-cooled server processor built by Microsoft. The goal was to enable a higher density of servers at higher efficiencies. Because we are reimagining the entire stack, we purposely think through every layer, so these systems are actually going to fit in our current data centre footprint.”
Azure Cobalt CPU
For the second chip revealed at the Ignite Conference, the Azure Cobalt CPU, named after the blue pigment is a 64-bit processor consisting of 128 computing cores and is built on an Arm Neoverse CSS design and customised for Microsoft. Moreover, its function is to power general cloud services on Azure and optimise to deliver greater efficiency and performance in cloud-native offerings.
In Borkar’s words, he explained: “We have put a lot of thought into not just getting it to be highly performant, but also making sure we are mindful of power management. We made some intentional design choices, including the ability to control performance and power consumption per core and on every piece of virtual machine.”
Furthermore, at the Ignite Conference, Borkar hinted that the company is already powering programs, including Microsoft Teams and Azure SQL, with plans to make virtual machines available to customers next year for their workloads. While he once again refrains from disclosing the full system specifications or benchmarks, he stated: “Our initial testing shows that our performance is up to 40 per cent better than what is currently in our data centres that use commercial Arm servers.”
Wes McCullough, CVP of the hardware product development, added: “The architecture and implementation are designed with power efficiency in mind. Therefore, we are making the most efficient use of the transistors on the silicon. Multiplying those efficiency gains in servers across all our data centres will add up to a pretty big number. It will definitely power new virtual machines for customers in the coming year.”
Rani Borkar Refuses to Draw Comparison Between Microsoft’s AI Chips and NVIDIA at Ignite Conference
Although Microsoft has succeeded in bringing out its own custom chips, Rani Borkar and many others at the Ignite Conference refuse to boast about their accomplishment and dare not draw comparisons with NVIDIA, the current juggernaut of high-performance AI chips. For example, Ben Bajarin, chief executive of analyst firm Creative Strategies, stated: “The Maia AI chip is not something that’s displacing NVIDIA.”
As for Rani Borkar, she was adamant about not discussing comparisons, instead reiterating that partnerships with NVIDIA and AMD are still very crucial for the future of Azure’s AI cloud. “At the scale at which the cloud operates, it’s vital to optimise and integrate every layer of the stack, maximise performance, diversify the supply chain, and frankly give our customers infrastructure choices.”
Lastly, she mentioned that Microsoft is developing these chips for its own Azure cloud workloads, not to sell to others like NVIDIA, AMD, Intel, and Qualcomm. Instead, the company will use them to power its own subscription software offerings and as part of its Azure cloud computing service, such as the quietly launched Copilot for Microsoft 365 for a $30-per-month premium per user during the Ignite Conference.
Thus, with Microsoft leaning towards the more standardisation route, would NVIDIA continue to reign supreme at the top of the chip wars hierarchy and will other tech giants follow in Microsoft’s footsteps? Let us know your thoughts on our Instagram and Twitter pages.