Microsoft Xbox Unveils a Highly Unconventional AI Model for ‘Gameplay Ideation’

Just recently, Microsoft Xbox introduced a fascinating project called Muse, described as “a generative AI model crafted for gameplay ideation.” Accompanying this unveiling were an open-access article on Nature.com, a detailed blog post, and a YouTube video to boot. But what exactly is “gameplay ideation”? In Microsoft’s terms, it’s about generating game visuals, controller actions, or both. However, its actual application seems fairly niche and doesn’t simplify the game development process dramatically.

Nevertheless, the data surrounding this is indeed intriguing. The team trained Muse at scale using H100 GPUs, requiring roughly one million training updates to transform one second of real gameplay into nine additional seconds of simulated gameplay that is responsive and accurate to the game engine. Interestingly, the training data mainly came from existing multiplayer game sessions.

Rather than running the game on a single computer, Microsoft tackled this by using a cluster of 100 Nvidia H100 GPUs. This approach is far more costly and energy-intensive but yields an output resolution of only 300×180 pixels for those extra nine seconds of extended gameplay.

A highlight of the Muse demonstration was its ability to duplicate existing props and enemies in the game environment, imitating their functionality. Yet, with all the investment in hardware, energy, and AI training, one might wonder why not simply use conventional development tools to introduce enemies or props?

Indeed, while Muse successfully replicated object permanence and preserved the behavior from the original game, its practical applications appear inefficient when compared to the traditional methods used in video game development.

As for future possibilities, Muse could evolve to achieve more captivating results. However, it falls into a larger category of projects aiming to simulate gameplay entirely via AI. Although it manages to retain a degree of engine accuracy and object permanence, using this as a development or testing tool seems far from ideal. After thoroughly examining the materials, it’s challenging to grasp why anyone would prefer this method for creating or playing games.

If you’re keen to keep up with the latest in gaming tech, remember to subscribe to Tom’s Hardware’s amazing newsletter for top news and detailed reviews delivered right to your inbox.