Microsoft Research has created Muse, a first-of-its-kind generative AI model that can generate a game environment based on visuals or players’ controller actions. It understands a 3D game world and game physics and can react to how players interact with a game.

Microsoft debuted Muse, a generative AI model that can be trained to output video game visuals and predict controller inputs. The company describes the technology as the future of gameplay ideation, not as something to be used in place of human creativity during game development.

Microsoft views the Muse AI model more as a tool for game preservation and concept brainstorming than a way to replace human-made video games.