Artificial Intelligence (AI) researchers and developers are continuously seeking new ways to create, train, test, and refine AI technology and potential. While specific tasks such as understanding speech, searching for information, and recognizing images are becoming much more manageable for AI, what is often referred to in the field as general intelligence is more difficult to teach and program. General intelligence includes all of the complex, layered, interactive, and subtle things humans are able to process, evaluate, and do moment to moment with great flexibility and creativity. Researchers at Microsoft have come up with a novel way to help AI evolve and learn in this direction—through the use of AI in the video game, Minecraft.

Project Malmo, Minecraft, and Artificial Intelligence

Minecraft is a hugely popular “sandbox game;” a term used to describe the open empowerment and lack of rules applied to players. Minecraft players use cubes to build what they like; collaborate, communicate, and fight with others; and go on unlimited adventures in survival, creative, or adventure modes. Minecraft’s creator, Swedish game designer Markus Alexej Persson, AKA Notch (his in-game name), is also the co-founder of Mojang game developer. Notch has been a featured creator at The Museum of Modern ArtSoProGaming explains that The PC version of this game is well known for its variety of mods (modifications that alter how a game originally worked). These mods are carried out by third parties and offer new characters, items, and activities within the game. Because Minecraft offers so much creative freedom to players, it is a great place for an AI-controlled character to roam, learn, and evolve. 

Katja Hofmann, a researcher at the Machine Intelligence and Perception group at Microsoft Research Cambridge, is the Research Lead for this initiative, called Project Malmo (previously named Project AIX). She has a Ph.D. in Computer Science from the University of Amsterdam and has focused much of her work on developing AI’s information search, retrieval, and recommendation abilities. Her long-term goal is “to develop AI systems that learn to collaborate with people, to empower their users, and help solve complex real-world problems.” Project Malmo is built as an experimentation platform on top of Minecraft with a mod for its Java version and code that helps the AI bot or agent to play. This two-part platform runs on Mac OS, Windows, and Linux and the AI players can be programmed in any language. Microsoft shares that Project Malmo integrates, “(deep) reinforcement learning, cognitive science, and many ideas from artificial intelligence.” Through reinforcement learning, AI agents are allowed to complete extensive trial and error explorations with a given task and will receive a reward for completing it successfully.

Project Malmo Goes Public

In the summer of 2016, Project Malmo became open source. Now, both beginning and expert programmers can try it on Git-Hub here. According to LiveScience, the new, publically launched version also offers the option to have chat interactions with the AI player. Computer scientists and others using Malmo can now create agents that can learn how to converse with people and each other. The new open source version also offers overclocking or speeding up of the game. This allows research experiments within the Project Malmo platform to be carried out a faster rate.

Collaboration and Education in Project Malmo

One of the most powerful attributes of Project Malmo for AI researchers is that it brings them together in a common, shared setting and open source environment. This is often preferable to working independently and trying to compare different, unconnected experiments, theories, algorithms, and results in realms such as symbolic AI, reinforcement learning, and other AI topics. Researchers can use Project Malmo to more easily work together, communicate, and brainstorm; because it reduces researcher isolation and increases experimentation speed, Project Malmo is a fertile platform for collaboration. 

Within the first few days after the project went public, it had over 20,000 views on GitHub. Another reason the public version of Project Malmo is popular is because of its applications in educational settings. Even novice coders can experience the AI system and the Microsoft Blog says that, “they’ve been pleasantly surprised to find that everyone from tweens with an early passion for programming to professors trying to train the next generation of AI researchers want to work with it as well.” Professor Jose Hernandez-Orallo of the Technical University of Valencia, Spain is using Project Malmo in his classes and he states that it will “have an impact in education, at least at the university level.” It’s easy to see that programming and experimenting in Minecraft might increase the amount of fun, novelty, and engagement for students of many ages and levels of experience. 

The hope is that as the AI agents or bots in Project Malmo take on challenges such as interacting and building with objects, they will also virtually explore, communicate, and learn in much the same way human-controlled agents do. As senior researcher Fernando Diaz puts it, “we’re trying to program it to learn, as opposed to programming it to accomplish specific tasks.”

Keep up with the Project Malmo researchers at the Microsoft Blog, where a recent entry offers insights from computer scientists on the subtlety and importance of collaborating to get their Minecraft agent to climb up a hill. If you try Project Malmo on GitHub, you can also check out this helpful Project Malmo community discussion forum here. Project Malmo Research Lead Katja Hofmann is on Twitter @katjahofmann

By Julia Travers