Code Bytes: There are plenty of artificial intelligence agents that have learned to beat video games, but now a team of scientists have taken it one step further by creating an AI that can actually make its own games just by watching someone play them.

But what if there is an AI which doesn’t require anything to build a game, not even the source code. Researchers at the Georgia Institute of Technology are running an experiment where the AI analyzes Super Mario gameplay and recreate the game engine on its own.

“Our AI creates the predictive model without ever accessing the game’s code, and makes significantly more accurate future event predictions than those of convolutional neural networks,” said Matthew Guzdial, lead researcher on the project and a Ph.D. student in computer science. “A single video won’t produce a perfect clone of the game engine, but by training the AI on just a few additional videos you get something that’s pretty close.”

The research team used Nintendo’s classic “Super Mario Bros.” to train the AI using videos of two different play styles: a “speedrunner” play style, where a player rushes through the game as fast as possible, and an “explorer” play style, where a player spends time exploring each level to find powerups or hidden areas. Based on this data, the AI was able to recreate several of the game’s core mechanics, including jumping, defeating enemies, avoiding pits and other environmental hazards and so on. The AI also learned more specific rules about how certain mechanics function, such as Mario being unable to jump again once he is already in the air.

“The technique relies on a relatively simple search algorithm that searches through possible sets of rules that can best predict a set of frame transitions,” explained Mark Riedl, associate professor of interactive computing and co-investigator on the project. “To our knowledge this represents the first AI technique to learn a game engine and simulate a game world with gameplay footage.”

The system with limited capacity is nascent for now, but in the future, it might be able to handle 3D game engines. Moreover, the researchers want their AI to be efficient enough for a full game generation. And possibly, their AI could be improved to interpret real life situations. Probably, someday, to the extent humans understand it.

“Intelligent agents need to be able to make predictions about their environment if they are to deliver on the promise of advancing different technology applications,” said Guzdial. “Our model can be used for a variety of tasks in training or education scenarios, and we think it will scale to many types of games as we move forward.”