“SIMA takes one step further and shows stronger generalization to new games,” he says. “The number of environments is still very small, but I think SIMA is on the right track.
A New Way to Play
SIMA shows DeepMind putting a new twist on game playing agents, an AI technology the company has pioneered in the past.
In 2013, before DeepMind was acquired by Google, the London-based startup showed how a technique called reinforcement learning, which involves training an algorithm with positive and negative feedback on its performance, could help computers play classic Atari video games. In 2016, as part of Google, DeepMind developed AlphaGo, a program that used the same approach to defeat a world champion of Go, an ancient board game that requires subtle and instinctive skill.
For the SIMA project, the Google DeepMind team collaborated with several game studios to collect keyboard and mouse data from humans playing 10 different games with 3D environments, including No Man’s Sky, Teardown, Hydroneer, and Satisfactory. DeepMind later added descriptive labels to that data to associate the clicks and taps with the actions users took, for example whether they were a goat looking for its jetpack or a human character digging for gold.
The data trove from the human players was then fed into a language model of the kind that powers modern chatbots, which had picked up an ability to process language by digesting a huge database of text. SIMA could then carry out actions in response to typed commands. And finally, humans evaluated SIMA’s efforts inside different games, generating data that was used to fine-tune its performance.
After all that training, SIMA is able to carry out actions in response to hundreds of commands given by a human player, like “Turn left” or “Go to the spaceship” or “Go through the gate” or “Chop down a tree.” The program can perform more than 600 actions, ranging from exploration to combat to tool use. The researchers avoided games that feature violent actions, in line with Google’s ethical guidelines on AI.
“It’s still very much a research project,” says Tim Harley, another member of the Google DeepMind team. “However, one could imagine one day having agents like SIMA playing alongside you in games with you and with your friends.”
Video games provide a relatively safe environment to task AI agents to do things. For agents to do useful office or everyday admin work, they will need to become more reliable. Harley and Besse at DeepMind say they are working on techniques for making the agents more reliable.
Updated 3/13/2024, 10:20 am ET: Added comment from Linxi “Jim” Fan.