The Past and Present of Machine Learning in Gaming

Machine learning represents a crucial area of artificial intelligence (AI). Over time, machine learning can teach a piece of software how to behave when it encounters certain types of environments or problems. While some scientists use machine learning to advance the future of AI, others have focused on exploring how machine learning can function within artificial worlds. In other words, these scientists introduce machine learning to video games to see if the AI can learn how to play.

Early Attempts at Machine Learning in Gaming
Computer scientists have been testing the abilities of machine learning in video games for over 60 years. In 1949, Claude Shannon published a research paper titled “Programming a Computer for Playing Chess.” Shannon’s paper estimated that chess has more than 10^120 possible positions. Even today’s supercomputers would find it impossible to solve chess problems with brute force instead of playing against their opponents.

By 1997, IBM had developed a computer called Deep Blue that managed to defeat world chess champion Garry Kasparov in a six-game match. This was the first time that AI had beaten a human in chess.

It took IBM several years to teach Deep Blue how to play chess well enough to compete against a master. Early versions of the AI couldn’t even beat basic chess-playing software run on a personal computer. Over time, though, Deep Blue learned how to use the rules of chess to its advantage. Like a human player, it discovered that it could predict and respond to its opponent’s moves. It may make some people uncomfortable to think about Deep Blue in this way, but the computer was thinking about how to beat its opponent. It wasn’t just following rules.

More recently, AI researchers introduced machine learning to “Super Smash Bros.,” a hand-to-hand fighting game that features some of Nintendo’s most popular characters. The researchers introduced four rules about the game’s goals, strategies, tactics, and chains of button presses to see how the AI would perform.

The AI excelled at beating opponents in “Super Smash Bros.” Even the world’s best players faced great difficulty competing against the AI. In fact, the AI performed so well that few human players managed to hit its game character once. (Readers can see videos of this at GitHub.)

Adding Evolutionary Genetics to AI

In 2015, Seth Bling revealed that he had created an AI called MarI/O that learned to beat a level of “Super Mario World” in just 34 tries. Bling gave the AI neural networks and algorithms so it could decide which attempts gave it the information it needed to move forward in the game. Other than that, Bling gave it nothing. MarI/O didn’t even know to press left to move left. It had to discover that on its own.

Bling’s project was revolutionary because it used aspects of evolutionary genetics. Bling wanted to see how the AI’s approach to playing would evolve. He found the “species” that worked best would dominate the AI’s progress. Eventually, the AI had taught itself everything that it needed to complete a level successfully.

Introducing AI to Games of Imperfect Information

Chess and “Super Smash Bros.” are considered games of perfect information because both players have full access to the rules and the board. Other than an opponent making a particularly inventive move, no one encounters surprises in chess. Other games of perfect information include backgammon, checkers, tic-tac-toe and Go.

Games of imperfect information operate in a more realistic way that can represent how the real world functions, or at least how the rules of a fictional world function. With games of imperfect information, players have to make quick decisions. Players also only have access to certain aspects of the game. For instance, Player A may not have the ability to see Player B’s board. This creates more uncertainty for machine learning to handle.

Current thinking suggests that exposing machine learning to games of imperfect information could lead to significant improvements in technologies like autonomous cars. Operating a car safely requires an understanding of real-world physics as well as an ability to respond to unexpected events.

Somewhat surprisingly, “Minecraft” may offer one of the best environments for AI to learn in. The “Minecraft” universe follows specific rules that are very similar to those created by Earth’s physics. Within the game, though, AI can make mistake after mistake without harming anything. If it made those same mistakes while driving a car, companies would end up spending a lot of cash on replacement cars and driving courses.

Machine learning has already shown humans how impressive artificial intelligence can become when given enough time to train itself. In the near future, researchers may use video games to teach AI how to perform all manner of tasks and recognize objects that exist in the world.

No one should expect AI to find the same solutions that humans developed, though. AI and humans do not experience environments in the same ways. This reality could cause problems for researchers who want to make AI a part of the real world.