These five algorithms worked together to beat humans at a video game

On Monday, non-profit AI research company OpenAI published a blog post about OpenAI Five, a group of five neural networks designed to work as a team while playing the real-time computer strategy game called Dota 2. According to the post, OpenAI Five can now beat a team of five human amateur players at the game, albeit with specific restrictions placed on gameplay. In August, it will attempt to beat a team of professional Dota 2 players at The International (TI), an annual Dota 2 tournament hosted by the game’s developer, Valve Corporation.

Team algorithm

In Dota 2, two teams of five players battle to destroy the other team’s “Ancient,” a structure at the center of their base. Each player controls a different character, known as a “hero.” These heroes have their own abilities, strengths, and weaknesses, and a team’s ability to cooperate is key to its success. The developers assigned each OpenAI Five algorithm a specific hero, placing restrictions on the characters to account for areas of the game they hadn’t integrated.

Practice makes perfect

OpenAI Five trained first as individual algorithms in one-vs-one matches and then as a team by playing against itself and past versions of itself. As a team, it collected 180 years worth of experience during each day of training, eventually picking up on strategies typically used by professional Dota 2 players. None of the algorithms could communicate with one another — cooperation was simply one of each algorithm’s incentives. This cooperation continued when a human player replaced one of the algorithms.

The heroes we need

With OpenAI Five, we could be seeing a preview of the future of AI. In Dota 2, each algorithm has to choose between roughly 1,000 potential moves every one-eighth of a second — far more complex than a game like Go, in which an AI has to choose between 250 moves at a time. Algorithms usually don’t operate as teams, either, but this Dota 2 project shows they’re more than capable of cooperating with each other — and humans, too.