OpenAI

You’re reading an excerpt of Making Things Think: How AI and Deep Learning Power the Products We Use, by Giuliano Giacaglia. Purchase the book to support the author and the ad-free Holloway reading experience. You get instant digital access, plus future updates.

OpenAI, a research institute started by tech billionaires including Elon Musk, Sam Altman, Peter Thiel, and Reid Hoffman, had a new challenge. OpenAI was started to advance artificial intelligence and prevent the technology from turning dangerous. In 2016, led by CTO and co-founder, Greg Brockman, they started looking on Twitch, an internet gaming community, to find the most popular games that had an interface a software program could interact with and that ran on the Linux operating system. They selected Dota 2. The idea was to make a software program that could beat the best human player as well as the best teams in the world—a lofty goal. By a wide margin, Dota 2 would be the hardest game at which AI would beat humans.

At first glance, Dota 2 may look less cerebral than Go and chess because of its orcs and creatures. The game, however, is much more difficult than those strategy games because the board itself and the number of possible moves is much greater than the previous games. Not only that, but there are around 110 heroes, and each one has at least four moves. The average number of possible moves per turn for chess is around 20 and for Go, about 200. Dota 2 has on average 1,000 possible moves for every eighth of a second, and the average match lasts around 45 minutes. Dota 2 is no joke.

Figure: OpenAI team gathered at its headquarters in San Francisco, California.

Its first challenge was to beat the best humans in a one-on-one match. That meant that a machine only played against a single human controlling their own creature. That is a much easier feat than a computer playing versus a team of five humans (5v5) because playing one-on-one is a more strategic game and a much tougher challenge for humans. With a team, one player could defend and another attack at the same time, for example.

To beat the best humans at this game, OpenAI developed a system called Rapid. In this system, AI agents played against themselves millions of times per day, using reinforcement learning to train a multilayer neural network, an LSTM specifically, so that it could learn the best moves over time. With all this training, by August of the following year, OpenAI agents defeated the best Dota 2 players in one-on-one matches and remained undefeated against them.

Figure: OpenAI Matchmaking rating showing the program’s skill level for Dota 2 over time.

OpenAI then focused on the harder version of the game: 5v5. The new agent was aptly named OpenAI Five. To train it, OpenAI again used its Rapid system, playing the equivalent of 180 years of Dota 2 per day. It ran its simulations using the equivalent of 128,000 computer processors and 256 GPUs. Training focused on updating an enormous long short-term memory network with the games and applying reinforcement learning. OpenAI Five takes an impressive amount of input: 20,000 numbers representing the board state and what the players are doing at any given moment.*

Figure: The Casper Match at which OpenAI beat the Casper team.

In January of 2018, OpenAI started testing its software against bots, and its AI agent was already winning against some of them. By August of the same year, OpenAI Five beat some of the best human teams in a 5v5 match, albeit with some limitations to the rules.

So, the OpenAI team decided to play against the best team in the world at the biggest Dota 2 competition, The International, in Vancouver, Canada. At the end of August, the AI agents played against the best team in the world in front of a huge audience in a stadium, and hundreds of thousands of people watched the game through streaming. In a very competitive match, it lost against the human team. But in less than a year, in April 2019, OpenAI Five beat the best team in the world in twin back-to-back games at the International.*