How AI in Esports is Changing Our Games

One of the overwhelming stories in science over the last decade is the huge breakthrough in machine learning spurred largely by neural networks and increased computational power. This revolution has also led to the use of machine learning and AI in esports and gaming.

From traditional games like Chess and Go, to real time strategy (RTS) games like Starcraft, and even to online poker, artificial intelligence has changed the landscape of esports. These games have been revolutionized by the influence of computers.

In this article, we’ll start by taking you through a short history of the AI/ML boom in the 15 years. Then we’ll talk about how machine learning techniques have revolutionized traditional strategy games like chess and go. We’ll later talk about the use of AI in esports more generally.

To receive email updates when new articles are posted, use the subscription form below!

The Machine Learning and AI Revolution

Everywhere you go today you hear about “machine learning this” or “artificial intelligence that”. In fact for research scientists like myself, the phrase “AI/ML” has become a bit of a punchline. Instead of doing classical science everyone wants to throw AI at a problem and solve it. Historically, the workflow of research was “hypothesis – lab testing – data analysis – conclusions”. But anymore it feels like research is conducted like “problem – ML – ??? – profit”.

This fact is even more frustrating because machine learning and AI techniques are actually extraordinarily good at what they do. Neural networks in particular are wildly good at solving problems in extremely diverse domains.. It’s a bit scary, actually, mostly because we don’t really know why they work. But boy do they work.

Neural networks are not new. One of the fundamental papers suggesting that neural networks might be useful in machine learning was published in 1989!

But then, in 2015 Yann LeCun, Yoshua Bengio, and Geoffrey Hinton published the seminal paper “Deep Learning” in Nature. Nature is widely known as hosting the most important papers in science, but even this paper was exceptional by Nature’s standards.

Convolutional neural networks were born with this paper and decades old computer vision problems saw significant progress. Image classification and speech recognition by computers in particular were problems that saw great leaps forward.

The other thing that allowed for the AI/ML revolution in the last 10 or 15 years was massive steps forward in computational power by CPUs and, even more importantly, GPUs (Graphics Processing Units). It is kind of funny – the same technology used in Adobe after effects and to play counterstrike on laptops led to massive scientific breakthroughs.

It wasn’t necessarily that there was a single breakthrough that led to massive increases in computer power. Rather, the cumulative effect of Moore’s law over the course of decades finally reached a point where extremely large neural networks became a feasible methodology instead of just an academic exercise.

Today neural networks are one of the biggest buzzwords in the world. If you have neural networks on your resume, you can work wherever you want. If your company “does neural networks”, they’re going to make money. It was natural to see machine learning progress to the point where AI in esports became a reality.

AlphaZero and Machine Learning in Chess

While chess is a “classical game”, to me it also counts as an esport. Chess-playing computers are as old as time. Indisputably, the most famous events in chess engine history were the matches between Gary Kasparov and Deep Blue in 1996 and 1997. Gary Kasparov was the best human in the world and Deep blue was the best chess engine in the world.

In the first match, Gary Kasparov was narrowly able to hold the computer off. However, in the 1997 rematch Deep Blue defeated the world champion. Computers never looked back (figuratively) and the competition between human and engine has never been close again.

In the last few years, chess engines have incorporated neural networks.

In 2017, Google’s machine learning subsidiary DeepMind developed a chess playing engine using neural networks. This program was called AlphaZero. The most incredible part about AlphaZero is that it was not told anything about what humans had learned about chess over the past centuries. Rather, it learned to play by playing against itself. And it got really good.

Before AlphaZero, most computer programs used human heuristics (or rules of thumb) to evaluate chess positions and decide on the best moves. For example, chess engines before AlphaZero were told that queens were more valuable than rooks and that bishops were more valuable than pawns. However, this information doesn’t exist in the rules of chess, it is something humans learned by playing a lot of chess over the years.

AlphaZero, though, wasn’t told anything except the rules of the game. Rather, AlphaZero was simply told to play against itself and learn what worked and what didn’t via lots of repetitions. For example, perhaps in one game AlphaZero captures a rook with a queen early on and subsequently loses. The program will then learn that trading the queen for the rook was not advantageous. That is, the program inferred the heuristics we already know to be true.

This style of learning is common in neural network literature and is called reinforcement learning. By striving for wins and avoiding losses, the network reinforces good moves and eschews bad ones. It’s a clever paradigm – playing against itself – and it turns out to work very well.

Machine Learning and Cheating in Chess

Machine learning is not only used in programming chess engines. AI in esports is also widely employed to detect cheating. If machine learning can do anything well, it can infer patterns that are tricky for humans to pick up on. Cheating causes a change in behavior and, therefore, creates recognizable patterns. This problem is ripe for AI/ML.

Cheating might not seem like a big deal in esports, but it really is. Cheating affects everyone in the process. Tournament organizers need to discourage cheating in order to ensure fair play. Competitors don’t want cheating because they themselves are the ones who will lose out. One of the biggest online Casinos in Michigan in particular wants to avoid cheating because it will negatively impact their clients’ interest in betting on the event.

Recently, Hans Niemann and Magnus Carlsen have been embroiled in a cheating scandal. Carlsen – the many time reigning world champion and undisputed best player in the world – publicly accused (ish) Niemann of cheating during live and online chess events. This accusation prompted Chess.com to confirm that their cheat detection software has found Niemann to be cheating in the recent past.

Though the Chess.com cheat detection algorithm is proprietary, I would be willing to bet that many different machine learning algorithms go into the determination.

AI for RTS Games

Chess isn’t the only strategy game that has seen AI make an impact. Nearly any game where decision making and pattern recognition is the key feature can see benefits from AI. While AI made an early impact in games like chess, more recently AI has made an impact in other esports games.

Classical games like Chess and Go are discrete. That means that at every step there are finitely many things that can happen. That means a computer knows what choices it has and can choose accordingly.

The opposite of being discrete is being continuous. Certain games present infinitely many options at each step which makes the computer’s job much harder. Think about first person shooter games. The choices a computer would have to make (where to aim, where to move, precisely when to shoot, etc.) are far more numerous in a first person shooter than choosing which piece to move in chess.

But that doesn’t mean there is no AI in esports and real time strategy (RTS) games. A great example comes from the same lab that created the novel chess AI. In 2019, DeepMind showcased their AI that plays StarCraft II, one of the longest running and most popular game in esports.

In addition to being “real-time” (or continuous as I called it above), the DeepMind team identified many aspects of the game that make it difficult for an AI to play. Two of these include (paraphrasing their own descriptions):

• Imperfect information – In chess, both players have all the information in the game. In poker, part of the fun is that different players have different information. In Starcraft, information is a resource that influences the gameplay. It can be difficult to make the best choice when you don’t have all the info that goes into choosing.
• Tradeoffs between immediate and long term value – StarCraft is a game where investing resources early can lead to more resources later in the game. It is not unlike saving money for retirement. However, if you save too much, you can die before you reap the benefits of your investments. Because AI in esports works by searching decision trees to make their decisions, events with long-term payoff can be hard to evaluate.

The AI, called AlphaStar, proved to be on-par with the best players in the world. However, the StarCraft AI only made it into the top 200 of the leaderboards unlike the Chess engines which far surpassed the best players. Still, the fact that esports AI’s can now become world class at the hardest games is impressive in itself.

The Future of AI in Esports

One last question I want to ask is this: so what? It is likely that soon all esports will be dominated by a superior engine. How will this change the games?

I want to focus on one way: changing the meta. (The “meta” refers to the overwhelming strategy taken by most players because it is seen to be the best in some sense).

When humans play games, they tend to repeat patterns, strategies, and motifs that have done well in the past. This often results in an incomplete exploration of the space of strategies for the game. That is, there might be good strategies that nobody has thought of or tried simply because “if nobody is doing it, it must be bad”.

But computers are not risk averse. Computers have no sense of risk. They simply do what they have computed to be the best. The result is new ideas, strategies, and motifs that get incorporated back into the human meta.

In chess, for example, the dawn of superhuman computer engines has resulted in a complete change in gameplay. From the King’s gambit being refuted to the French defense rapidly losing popularity, engines have completely revamped how professional chess is played at the top level.

So, instead of considering AI in esports a bad thing, I like to think of it as a tool to produce new ideas and creativity to be incorporated by humans. Another way to say this is that AI in esports is another tool that humans can use to their own benefit. While traditional sports may benefit in the future from AI and ML through, for example, better analysis of tracking data, the AI revolution in esports has already arrived.

To receive email updates when new articles are posted, use the subscription form below!