Tanish PatelIn 2019, the DeepMind team in collaboration with Blizzard Entertainment introduced AlphaStar, a machine learning program to reach the level of the best players in the esports discipline. How did it change the world of machine learning and does it put an end to the esports scene today?
What is AlphaStar?
AlphaStar is the successor to AlphaGo, the program that defeated humanity in the game of go. Like AlphaGo, it was developed by the DeepMind team, led by Demis Hassabis.
But the new project was led by another artificial intelligence specialist from Google Brain, Oriol Vinyals, who participated in the development of machine translation and speech recognition services for the American company.
Image source: https://pixabay.com/illustrations/artificial-intelligence-brain-think-3382507/
Both Google Brain and DeepMind are primarily engaged in experimental projects without any practical use.
In addition to AI game systems, DeepMind’s famous projects include WaveNet, a neural network speech synthesis system, and medical initiatives, such as an experimental system for making diagnoses by analyzing images in ophthalmology.
In fact, Google Brain engineers often engage in projects that are closer to art than research experiments.
As Vinyals points out, the development of the StarCraft bot may be useful for managing robots, personal assistants, autonomous vehicles, and in general to influence the future of AI.
How StarCraft is different from chess and Go?
Despite the common ideas of the two projects, AlphaStar is very different from AlphaGo, just as these games are different from each other.
First, StarCraft belongs to a completely different class of games. It is a game with closed information. The difference is that in open-information games like chess, checkers (long beaten by AI), go (recently beaten by AI), and many others, everyone knows everything about what’s happening on the playing field.
Chess players always see the board and amount of pieces, the only thing impossible to predict is the next move of the opponent.
Image source: https://unsplash.com/photos/nAjil1z3eLk
In most card games and StarCraft, on the other hand, players have no information about their opponents. In such games it is usually much more difficult to predict the pace of the game with brute force, than in games with open information – the uncertainty is much higher.
Of course, not all games with open information can calculate all future moves. In Go, the number of possible combinations of stones on the board exceeds the number of atoms in the universe.
StarCraft is very different in this respect. It’s practically useless to calculate your opponent’s moves. So AlphaStar doesn’t even try to do that; instead, it learns how to find a winning strategy based on the statistics of other players’ games that have already been played.
Compared to Go, StarCraft has a few important differences: it’s a real-time game where everyone is playing at the same time.
The outcome becomes clear after thousands of small decisions, and it’s very difficult to predict who has the advantage because you don’t know how your opponent is playing.
Therefore, in StarCraft, teaching the AI to distinguish right moves from wrong moves based solely on the outcome of the match is nearly impossible, it would take years, if not millennia, to play.
Image source: https://unsplash.com/photos/xdXxY5C9PUo
This is when reinforcement learning comes into play.
In AI development services, reinforcement learning is a technique that has been around for decades. There are high expectations for it now.
Google DeepMind, for example, has relied almost entirely on reinforcement learning combined with deep learning on neural networks.
Generally speaking, these two techniques are quite different. They appeared and developed independently of each other. But now they are often tried to be combined into one system, and there is great hope that they should work well together.
As for the role of computer games in AI research (chess, go, Dota 2, and now StarCraft), they are now used as a convenient testing ground, where game conventions and rules help a lot to create something that works. Real-life and any business tasks are, of course, much more complicated.
But if you try to imagine which real-world tasks have the best chance of benefiting from research and application of reinforcement learning, it would probably be robot learning and unmanned driving.
Can AlphaStar outplay humans?
So should gamers be worried? Do players have any chances against AlphaStar in RTS games?
The answer is probably maybe (assuming the machine has some limitations that bring it closer to human capabilities). Unfortunately, as AlphaStar does not exist anymore, we can’t answer this question.
Although we have seen numerous big victories against the best pro players, many people are still skeptical about AlphaStar’s potential.
First of all, the biggest strength of AlphaStar is its ability to micro units and stunning APM (action per minute). Although engineers put some restrictions on the AlphaStar in terms of APM, players still spotted some unfair moves that allowed viewing the whole map just in milliseconds.
However, StarCraft at its highest level is not only a game of brain speed and APM, but also of maximum depth in strategy, tactics, and mind games.
Also, on the long-distance, it’s not clear how AlphaStar would adapt to the new meta and changes. StarCraft and other RTS games always see new updates, which will require AI to relearn.
Image source: https://bnetcmsus-a.akamaihd.net/cms/blog_header/2g/2G4VZH5TIWJF1602720144046.jpg
That also works in the opposite direction; we don’t know how people would adapt to AlphaStar’s strategies.
A perfect example is Dota 2, which also introduced its machine algorithm OpenAI founded by Elon Musk. Here we could see how people defeated the machine after all.
After all, the gaming community found many loopholes to easily beat OpenAI in the 1vs1 scenarios.
It’s also worth noting that the goal of DeepMind was never to show the world that a machine can annihilate players (no one doubts that the machine has a greater computational potential). The goal of DeepMind was to improve or test the machine’s ability to learn, instead of following programmed scenarios.
And we can clearly see that they achieved their goals: today machine learning is widely used in many fields.
Reinforcement learning is effective primarily where the system needs to analyze the environment and choose a behavioral policy based on the feedback it receives.
It is used in areas such as:
- Planning
- Perception systems
- Bots for computer games
- Crypto and Trading bots
- Natural language processing
- Medicine
This is why you should not put AlphaStar and other programs in opposition to humans.
Of course, humans will never be able to achieve the computational abilities of a machine.
But did chess become extinct after AI beat all the top chess players? Do we really enjoy listening to music produced by artificial intelligence?
Still, although the world of machines and humans is closely intertwined, we live in different universes, and it is unlikely that the computational abilities of machines will ever surpass the beauty of human thought.
About the author:
Robyn McBride is a journalist, tech critic, and author of articles about software, AI, and design. She is interested in modern image processing, tech trends, and digital technologies. Robyn also works as a proofreader at Computools.
Jassica Dean
Ginny D. Adams
Linda
United States
United Kingdom
India
Canada
Singapore
Add comment