AlphaGo: AI Defeating World Champion at Go

In March 2016, a program called AlphaGo beat Lee Sedol, a winner of 18 international Go titles, four games to one in Seoul. Only five months earlier, the same system had swept the European champion Fan Hui 5-0, and the paper describing that result called it a feat "previously thought to be at least a decade away." AlphaGo did not win by calculating more moves than a human. It won by learning which moves were worth calculating at all, using two neural networks to steer a search.

This article walks through how that system was built, what actually happened in the five games, and how its successors, AlphaGo Zero and AlphaZero, dropped the human data entirely.

Why Go broke the old playbook

Computer chess had a clear recipe by the 1990s: search a huge tree of possible moves, score the leaves with a hand-written evaluation function, and prune branches that cannot matter. When IBM's Deep Blue beat Garry Kasparov 3.5 to 2.5 in May 1997, it was evaluating about 200 million positions per second on 32 processors working in parallel, according to IBM's history of the machine.

Go defeated the same recipe for two reasons, and both show up in the numbers. The first is sheer size. The 2016 Nature paper by David Silver and colleagues gives rough values for how many moves are available per turn (the breadth, b) and how many turns a game lasts (the depth, d): about b ≈ 35 and d ≈ 80 for chess, and about b ≈ 250 and d ≈ 150 for Go. Raising 35 to the 80th power gives roughly 10^123 possible lines of play. Raising 250 to the 150th power gives roughly 10^360. Those exponents are arithmetic on the paper's estimates, not counts of anything a computer could enumerate.

The second reason is that nobody could write down a good evaluation function. In chess, material (a queen is worth about nine pawns) gives a crude but useful score. In Go, every stone is identical, and whether a group of stones is alive, dead, or merely weak depends on the whole board. A position can look balanced for 100 moves and then turn on a single exchange.

A Go board has 361 intersections. In January 2016, John Tromp finished computing how many of the possible arrangements of stones are legal positions: about 2.08 x 10^170, or roughly 1.2 percent of all arrangements. The commonly quoted estimate for the number of atoms in the observable universe is around 10^80, so the count of Go positions is not just larger but larger by about 90 orders of magnitude. Source: John Tromp's Go counting page.

Before AlphaGo, the best Go programs relied on a technique called Monte Carlo tree search (MCTS), which scores a position by playing many fast, semi-random games to the end and counting who wins. According to Wikipedia's history of computer Go, the approach lifted programs to low-dan amateur ranks by 2009, and in 2013 the program Crazy Stone beat the 9-dan professional Yoshio Ishida while receiving a four-stone handicap. A four-stone head start is a large gift. Programs were strong amateurs, and no better.

Chess versus Go at a glance

Property	Chess	Go	Source
Typical moves available per turn (b)	about 35	about 250	Silver et al., Nature, 2016
Typical game length in moves (d)	about 80	about 150	Silver et al., Nature, 2016
Rough lines of play (b to the power d)	about 10^123	about 10^360	Arithmetic on the figures above
Legal board positions	not compared here	about 2.08 x 10^170	Tromp, 2016
Piece values	Differ by piece	All stones identical	Rules of the game
First strong result against a top human	1997 (Deep Blue)	2016 (AlphaGo)	IBM; DeepMind

The point of the table is not that Go is "harder" in some mystical way. It is that the two levers chess programmers pulled, deeper search and better hand-built scoring, both stopped working at once.

The four pieces inside AlphaGo

The system described in the 2016 paper has four learned components, trained in stages. It helps to think of a strong human player's two instincts. One is the flash of "these three or four moves look right," which narrows the options. The other is a gut sense of "I am ahead" or "I am behind" when looking at a board. AlphaGo gives each instinct its own neural network, then adds a search that checks the instincts against calculation.

The networks are convolutional neural networks, the same family used for image recognition. The board goes in as a 19 x 19 grid, like a photograph with 361 pixels, and layers of small filters pick out local patterns such as groups, liberties (empty points next to a stone), and shapes. For background on that building block, see how neural networks work and neural networks and deep learning.

The supervised policy network: learning to imitate experts

A policy network answers one question: given this position, how likely is each legal move to be the one a strong player picks? The first version was trained by supervised learning (learning from labeled examples) on 30 million positions from the KGS Go Server, a site where strong amateurs play online. The 13-layer network predicted the expert's actual move 57.0 percent of the time on held-out positions. The best earlier published result at the time of submission was 44.4 percent.

A gap of a dozen points in prediction accuracy sounds modest, but the paper reports that small accuracy gains translated into large gains in playing strength. The authors tried networks with 128, 192, 256, and 384 filters per layer and found that wider networks predicted better and played better, though they were slower to evaluate during search.

The rollout policy: speed over accuracy

Search needs a very fast way to play out games. So the team also trained a tiny linear model on small pattern features. It predicted expert moves only 24.2 percent of the time, but it needed just 2 microseconds to choose a move, compared with 3 milliseconds for the 13-layer network. That is roughly 1,500 times faster. Fast and rough is the right trade when you need to simulate thousands of games.

The reinforcement learning policy: playing itself

Imitating humans caps you at human habits. To push past that, DeepMind copied the supervised network and let it play games against earlier versions of itself, rewarding it only for winning (+1) or losing (-1) at the end of the game. The weights moved by policy gradient, a method that makes the moves from winning games a bit more likely and those from losing games a bit less likely. The opponent was picked at random from a pool of earlier versions to stop the network from overfitting to one rival. For a deeper look at the general method, see our explainer on reinforcement learning.

The improved network won more than 80 percent of its games against the supervised network. With no search at all, it beat Pachi, a strong open-source Go program that runs 100,000 simulations per move, in 85 percent of games. The supervised network alone had won only 11 percent against Pachi.

The value network: judging who is ahead

The fourth piece is the one that most resembles intuition. A value network takes a position and outputs a single number: the predicted chance of winning. Building it ran into a subtle trap that the paper describes openly.

Training first on whole games from the human KGS database, the network memorized the games instead of learning to evaluate. Successive positions from one game are nearly identical, differing by a single stone, yet they share the same winner label. The network reached a mean squared error of 0.19 on its training positions but 0.37 on new ones, a sign of overfitting.

The fix was to generate fresh data. The team made 30 million distinct positions, each sampled from a separate self-play game, so no two training examples came from the same game. With that dataset the error was 0.226 on training and 0.234 on test positions, which means the network was generalizing. The paper adds that a single evaluation by the value network came close to the accuracy of running Monte Carlo rollouts with the strong reinforcement learning policy, while using about 15,000 times less computation.

The value network illustrates a general lesson in machine learning: if your examples are highly correlated, a model can ace its practice tests and still have learned nothing useful. AlphaGo's team fixed the problem by changing how the data was generated, not by changing the model.

How the search combines the networks

All four pieces feed a Monte Carlo tree search. Picture a decision tree where each branch is a move and each node is a position. Ordinary MCTS grows the tree unevenly, spending more time on branches that look promising. AlphaGo changes what "promising" means.

Each simulation does the following:

1. Select. Starting from the current position, walk down the tree by repeatedly choosing the move with the highest score, which is the average result so far plus a bonus. The bonus is proportional to the policy network's prior probability for that move and shrinks as the move is visited more often, so good-looking but rarely tried moves get a chance.
2. Expand. When the walk reaches a position never seen before, add it to the tree and ask the supervised policy network for move probabilities.
3. Evaluate. Score the new position in two independent ways: the value network's estimate, and the result of a quick rollout using the fast policy. The paper blends them with a mixing weight, set to 0.5 in the final system.
4. Back up. Push the result up the path so that every position along the way updates its average.

After many simulations, AlphaGo plays the move that was visited most often, not the one with the highest average. Visit count turns out to be a sturdier signal than a noisy average.

The authors tested the mix. A version using only the value network (weight 0) already beat every other Go program they tried. A version using only rollouts did worse. The blended version performed best, winning at least 95 percent of games against the single-component variants. The two evaluation methods are complementary: the value network approximates how games played by the strong policy tend to end, while rollouts check concrete tactics through actual play-outs.

One more detail from the paper is worth keeping. The supervised policy network worked better inside the search than the stronger reinforcement learning one. The authors suggest that humans choose a wide set of plausible moves, which gives search more to work with, while the self-play network concentrates on a single best move. The reinforcement learning network's main job was producing the games that trained the value network.

An analogy: the apprentice and the surveyor

Think of a master carpenter hired to inspect a house. One apprentice (the policy network) has seen thousands of houses and instantly points at the three walls most likely to hide a problem. A second apprentice (the value network) glances at the floor plan and says "this one is sound" or "this one is trouble." The master then opens up only the flagged walls and checks them by hand (the rollouts). Without the apprentices, the master would have to open every wall in the building.

Hardware: how much computer was needed

AlphaGo ran in two forms. The paper says the final single-machine version used 40 search threads, 48 CPUs, and 8 GPUs. A distributed version used 1,202 CPUs and 176 GPUs. The distributed version won 77 percent of games against the single-machine version and 100 percent of games against other programs.

Version	CPUs	GPUs	Result reported
Single machine (Nature paper)	48	8	494 wins in 495 games (99.8%) against other Go programs
Distributed (Nature paper)	1,202	176	77% against the single-machine version; 5-0 against Fan Hui
Lee Sedol match	Not stated by DeepMind in the Nature paper	Not stated	The Economist reported 1,920 CPUs and 280 GPUs, per Wikipedia's account of the match

Those internal tournament games gave each program about five seconds per move. On handicap games, where AlphaGo gave opponents four free stones, it still won 77, 86, and 99 percent against Crazy Stone, Zen, and Pachi respectively.

The paper also notes how selective the search was compared with Deep Blue. AlphaGo evaluated thousands of times fewer positions than Deep Blue did in its chess match against Kasparov, and compensated by choosing positions more intelligently.

The Fan Hui match, October 2015

Fan Hui, a professional ranked 2 dan and the winner of the European championships in 2013, 2014, and 2015, played a formal five-game match against the distributed AlphaGo from October 5 to 9, 2015. AlphaGo won 5-0, the first time a computer program had beaten a professional on a full-size board without a handicap. The Nature paper describing the result is dated January 28, 2016.

A 2-dan professional is a strong player but well below the top of the world rankings, so the result did not end the argument. It set up the match with Lee Sedol, the player widely considered the strongest of his decade.

Seoul, March 2016: the five games

The Lee Sedol match was played in Seoul from March 9 to 15, 2016, under a one million dollar prize pool. The time control was two hours of main time per player plus three 60-second overtime periods, with Chinese rules and a 7.5-point komi (the compensation given to White for moving second). DeepMind said that after AlphaGo's win, the prize money would go to charities including UNICEF and Go organizations. Lee received a $170,000 appearance fee and win bonus, according to Wikipedia's account of the match.

Game	Date (2016)	Black	White	Result	Moves
1	March 9	Lee Sedol	AlphaGo	Lee resigned	186
2	March 10	AlphaGo	Lee Sedol	Lee resigned	211
3	March 12	Lee Sedol	AlphaGo	Lee resigned	176
4	March 13	AlphaGo	Lee Sedol	AlphaGo resigned	180
5	March 15	Lee Sedol	AlphaGo	Lee resigned	280

Source: Wikipedia, AlphaGo versus Lee Sedol. All five games ended by resignation. Colors were meant to be assigned randomly for Game 5, but Lee asked to play Black after Game 4, and DeepMind's Demis Hassabis agreed.

DeepMind's own page on the project says the match was watched by more than 200 million people worldwide. Wikipedia's account is more granular, reporting about 60 million viewers of Chinese-language coverage of Game 1 and an English stream that averaged 80,000 concurrent viewers.

Game 2 and Move 37

In Game 2, AlphaGo played its 37th move, a shoulder hit (a stone placed diagonally next to an opponent's stone). Michael Redmond, a 9-dan professional commentating the match, called the move "creative" and "unique," and An Younggil, an 8-dan professional, described it as "a rare and intriguing shoulder hit." DeepMind's project page describes it as a move with a 1 in 10,000 chance of being used.

That number measures how unusual the move was, not how weak. In the architecture described above, an unusual move simply starts with a low prior, which means the search is not pointed at it first. Given enough simulations, the value network and rollouts can still discover that it is good. That is the combination working as designed: the prior suggests where to look, and calculation is free to overrule it.

A quote often attached to Move 37 is that Lee Sedol said "this was not a human move." Wikipedia's account of the match does not carry that line, so we do not repeat it as fact here. What the record does support is Lee's remark after Game 2, "AlphaGo played a nearly perfect game," reported by the BBC.

Game 4 and Move 78

Lee's one win came from a move that now has its own name. His white move 78 in Game 4 was described as "a brilliant tesuji" (a clever tactical move) and Gu Li, a 9-dan professional, called it a "divine move." DeepMind's page notes that Lee's move also had roughly a 1 in 10,000 chance of being played, and that it is known as "God's Touch."

AlphaGo's reply at move 79 was poor, even though it estimated its own chance of winning at about 70 percent. Around move 87 its win estimate fell sharply and it played a run of weak moves through move 101. One analyst cited in Wikipedia's account, Ormerod, suggested the cause was a typical Monte Carlo weakness: a key line was "off the search radar" because the search had pruned it. AlphaGo resigned when its estimated chance of winning dropped below 20 percent.

This episode is the most useful corrective to the "invincible machine" story. A system that allocates search by learned priors can be blind to a line its priors rate as unlikely. Lee found one such line, and the program's confident evaluation then collapsed.

What Lee said

Lee Sedol's comments after the match, as attributed on Wikipedia, ranged widely. After Game 3 he said he had "misjudged the capabilities of AlphaGo and felt powerless," and that it was "Lee Se-dol's defeat" and "not a defeat of mankind." He called his Game 4 win a "priceless win" that he would not exchange for anything. On November 19, 2019, Lee announced his retirement from professional play, saying he could never be the top overall player because of AI and describing AI as "an entity that cannot be defeated."

After Seoul: Master and Ke Jie

DeepMind kept going. A stronger version called AlphaGo Master played online games under pseudonyms in late 2016 and early 2017. As of January 5, 2017, its online record was 60 wins and no losses. In May 2017 in Wuzhen, China, Master played the world number one Ke Jie in a three-game match and won all three. After that match, AlphaGo retired, and the Chinese Weiqi Association awarded it a professional 9-dan ranking, the top rung of the professional ladder.

AlphaGo Zero: dropping the human data

The October 2017 paper on AlphaGo Zero made a stronger claim than the original. Silver and colleagues wrote that the program is "trained solely by self-play reinforcement learning, starting from random play, without any supervision or use of human data." Four design changes, taken from the paper, separate it from AlphaGo Fan and AlphaGo Lee:

• No human games. Training started from random moves.
• Minimal input. The network sees only the positions of black and white stones.
• One network. A single network outputs both move probabilities and a value, rather than two separate networks.
• No rollouts. The search relies only on the network, with no fast random playouts.

The central idea is that search is placed inside the training loop. In each position, a tree search guided by the current network produces move probabilities that are usually much stronger than the raw network's own. The network is then trained to match those improved probabilities and to predict the eventual winner of the game. A stronger network makes the next round of search stronger, and so on. The paper describes search as a "policy improvement" step and self-play as a "policy evaluation" step.

The numbers from the paper

Quantity	Value
Training time reported	About 3 days
Self-play games generated	4.9 million
Simulations per move during self-play	1,600 (about 0.4 seconds of thinking)
Parameter updates	700,000 mini-batches of 2,048 positions
Network depth	20 residual blocks
Time to pass AlphaGo Lee	36 hours
Match result against AlphaGo Lee	100-0, under the same 2-hour time controls as the Seoul match
Hardware at match time	One machine with 4 TPUs, versus 48 TPUs for AlphaGo Lee

A TPU is a tensor processing unit, a chip Google designed for neural network arithmetic. Using one machine with four of them against a distributed system with 48 is a compact way to show how much of the gain came from the algorithm rather than from raw hardware.

The paper also compared the self-taught network with one trained to imitate human experts using the same architecture. The imitation network predicted professional moves better, yet the self-taught player beat it within the first 24 hours of training. The authors read this as a sign that AlphaGo Zero was learning a strategy "qualitatively different to human play." In a separate comparison on a fixed dataset, switching from the older convolutional architecture to residual blocks raised playing strength by over 600 Elo, and merging the policy and value outputs into one network added around another 600. Elo is a rating scale in which a 230-point gap, per the 2016 paper's tournament figure, means roughly a 79 percent chance of winning.

DeepMind's blog post on AlphaGo Zero adds that after about 40 days of training, the system outperformed AlphaGo Master, the version that had beaten Ke Jie.

AlphaZero: one algorithm, three games

In December 2017, the same group posted an arXiv preprint, "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm." The new system, AlphaZero, applied nearly the same method to chess, shogi (Japanese chess), and Go, using only each game's rules. A peer-reviewed version appeared in Science in December 2018.

The preprint reports that training ran for 700,000 steps with mini-batches of 4,096 positions, using 5,000 first-generation TPUs to generate self-play games and 64 second-generation TPUs to train the networks. AlphaZero passed Stockfish, a top chess engine, after about four hours (300,000 steps), passed the shogi engine Elmo in under two hours, and passed AlphaGo Lee in Go after about eight hours.

Matchup (100 games, 1 minute per move, preprint)	AlphaZero wins	Draws	AlphaZero losses
Chess vs Stockfish	28	72	0
Shogi vs Elmo	90	2	8
Go vs 3-day AlphaGo Zero	60	0	40

Search style differed sharply from the older engines. The preprint says AlphaZero examined about 80,000 positions per second in chess and 40,000 in shogi, against about 70 million for Stockfish and 35 million for Elmo. Using its network to concentrate on the most promising lines, it searched roughly a thousand times fewer positions.

Caveats worth knowing

The December 2017 chess result drew criticism. As summarized on Wikipedia, Stockfish's developer Tord Romstad said the fixed one-minute move time prevented Stockfish's time management from working, that the version used was about a year old, and that the hash table was too small for the number of search threads used. Other players argued an opening book would have narrowed the gap.

The December 2018 Science version responded with a larger test. Against a newer Stockfish development version with endgame tablebases and a 32 GB hash, running at 3 hours plus 15 seconds per move, AlphaZero won 155 games, lost 6, and drew 839 out of 1,000. So the headline result held under revised conditions, with far more draws than the 2017 table shows.

The authors themselves note that Go suits convolutional networks well: its rules are translation-invariant, defined through local liberties, and symmetric under rotation and reflection, so training data can be augmented eightfold. Chess and shogi break those symmetries, which makes them, in the paper's words, "less innately suited" to the architecture. That AlphaZero worked there anyway is the interesting part.

What AlphaGo does and does not show

It is tempting to read AlphaGo as proof that machines now understand strategy. The record supports something narrower.

What it shows. Learned evaluation can substitute for hand-written evaluation in a domain where nobody could write one. Search guided by a learned prior can handle a branching factor that defeats brute force. Self-play can generate training data without human labelers when the environment supplies a clean win or loss signal.

What it does not show.

• General intelligence. Go is a game with perfect information, fixed rules, and an unambiguous outcome. The AlphaGo Zero and AlphaZero papers rely on a simulator that can play millions of games cheaply. Most real problems lack all three of those features.
• Understanding. Whether a network "understands" Go is a philosophical question without an agreed test. What can be measured is that it plays superhuman moves and, as Game 4 showed, can fail in positions where its search did not look.
• Free lunch on compute. The 2016 system used 176 GPUs in its distributed form, and AlphaZero used thousands of TPUs to generate self-play games. The algorithmic improvements in AlphaGo Zero were real, but self-play at this scale is expensive.

The same family of ideas has spread. Reinforcement learning from human feedback, used to tune language models, borrows the reward-driven loop but replaces the game's win signal with learned human preference scores; see how AI models learn from feedback and RLHF versus RLAIF versus DPO. DeepMind's protein-structure system is covered in AlphaFold; it is a separate system with its own approach, not a re-run of the game-playing programs. For a gaming-focused view of the same lineage, see From AlphaGo to StarCraft.

If you want the scaling story behind systems this size, see why AI needs so much computing power and the broader history of artificial intelligence. For the underlying method, the companion piece on reinforcement learning covers value functions and policies in general terms.

Frequently Asked Questions

Q: How many games did Lee Sedol win against AlphaGo? A: One. AlphaGo won Games 1, 2, 3, and 5, and Lee won Game 4 on March 13, 2016, for a 4-1 final score. All five games ended by resignation.

Q: What was Move 37? A: It was AlphaGo's 37th move in Game 2, a shoulder hit that professional commentators called creative and unusual. DeepMind describes it as a move with a 1 in 10,000 chance of being used. It helped AlphaGo win the game.

Q: Was AlphaGo just brute-force search like Deep Blue? A: No. Deep Blue evaluated about 200 million positions per second with a hand-built scoring function. AlphaGo evaluated thousands of times fewer positions, according to the Nature paper, and relied on learned networks to decide which positions deserved attention and how to score them.

Q: What is the difference between AlphaGo, AlphaGo Zero, and AlphaZero? A: AlphaGo learned first from 30 million human positions, then from self-play, and used two networks plus rollouts. AlphaGo Zero used no human data, one network, and no rollouts. AlphaZero generalized the Zero method to chess and shogi as well as Go, using only the rules of each game.

Q: Why did AlphaGo lose Game 4? A: Lee Sedol's move 78 created a line that AlphaGo's search had pruned away. The program played a series of weak moves from about move 87 and resigned when its estimated win chance fell below 20 percent. Observers linked the failure to a known weakness of Monte Carlo search.

Q: How strong was AlphaGo Zero compared with the version that beat Lee Sedol? A: It won 100 games to 0 against the exact AlphaGo Lee version that played in Seoul, under the same two-hour time controls, using one machine with 4 TPUs against AlphaGo Lee's 48. It surpassed AlphaGo Lee after about 36 hours of training.

The takeaway

The striking thing about AlphaGo is not that a computer won a game. It is how the win was assembled. A policy network narrowed 250 options to a handful, a value network judged positions that no rule could score, and a tree search used both to check each other. When the pieces were rearranged in AlphaGo Zero, so that search became part of the training loop and human games were removed, the system got stronger and simpler at the same time.

The Game 4 collapse is the other half of the lesson. A search steered by learned priors is efficient because it ignores most of the tree, and that same efficiency creates blind spots. Understanding both halves is the right way to read later claims about game-playing AI, and about any system that learns where to look before it looks.

Sources

• Silver, D., et al. (2016). "Mastering the game of Go with deep neural networks and tree search." Nature, 529, 484-489. Source for the network, search, hardware, and Fan Hui figures.
• Silver, D., et al. (2017). "Mastering the Game of Go without Human Knowledge." Nature, 550, 354-359. Source for AlphaGo Zero figures.
• Silver, D., et al. (2017). "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm." arXiv:1712.01815. Source for AlphaZero preprint figures.
• Google DeepMind. AlphaGo project page and AlphaGo Zero blog post.
• Wikipedia. AlphaGo versus Lee Sedol, AlphaGo, AlphaZero, and Computer Go.
• IBM. Deep Blue.
• Tromp, J. Number of legal Go positions.

---

Read next:

• Reinforcement Learning: Agents Learning Through Interaction
• AlphaFold: AI Solving Protein Folding in Hours
• Neural Networks and Deep Learning: Mimicking the Brain