AlphaZero: Self-Teaching Chess AI

On December 5, 2017, DeepMind posted a paper describing AlphaZero, a program that learned chess by playing against itself, starting from random moves and knowing only the rules. After about four hours of training it was already stronger than Stockfish 8, a leading chess engine, and after nine hours it beat that engine in a 100-game match without losing a game. The same algorithm learned shogi and Go. This article explains how the self-play loop works, what the match results show, and what critics pointed out.

What AlphaZero is, and what it is not

AlphaZero is a single algorithm applied to three board games: chess, shogi (Japanese chess), and Go. The paper by David Silver, Thomas Hubert, Julian Schrittwieser, Demis Hassabis, and colleagues describes it as a generalization of AlphaGo Zero, DeepMind's earlier Go program that learned without human game records. The authors state that, starting from random play and knowing only the rules, AlphaZero reached superhuman strength in all three games within 24 hours and defeated a world-champion program in each (Silver et al. 2017, arXiv:1712.01815). A peer-reviewed version appeared in Science in December 2018 (volume 362, pages 1140 to 1144).

It helps to be precise about what "from scratch" means. AlphaZero was given the legal moves and the win, draw, and loss conditions of each game. It was not given human games, opening books, or a hand-written scoring formula for positions. The program that learned Go, for comparison, is the subject of AlphaGo: AI Defeating World Champion at Go, and the broader learning paradigm is covered in Reinforcement Learning Explained.

Two things AlphaZero is not. It is not a general intelligence: it plays games with perfect information, fixed rules, and a clean win-or-lose signal. And it is not magic: it is a neural network, a search procedure, and a training loop, each of which can be described in plain terms.

How a traditional chess engine thinks

To see what changed, start with the program AlphaZero beat. Stockfish uses an alpha-beta tree search, a method that examines a very large number of possible move sequences and discards branches that cannot improve on lines already found. Positions at the end of each line are scored by an evaluation function written by humans, with terms for material, pawn structure, king safety, and similar features (Wikipedia: Stockfish).

Picture a clerk with a ledger. The clerk checks every possible line of play, adds up a hand-built tally of each ending position, and picks the best total. Speed matters, so the engine is tuned to examine tens of millions of positions per second. In the 2017 matches, Stockfish 8 searched about 70 million positions per second, while AlphaZero searched about 80,000 per second in chess (Silver et al. 2017). That is roughly a thousand times fewer.

	Stockfish 8 (2017 match)	AlphaZero (2017 match)
Search method	Alpha-beta search	Monte Carlo tree search guided by a neural network
Evaluation	Hand-crafted function	Learned by the network
Positions searched per second (chess)	about 70 million	about 80,000
Hardware in the match	64 CPU threads, 1 GB hash	4 first-generation TPUs
Source	Silver et al. 2017	Silver et al. 2017

The point of the table is that AlphaZero compensated for searching far fewer positions by choosing which ones to search much more selectively. That selectivity comes from a neural network.

The recipe: one network, one search, one loop

AlphaZero has three parts, and they feed each other.

Part one: a network that outputs a policy and a value

A single deep neural network takes a board position and produces two outputs. The first is a policy, a probability for each legal move that says which moves look promising. The second is a value, a single number estimating the expected outcome of the game from the current position. For background on how such networks are built, see How Do Neural Networks Work.

The input to the chess network is a stack of 119 planes, each an 8 by 8 grid, that encodes the current position and the previous seven (so the network can see repetitions) plus rule-related features such as castling rights and a repetition count. The network's move output is organized as 8 × 8 × 73 = 4,672 possible moves, covering moves along queen-like lines, knight moves, and underpromotions. Illegal moves are masked out and the remaining probabilities are renormalized (Silver et al. 2017).

Part two: a search that spends effort where the network points

During play, AlphaZero does not simply play the network's top move. It runs a Monte Carlo tree search (MCTS), which builds a tree of possible continuations by repeated simulations. Each simulation walks down the tree choosing moves that balance two things: how good the network thinks the move is, and how little the move has been explored so far. At the end of the simulated line, the network's value estimate stands in for the result.

An analogy: a scout sent into a maze. A rule-based scout walks every corridor to the same depth. A scout guided by a hunch about which corridors look promising explores fewer paths but follows the interesting ones deeper. The network supplies the hunch, and the search checks the hunch against what actually happens a few moves ahead. During training, AlphaZero ran 800 simulations per move.

Part three: a loop that learns from its own games

The loop has a simple shape. The current network plays games against itself, using MCTS to choose each move. Every position in those games is stored with two labels. One is the search's move-probability distribution, based on how often each move was visited during the simulations. The other is the final result of the game: win, draw, or loss. The network is then adjusted so that its policy output moves closer to the search's distribution and its value output moves closer to the real outcome. The loss function combines a squared error on the value, a cross-entropy term on the policy, and a regularization term (Silver et al. 2017).

This is why the search matters for learning as well as for play. The search is usually a better move-chooser than the raw network, so training the network to imitate the search makes the network better, and a better network makes the next round of search better. Researchers describe this as policy improvement through search. It is a close cousin of what is described in How AI Models Learn From Feedback, with the difference that the feedback here comes from the game result, not from human raters.

How the loop avoids getting stuck

A learning system that always plays what it currently believes is best can lock in bad habits. AlphaZero adds exploration in two ways. In training, moves are chosen in proportion to how often the search visited them, not simply by taking the top choice. And random noise drawn from a Dirichlet distribution is added to the move probabilities at the root of the search. The noise parameter was 0.3 for chess, 0.15 for shogi, and 0.03 for Go, scaled roughly to the number of legal moves in each game (Silver et al. 2017). Because the loop generates its own training data, this approach also relates to the broader topic of synthetic data.

How much computing it took

The idea that a program learned chess "in four hours" is accurate but leaves out the scale. The paper reports that self-play games were generated using 5,000 first-generation TPUs (Google's custom chips for neural networks), while the network was trained on 64 second-generation TPUs. Training ran for 700,000 steps with mini-batches of 4,096 positions, and chess training produced about 44 million self-play games over 9 hours (Silver et al. 2017).

Quantity	Chess figure	Source
Self-play hardware	5,000 first-generation TPUs	Silver et al. 2017
Training hardware	64 second-generation TPUs	Silver et al. 2017
Training steps	700,000	Silver et al. 2017
Mini-batch size	4,096 positions	Silver et al. 2017
Self-play games	about 44 million	Silver et al. 2017
Training time before the 2017 match	9 hours	Silver et al. 2017
Time to pass Stockfish 8 in an internal benchmark	about 4 hours	Wikipedia: AlphaZero

"Four hours" is therefore the point at which the in-training network first matched Stockfish 8 in an internal comparison, using thousands of chips in parallel. The total of 9 hours is wall-clock time, not compute time. A reader should not picture a laptop. For a sense of why training of this kind needs so much hardware, see Why AI Needs So Much Computing Power. During the matches, AlphaZero ran on a single machine with four first-generation TPUs (DeepMind blog, December 6, 2018).

Training times differ by game. DeepMind's 2018 summary gives about 9 hours for chess, 12 hours for shogi, and 13 days for Go, and notes that AlphaZero first beat the relevant rival after about 4 hours for Stockfish, 2 hours for the shogi program Elmo, and 30 hours for the 2016 version of AlphaGo that beat Lee Sedol (the 2017 preprint gives 8 hours for that last comparison, so sources differ on it).

The matches, in numbers

DeepMind reported two sets of chess results, one in the 2017 preprint and a larger one in the 2018 Science paper.

Match	Games	AlphaZero wins	Draws	AlphaZero losses	Notes
2017 preliminary match	100	28	72	0	1 minute per move; Stockfish 8 with 64 threads and 1 GB hash
2017 matches from 12 popular openings	1,200	290	886	24	100 games per opening, 50 with each color
2018 final match	1,000	155	839	6	Three hours per game plus 15 seconds per move

Sources: Silver et al. 2017, Wikipedia: AlphaZero, and the DeepMind blog. In the 100-game match, AlphaZero won 25 games as White and 3 as Black, with the rest drawn.

A common error is to say AlphaZero "won 155 of 200 games." The correct figure is 155 wins, 839 draws, and 6 losses out of 1,000. The same table shows something subtler: most games were draws. Chess between two very strong players is drawish, and AlphaZero's edge showed up as a modest surplus of wins over losses. Scoring a win as one point and a draw as half, AlphaZero's 2018 result is about 57 percent, and by the standard Elo formula that corresponds to a rating gap of roughly 50 points. That is a clear but not crushing advantage, and the numbers deserve to be read with that in mind.

What it did with openings

AlphaZero played the 12 most popular human openings from its own self-play and, according to the paper, "independently discovered" and frequently played each of them. The table below shows its results from those openings against Stockfish, with 100 games per opening.

Opening	Wins	Draws	Losses
English (A10)	28	70	2
Queen's Gambit (D06)	17	81	2
Queen's Pawn Game (A46)	27	73	0
Queen's Pawn Game (E00)	22	77	1
King's Indian (E61)	16	82	2
French (C00)	43	57	0
Sicilian (B50)	21	75	4
Sicilian (B30)	14	85	1
Sicilian (B40)	20	71	9
Ruy Lopez (C60)	33	66	1
Caro-Kann (B10)	29	70	1
Réti (A05)	20	79	1

Source: Table 2 of Silver et al. 2017, with the White and Black results added together. The totals reproduce the 290 wins, 886 draws, and 24 losses above. Note that the one opening with a notable loss count, a Sicilian line (B40), still came out ahead. The overall picture is of a system that had arrived at much the same menu of openings that humans have favored for generations.

How it played

Beyond the results, the games drew attention for their style. Grandmaster Matthew Sadler and Woman International Master Natasha Regan, who studied the games, described a highly dynamic approach that values piece activity and mobility and gives material less weight than conventional engines do. Garry Kasparov said it "plays with a very dynamic style, much like my own!" (DeepMind blog). Demis Hassabis called its play "alien" and "like chess from another dimension" (Wikipedia: AlphaZero).

DeepMind also reported that in chess, AlphaZero independently rediscovered common human ideas such as openings, king safety, and pawn structure, and developed some novel strategies. In shogi, the professional Yoshiharu Habu said that moving the king to the center of the board goes against shogi theory, yet AlphaZero remained in control.

One way to understand the style: an evaluation function written by humans encodes human values, such as a pawn being worth about one point. A learned evaluation has no such prior. If a pawn sacrifice reliably leads to wins in millions of self-play games, the network learns to value the resulting position highly, whatever a textbook says. That is a plausible mechanism for the willingness to give up material that commentators noticed, though it is an interpretation and not something the training logs prove.

What critics pointed out

The December 2017 result was impressive and also immediately questioned. The main objections, as recorded in the public discussion, were these (Wikipedia: AlphaZero).

• Stockfish's settings. In the preliminary match, Stockfish ran with 64 threads and a 1 GB hash table at a fixed one minute per move. Tord Romstad, one of its original developers, criticized the fixed time control, the age of the version, and the small hash. The 2018 rematch used 44 CPU cores and a 32 GB hash, and added endgame tablebases.
• The hardware comparison. Hikaru Nakamura said he put little credibility in the 2017 result because AlphaZero ran on Google's supercomputing hardware while Stockfish ran on far more modest hardware.
• No opening database. Larry Kaufman argued that a 2017 match in which Stockfish had an opening database would have been closer.
• Version age. Mark Lefler called the result a remarkable achievement but noted that Stockfish had gained strength since the version DeepMind tested.
• Closed access. AlphaZero was never released, so independent testing was limited to the games and results DeepMind published.

Whether the 2018 rematch settles the fairness argument depends on how one weighs hardware that differs in kind. The cleanest reading is that AlphaZero was probably a little stronger than the engine it played under the stated conditions, and that the conditions were open to reasonable dispute. Questions of how to compare systems like this are discussed in AI Benchmarks Explained.

What came after

Because AlphaZero was not released, the chess community rebuilt the idea. Leela Chess Zero, announced on January 9, 2018, is an open-source engine based on AlphaZero's approach and adapted from the Go engine Leela Zero. Instead of one company's data center, it relies on volunteers generating self-play games on their own hardware. After DeepMind's December 2018 paper, its architecture and training details were folded into Leela. In 2022 it switched from residual networks to a transformer-based design (Wikipedia: Leela Chess Zero).

The tournament results are mixed in a way that is informative. Leela won the Top Chess Engine Championship (TCEC) Season 15 superfinal 53.5 to 46.5 in 2019 and regained the title in Season 17 in 2020, but Stockfish has won every superfinal against Leela from Season 18 on (Wikipedia: Stockfish).

Stockfish absorbed part of the lesson as well. In September 2020, Stockfish 12 introduced NNUE, an efficiently updatable neural network that replaced the hand-written evaluation with a network trained on millions of positions and fast enough to run on an ordinary processor, based on earlier work by shogi programmers. In July 2023, the hand-crafted evaluation was removed entirely. The strongest engine today is thus a hybrid: a classical alpha-beta search with a learned evaluation.

MuZero extended the approach in another direction. AlphaZero plans using a simulator programmed with the game's rules. MuZero learns its own model of the environment and needs no rules, and it matched AlphaZero in chess, shogi, and Go while also reaching state-of-the-art results on Atari games. A 2020 Nature paper describes it, and in February 2022 DeepMind reported a first real-world application: a MuZero variant that chose the quantization parameter for each frame in YouTube's VP9 video compression and reduced bitrate by an average of 4 percent without quality loss (Wikipedia: MuZero).

Other DeepMind systems, such as AlphaFold, which predicts protein structures, tackle different problems with different methods. They share a research lineage and a naming style with AlphaZero, not a design. AlphaFold is covered in AlphaFold: AI Solving Protein Folding.

What the result does and does not show

It shows that a general method can replace hand-built knowledge in these games. One algorithm, with the same overall design, produced a top-level player in three games with different rules. The authors present this as a step toward more flexible AI while acknowledging that it is early (DeepMind blog).

It does not show that self-play works everywhere. The method depends on a perfect simulator of the game, a clear outcome at the end, and the ability to play tens of millions of games cheaply. Most real problems lack at least one of those. A robot cannot practice a million times in an hour, and many tasks have no tidy score. That limitation is why MuZero's learned model and the use of human feedback in language models (RLHF and related methods) exist.

It does not show that the network "understands" chess. The system encodes patterns that win games. Some of those patterns overlap with human concepts, and the DeepMind team reported that human opening ideas appeared during training, but the network does not explain its choices. Interpretability research tries to read such models from the inside, as described in Mechanistic Interpretability.

It does not mean chess is solved. Chess has not been solved in the mathematical sense, meaning no one has proved what perfect play yields from the starting position. AlphaZero's results show it plays far better than humans and competitively with the best engines, nothing more. The high draw rate in the 1,000-game match is consistent with the view that chess between near-perfect players is often a draw, though that remains an inference and not a proof.

Frequently asked questions

Q: How did AlphaZero teach itself without human games? A: It played games against itself and learned from the outcomes. Each self-play game produced training examples: the search's preferred moves and the final result. The network was adjusted to predict both more accurately. Repeating this over about 44 million chess games gave a network and search that were strong enough to beat Stockfish 8.

Q: Did AlphaZero really learn chess in four hours? A: Partly. In an internal benchmark, the in-training AlphaZero first matched Stockfish 8 after about four hours, and total training before the 2017 match was nine hours. That used 5,000 first-generation TPUs for self-play and 64 second-generation TPUs for training, so the clock time hides a very large amount of computing.

Q: How many games did AlphaZero win against Stockfish? A: In the 2017 preliminary match, it won 28 of 100 games and drew the other 72 without losing. In the 2018 final match, it won 155 of 1,000 games, drew 839, and lost 6. Most games ended in draws, which is typical at this level of play.

Q: Was the match against Stockfish fair? A: It was disputed. Critics pointed to Stockfish's modest hash size and thread count in the first match, the lack of an opening book, the age of the Stockfish version, and the difference between CPU and TPU hardware. DeepMind's 2018 rematch gave Stockfish 44 cores, a 32 GB hash, and tablebases, and AlphaZero still won.

Q: Why did AlphaZero's style look different from human chess? A: Its evaluation was learned from results, not written by humans, so it did not inherit rules of thumb such as fixed piece values. Analysts described a dynamic style that prizes piece activity and mobility. It also played the common human openings, so it did not ignore tradition entirely.

Q: Can anyone run AlphaZero? A: Not the original. DeepMind did not release the program. The open-source Leela Chess Zero project, announced in January 2018, reproduces the approach with volunteer-generated games, and it has competed in top engine tournaments.

Q: Does AlphaZero work for tasks other than board games? A: Not directly. It needs a perfect simulator and a clear win-or-lose signal. Its successor MuZero learns a model of the environment and has been applied to Atari games and to video compression at YouTube, where it reduced bitrate by an average of 4 percent in a reported test.

The takeaway

AlphaZero's lasting contribution is less the match score than the demonstration that search and learning can bootstrap each other. A network proposes where to look, a search tests the proposal, and the test results become the next round of training data. Humans supplied the rules and the training machinery, and the program supplied the chess.

The caveats are part of the picture. The headline matches were fought under conditions that critics disputed, the program was never released, and the same recipe has not carried over to problems without a perfect simulator. What did carry over is the idea, visible now in open-source engines, in Stockfish's own neural evaluation, and in systems like MuZero that learn their own models. For a wider view of where this fits in the field's history, see The History of Artificial Intelligence.

---

Sources:

• Silver, D., et al. (2017). "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm." arXiv:1712.01815. https://arxiv.org/abs/1712.01815 (HTML version)
• Silver, D., et al. (2018). "A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play." Science 362, 1140-1144.
• AlphaZero: Shedding new light on chess, shogi, and Go, DeepMind, December 6, 2018
• Silver, D., et al. (2017). "Mastering the game of Go without human knowledge." Nature 550, 354-359. https://doi.org/10.1038/nature24270
• AlphaZero, Wikipedia
• Stockfish (chess engine), Wikipedia
• Leela Chess Zero, Wikipedia
• MuZero, Wikipedia

---

Read next:

• Scaling Laws Explained: Why Bigger Models Got Better
• Transfer Learning: Why AI Doesn't Start From Scratch
• AI Training vs. Inference: What Actually Happens When AI Learns and Thinks