Game AI: How NPC Behavior Actually Works

A guard in a stealth game glances down a hallway, spots you, and radios for backup. A flock of birds scatters when your character runs through a courtyard. A ghost in a forty-five-year-old arcade cabinet always seems to know exactly where you're headed before you get there. None of these behaviors involve anything close to real understanding. They run on a small set of algorithms, most of them decades old, that combine to produce the illusion of a thinking creature. This article covers that foundational toolkit: the finite state machines, pathfinding algorithms, and steering behaviors that still drive the overwhelming majority of NPCs in shipped games today. Newer machine-learning-driven approaches to NPC decision-making are a genuinely different subject and are covered separately in our companion piece on neural networks in game AI.

Key takeaways

• Finite State Machines (FSMs) are the oldest and still most common NPC architecture: a character exists in exactly one discrete state at a time (idle, patrol, chase, flee) and moves between states when defined conditions are met.
• A* pathfinding, published in 1968, remains the near-universal algorithm for getting an NPC from point A to point B efficiently through a game level (Hart, Nilsson & Raphael, 1968).
• Steering behaviors, formalized by animator Craig Reynolds, handle the smaller, second-by-second problem of how a character moves once it has a path: accelerating, turning, avoiding collisions.
• Flocking (boids) shows that dozens of independent agents following three simple local rules can produce convincing group behavior with no central coordinator (Reynolds, 1987).
• These four techniques predate modern machine learning by decades and remain the default choice for most shipped games because they are fast, predictable, and easy for a human designer to debug.

What "AI" actually means in a video game

Ask a player what makes a game's AI good, and they will usually describe something that feels alert, resourceful, or unpredictable. Ask an engineer what "AI" means in a shipped game, and the answer is almost always a set of rules a person wrote by hand, evaluated many times per second. The gap between those two descriptions is the entire subject of this article.

Non-player character (NPC) is the umbrella term for any character in a game controlled by the software rather than a human player. That covers everything from a background villager who exists purely for atmosphere to a boss enemy with a dozen interacting attack patterns. What every NPC needs, no matter how simple or complex, is some way to decide what to do next, and some way to physically get there once it has decided.

Those two problems split cleanly. Decision-making is the logic layer: should this guard patrol, investigate, or attack? Navigation and movement is the physical layer: given a decision to attack, how does the guard actually walk from where it is standing to where the player is standing without clipping through a wall or bumping into a teammate? Classic game AI solves the first problem with finite state machines and the second with pathfinding and steering. Understanding both, and how they connect, explains most of what you experience as "the AI" in a typical game.

Craig Reynolds' 1987 boids model, built to simulate flocks of birds for computer animation, used no central "flock controller" at all. Each simulated bird followed three purely local rules based only on its nearest neighbors, and coordinated group motion emerged from thousands of those local interactions running in parallel ([Reynolds, "Flocks, Herds, and Schools," SIGGRAPH 1987](https://www.red3d.com/cwr/papers/1987/boids.html)). For the wider picture of game technology and design, see the [Complete Gaming Guide 2026](/gaming/complete-gaming-guide-2026).

Finite State Machines: the classical foundation

The finite state machine (FSM) is the workhorse of game AI decision-making. The underlying concept comes from automata theory, a branch of computer science formalized well before video games existed, and it was adopted broadly by game developers starting in the 1980s and 1990s. It remains in wide use today because it maps directly onto how a designer thinks about a character: "when this happens, do that."

An FSM defines a character as being in exactly one state at any given moment. Each state describes what the character does right now, whether that is playing an idle animation, walking a patrol route, or firing a weapon. Transitions are the rules that move a character from one state to another when a specific condition becomes true.

A simple guard NPC might have states like these:

• Patrol: Follow a predefined route of waypoints around a level.
• Investigate: Move toward the location of a recent noise or disturbance.
• Alert: Stop, scan the surroundings, and call for backup.
• Combat: Attack a confirmed, visible enemy.
• Flee: Retreat toward cover if the guard's health drops below a threshold.

The transitions connect these states into a graph. If the guard is patrolling and hears a sound within a set radius, it transitions to Investigate. If it is investigating and spots the player, it transitions to Alert or straight to Combat. If it is in combat and its health crosses a low threshold, it transitions to Flee. None of this requires anything resembling reasoning. It is a lookup: given the current state and the current world conditions, what is the next state?

The appeal of FSMs is that they are completely legible. A designer or QA tester can point at any NPC at any moment and say exactly which state it occupies and exactly why, because the rules are written down explicitly rather than learned from data. That legibility is also why FSMs remain the default choice in shooters, stealth games, and RPGs decades after their introduction: certification processes and QA teams need behavior that fails in reproducible, debuggable ways, not behavior that might do something unexpected on the ten-thousandth playthrough.

The tradeoff is a problem informally called state explosion. As a designer adds more nuance, states such as weather-aware behavior, faction reputation, or dialogue history, the number of possible transitions between states grows far faster than the number of states themselves, because every new state potentially needs a transition to and from every existing one. A guard that only needs to patrol and fight might need five states and eight transitions. A guard that also reacts to time of day, squad orders, and player reputation can balloon into dozens of states with a transition graph that becomes difficult for anyone to reason about or safely modify.

Hierarchical finite state machines are the classical fix. Instead of one enormous flat graph, states are nested: a top-level Combat state might contain its own small FSM choosing between melee, ranged, and cover-seeking sub-states, while the outer graph only has to know about "Combat" as a single unit. This keeps the outer graph small and each inner graph focused on a narrow problem, and it is a pattern that shows up throughout non-game software engineering too, not just in NPC design.

What is the difference between a state machine and a behavior tree?

A finite state machine has one active state at a time and moves between states via explicit transitions, which makes it fast to build but prone to a tangled web of transitions as complexity grows. A behavior tree instead re-evaluates a hierarchical tree of conditions and actions from the root every frame, which scales to complex behavior more gracefully because new behaviors are added as new branches rather than new transitions from every existing state. Behavior trees, along with newer utility-based and planning approaches, are covered in depth in our companion article on modern NPC decision architectures.

A worked example: the ghosts in Pac-Man

Before diving into pathfinding mathematics, it helps to look at one of the earliest and most thoroughly documented examples of game AI: the four ghosts in Pac-Man (1980). Their behavior looks purposeful, even personality-driven, but it comes from a handful of simple, hand-authored rules that have been reverse-engineered in detail by hobbyists and documented extensively, most famously in Jamey Pittman's "Pac-Man Dossier" (Pittman, via Game Developer).

Each ghost alternates between two global modes: scatter mode, where it heads toward a fixed corner of the maze assigned to it, and chase mode, where it actively pursues Pac-Man. That alternation is itself a tiny finite state machine, ticking on a timer rather than on player behavior.

What makes the four ghosts feel distinct is not four different algorithms but four different targeting rules layered on top of the same underlying maze-navigation logic:

• Blinky (red) targets Pac-Man's current tile directly, making him the most aggressive and persistent chaser.
• Pinky (pink) targets a point several tiles ahead of Pac-Man's current facing direction, which makes her appear to anticipate and cut off the player.
• Inky (cyan) computes its target using both Pac-Man's position and Blinky's position, producing the least predictable, most erratic-looking movement of the four.
• Clyde (orange) chases Pac-Man directly when far away but retreats to his scatter corner when close, giving him a reputation for being the most passive ghost.

Every one of those targets is just a coordinate on the maze grid. Once the target is chosen, the ghost's actual movement decision at each intersection is a simple rule: at every junction, pick the direction that gets closer to the target tile, with reversing direction forbidden except when changing global modes. There is no search algorithm evaluating multiple paths and no learning involved. Four cheap targeting formulas, run against one shared movement rule, produced four ghosts that players have spent over four decades describing as having distinct personalities (GameInternals: Understanding Pac-Man Ghost Behavior).

The lesson generalizes well beyond one 1980 arcade cabinet: a lot of what reads as "personality" in NPC design is a small number of shared mechanical building blocks reused with different parameters, not fundamentally different AI systems for each character.

A* pathfinding: finding the way

An NPC with a perfectly reasoned decision to attack the player is useless if it cannot physically walk there. Pathfinding is the problem of finding a collision-free route from a starting point to a destination through a game level's geometry, and it is one of the most consistently important problems in game AI regardless of genre.

The dominant algorithm, by a wide margin, is A* (pronounced "A-star"), introduced by Peter Hart, Nils Nilsson, and Bertram Raphael in a 1968 paper titled "A Formal Basis for the Heuristic Determination of Minimum Cost Paths" (IEEE Transactions on Systems Science and Cybernetics, 1968). The paper has been cited more than 12,000 times and its core idea has barely changed in nearly six decades of use.

A* works on a graph: a network of nodes connected by edges, each edge carrying a cost. In a game, that graph might represent rooms, waypoints, or grid cells across a level. A* keeps two running lists of nodes, one for nodes it still needs to check and one for nodes it has already finished checking, and at each step it picks the most promising unchecked node to examine next. "Most promising" is defined by a scoring formula:

f(n) = g(n) + h(n)

Here, g(n) is the real, already-known cost of the cheapest path found so far from the start to node n, and h(n) is a heuristic, an estimate of the remaining cost from n to the goal, commonly just the straight-line distance. A* always expands the node with the lowest combined score first, which means it is simultaneously accounting for the ground already covered and a smart guess about the ground still to cover. As long as the heuristic never overestimates the true remaining cost, a property called admissibility, A* is mathematically guaranteed to find the shortest possible path, and it typically examines far fewer nodes than uninformed search algorithms like plain Dijkstra's algorithm because the heuristic steers the search toward the goal instead of expanding outward in every direction equally. Amit Patel's widely used Red Blob Games tutorial series remains one of the clearest public explanations of exactly how that tradeoff plays out in practice (Red Blob Games, Introduction to A*).

For a modern 3D game, the graph A* searches is almost never the raw level geometry. It is a navigation mesh, or NavMesh: a simplified set of connected polygons covering only the walkable surfaces of a level, with stairs, ledges, ceilings, and unreachable gaps stripped out ahead of time. Building a NavMesh happens once, when a level is authored or loaded, and turns what could be millions of individual grid cells into a few thousand polygons that A* can search in a fraction of a millisecond. This is the reason a guard in a modern open-world game can compute a route across a sprawling map without any visible delay: the hard geometric work was already done offline, and the runtime search only has to reason about a small, pre-simplified graph.

How does pathfinding handle moving obstacles like other characters?

Static level geometry is baked into the NavMesh in advance, but characters and destructible objects move during play, so pathfinding pairs A* with a separate local-avoidance layer, often an algorithm family called reciprocal velocity obstacles, that nudges an NPC's exact position frame by frame to avoid a nearby moving character without triggering a full, expensive recomputation of the entire path. Only significant, persistent blockages, such as a door slamming shut, typically force a fresh A* search.

Steering behaviors: movement after the path is found

A* answers "which sequence of waypoints should I follow," but it says nothing about how a character should actually accelerate, turn, or slow down while following those waypoints. That second, finer-grained problem is what animator and researcher Craig Reynolds addressed with steering behaviors, first described broadly in his 1987 boids paper and expanded in a dedicated 1999 paper, "Steering Behaviors for Autonomous Characters" (Reynolds, GDC 1999, referenced via redblobgames.com and red3d.com).

A steering behavior is a small function that takes a character's current position and velocity and produces a steering force: a nudge that adjusts direction and speed toward some goal. Reynolds described a small library of these that combine cleanly:

• Seek: steer directly toward a target position.
• Flee: steer directly away from a threat.
• Arrival: like seek, but decelerate smoothly on approach instead of overshooting and snapping to a stop.
• Pursuit: steer toward a moving target's predicted future position, not its current one, which is what lets a chasing NPC actually catch a moving player instead of always trailing slightly behind.
• Obstacle avoidance: steer around nearby geometry detected just ahead of the character's current path.
• Separation: steer away from other nearby characters to avoid crowding or overlapping.

Individually, each of these behaviors is a few lines of vector math. What makes them powerful is that they combine by simple weighted addition: a character following a waypoint path while dodging a wall and staying clear of a squadmate is just seek, obstacle avoidance, and separation forces added together and applied to the same character each frame. This is a genuinely different layer of the problem from A*. A* decides the overall route once, in advance; steering behaviors decide the physical, moment-to-moment motion along that route dozens of times per second, and they are what stops a game character's movement from looking like a robot snapping between fixed waypoints.

A* and steering behaviors solve two different scales of the same problem. A* answers "which waypoints should I visit, in what order, to get from here to there efficiently." Steering behaviors answer "given that I need to reach the next waypoint, exactly how should I accelerate and turn to get there smoothly, right now, while avoiding whatever is immediately in front of me." Most navigating NPCs run both, layered on top of each other, every single frame.

Flocking and boids: group behavior from local rules

Reynolds' most famous contribution, developed for the same 1987 SIGGRAPH paper that introduced steering, is the boids model of flocking. It was originally built to animate flocks of birds and schools of fish for computer graphics, not for interactive games at all, but its underlying idea turned out to be directly useful for any game that needs a crowd, herd, or swarm to move convincingly without a designer scripting each individual member.

Boids works by giving every simulated agent, or "boid," exactly three rules, each based only on that boid's nearby neighbors rather than any global information about the whole flock:

1. Separation: steer to avoid crowding nearby flockmates.
2. Alignment: steer toward the average heading of nearby flockmates.
3. Cohesion: steer toward the average position of nearby flockmates.

No boid knows anything about the flock as a whole, and there is no central controller deciding where the group goes. Each agent only ever looks at a handful of neighbors within a small radius and reacts to them using the same three simple rules. Run that locally simple logic across a large group simultaneously, and the result is a convincing, organic-looking flock that banks, splits around obstacles, and re-merges, with no individual agent, and no designer, ever specifying the group-level motion directly (Reynolds, "Flocks, Herds, and Schools: A Distributed Behavioral Model," SIGGRAPH 1987).

That distributed idea proved durable well beyond bird animation. It became one of the first widely cited examples of what researchers call emergent behavior, where a simple, local rule set produces complex, coordinated-looking global behavior that nobody explicitly programmed. Boids-derived logic has since been used for everything from background wildlife and enemy swarms to crowd simulation in city-scale open-world games, and the 1992 film Batman Returns used a modified version of the original boids software to generate its computer-simulated bat swarm and penguin army, one of the earliest applications of the technique outside pure research.

What makes flocking different from pathfinding?

Pathfinding answers where a single character should go through fixed level geometry. Flocking answers how a group of characters should move relative to each other in open space, using only local information about nearby neighbors rather than any global map. Games frequently combine the two: a flock of background NPCs might use A* collectively to head toward a destination while individually using boids rules to avoid colliding with each other along the way.

Perception systems: giving NPCs something to react to

None of the decision-making or movement machinery matters if an NPC has no way to notice the player in the first place. Game perception systems are simple, deliberately approximate simulations of sight and hearing, built to produce plausible reactions rather than model real biological perception.

Sight is typically modeled as a field-of-view cone: an angle, often somewhere between 120 and 180 degrees, combined with a maximum viewing distance. If the player's position falls within that cone, the game fires a raycast, a straight line traced through the level geometry from the NPC's eyes to the player's position. If that line reaches the player without being blocked by a wall or other obstacle, the player counts as visible. Many implementations soften this with a wider, lower-confidence peripheral zone around the central cone, loosely mirroring how human peripheral vision detects motion without resolving detail.

Hearing is usually modeled as an event with a position and an intensity value, fired whenever the player does something noisy, such as running or firing an unsuppressed weapon. Any NPC within range of that event's position receives it and can choose to react, typically by transitioning into an Investigate state in its FSM. Walls and closed doors commonly reduce the effective range of a sound event, which is why sneaking through a building with doors closed behind you is usually safer than leaving them open.

Memory is what lets an NPC keep reacting after losing direct sight or sound of the player. A typical implementation stores a last known position with a timestamp; the NPC moves toward that position and searches nearby before eventually giving up and returning to its default behavior if the player is not found within a set time window. This is a deliberately shallow simulation of memory. The NPC is not "remembering" in any meaningful sense; it is comparing a stored coordinate against a timer.

Common misconceptions about classic game AI

"NPCs are actually thinking about what to do." They are evaluating a fixed set of conditions written in advance by a designer. A guard that seems to weigh its options before attacking is running through an ordered checklist, not deliberating.

"Harder difficulty settings mean smarter AI." In classic architectures, difficulty is overwhelmingly implemented by changing numbers, not logic: wider sight cones, faster reaction times, more damage, better aim accuracy. The underlying FSM and pathfinding code is usually identical across difficulty settings; only the parameters fed into it change.

"Pathfinding always finds the best possible route." A* guarantees the optimal route only relative to the graph it is given and the cost values assigned to that graph's edges. If a level's NavMesh is generated with a mistake, or if a cost weight undervalues a dangerous shortcut, A* will confidently return a technically optimal path that looks wrong or even walks a character through obvious danger, because it is only as good as the graph and costs it was handed.

"NPCs never cheat." They frequently do, deliberately, and it is not considered a bug. Designers routinely give NPCs sight ranges or hearing radii more generous than their in-game models visually suggest, specifically to keep encounters challenging without requiring more expensive AI. Some pathfinding systems quietly teleport a stuck NPC back onto a valid NavMesh position rather than debug why it got stuck. These shortcuts are invisible during normal play and exist purely to serve the pacing of the game, not to simulate anything realistically.

Four classic techniques at a glance

Technique	Problem it solves	Introduced	Still used today for
Finite state machine	Which discrete behavior should this character run right now	1950s automata theory; game use from the 1980s–90s	Simple enemies, guards, most low-cost background NPCs
A* pathfinding	Finding the lowest-cost route through level geometry	1968 (Hart, Nilsson & Raphael)	Route planning on NavMeshes in nearly every 3D game
Steering behaviors	Moving smoothly along a chosen route while avoiding nearby obstacles	1987 boids paper, expanded 1999 (Reynolds)	Frame-by-frame acceleration, turning, and local avoidance
Boids / flocking	Coordinating a group's motion with no central controller	1987 (Reynolds, SIGGRAPH)	Crowds, wildlife, enemy swarms, background animation

Table: the four foundational game AI techniques covered in this article, the specific problem each one solves, and where they still show up in shipped games.

The takeaway

The techniques in this article, finite state machines for decisions, A* for routes, steering behaviors for movement, and boids for group motion, are each individually simple enough to explain on one page, and in combination they account for the overwhelming majority of NPC behavior players have experienced across five decades of games, from a 1980 arcade ghost to a modern open-world crowd scene. None of them require anything close to real reasoning, learning, or awareness; they require a designer who correctly anticipated the situations a character would face and wrote the rules to match. That is precisely why they remain the default choice in the games industry even as more computationally expensive, learning-based approaches mature elsewhere: they are fast, they are predictable, and when something goes wrong, a human can look at the code and know exactly why.

FAQ

Q: What is the most common AI technique used in video games? A: The finite state machine remains the single most widely used decision-making architecture for NPCs, because it is simple to author, fast to run, and easy to debug, even though more modern approaches like behavior trees have taken over in many AAA productions.

Q: Why do game developers use A instead of a more thorough search algorithm?* A: A* combines the guaranteed real cost already traveled with a heuristic estimate of the remaining cost, which lets it skip examining most of the level's geometry and focus its search toward the goal. Uninformed algorithms like Dijkstra's search outward in every direction equally and typically end up examining far more nodes for the same result.

Q: What is a NavMesh and why is it needed? A: A NavMesh is a simplified set of connected polygons that represents only the walkable surfaces of a game level, generated once ahead of time from the full level geometry. Searching a NavMesh with a few thousand polygons is dramatically faster than searching raw geometry or a fine grid with millions of cells, which is what makes real-time pathfinding practical in large game worlds.

Q: Is flocking the same thing as pathfinding? A: No. Pathfinding finds a route for one character through fixed level geometry toward a specific destination. Flocking coordinates the relative movement of a group of characters using only local information about nearby neighbors, with no fixed destination required. Games often layer both together for crowds or animal groups.

Q: How did the ghosts in Pac-Man have different personalities without machine learning? A: Each of the four ghosts uses the same underlying maze-navigation rule but computes a different target tile: Blinky targets Pac-Man directly, Pinky targets a point ahead of him, Inky's target depends on both Pac-Man and Blinky's positions, and Clyde switches between chasing and retreating based on distance. Four simple targeting formulas layered on one shared movement rule produced four ghosts players still describe as having distinct personalities more than four decades later.

Q: Do steering behaviors replace pathfinding? A: No, they solve a different part of the same problem. Pathfinding decides the overall sequence of waypoints a character should follow through a level. Steering behaviors decide how the character physically accelerates, turns, and avoids nearby obstacles while moving along that path, and most navigating NPCs run both systems together every frame.

Q: When should I look into neural-network-based NPC AI instead of these classic techniques? A: Classic techniques remain the right default for most shipped decision-making because they are cheap to run for many simultaneous NPCs and predictable for quality assurance. Neural-network approaches are more established today in dialogue systems and animation blending than in core combat decisions; the tradeoffs are covered in detail in our companion article on neural networks and NPC behavior.

---

Read next:

• How Neural Networks Are Making NPCs Smarter Than Ever
• AI-Powered Game Balancing: How Dynamic Difficulty Is Changing the Way We Play
• AI Agent — Glossary