How Rate Limiting Protects APIs

A single misconfigured script can send an API 10,000 requests per second by accident, the same volume a small-scale denial-of-service attack might generate on purpose. Rate limiting is the server-side mechanism that stops either scenario from taking a service down: it tracks how many requests a given client has made in a given time window and rejects or delays requests once that count is exceeded, typically returning the HTTP status code 429 Too Many Requests, a response code formally defined in RFC 6585 in 2012 (IETF, 2012). This guide walks through the algorithms that actually enforce that limit, why enforcing it gets much harder once an API runs on more than one server, and what the client is supposed to do when it gets throttled.

What rate limiting actually is, and why every production API needs it

Rate limiting is a technique for controlling the volume of requests a client is allowed to send to a server within a defined time period, rejecting or delaying requests once a threshold is crossed. It sits between the outside world and the application logic, acting as a gatekeeper that counts and decides rather than one that processes and responds.

Without it, an API has no defense against three distinct problems that all look identical from the server's point of view: too many requests arriving too fast. The first is abuse, where a bad actor deliberately floods an endpoint to disrupt service or extract data faster than the provider intends. The second is accidental overload, where a legitimate client has a bug, a retry loop with no backoff, a batch job that fires all its requests at once, and unintentionally behaves exactly like an attacker. The third is simple fairness: on a shared multi-tenant API, one customer's traffic spike should not be able to starve every other customer of capacity.

RFC 6585, the same specification that introduced the 429 status code, describes the underlying problem plainly: a server needs a standard way to tell a client "you are sending requests too fast" without inventing a proprietary error format for every API (IETF, 2012). Before 429 existed, APIs improvised with a generic 403 Forbidden or a custom error body, which made it much harder for client software to distinguish "you are blocked" from "you are just going too fast, try again shortly." Standardizing the status code let client libraries build automatic backoff behavior into their HTTP layer instead of every developer having to hand-roll it per API.

Rate limiting is also an economic control, not just a technical one. Many APIs, particularly those tied to a paid third-party dependency like an SMS gateway, a mapping service, or a hosted AI model, incur real financial cost per request. An unthrottled client, whether malicious or simply buggy, can generate an unexpected bill within minutes if nothing caps the request volume before it reaches the billed dependency.

The token bucket: the algorithm behind most modern rate limiters

The most widely deployed rate-limiting algorithm is the token bucket. Each client is assigned a virtual bucket that holds a fixed maximum number of tokens, and every incoming request consumes one token. Tokens refill continuously at a steady rate, for example ten tokens per second, so a client that has been idle can burst through a batch of stored-up tokens quickly, then gets throttled back down to the steady refill rate once the bucket runs dry.

Think of it like a parking garage with a strict number of spaces and a slow, steady trickle of cars leaving. If the garage has been empty for a while, several cars can pull in back-to-back with no wait. But once every space is full, a new car can only enter exactly as fast as an existing car leaves. The garage isn't measuring "how many cars per minute," it's measuring "how much capacity is free right now," which is a subtly different and more forgiving question.

That forgiveness is deliberate. Token bucket design intentionally allows short bursts of legitimate traffic, like a user rapidly clicking through several pages of a paginated result set, while still enforcing a hard average ceiling over a longer time horizon. This matches how real traffic actually behaves: human and even automated clients rarely send requests in a perfectly even stream, they cluster.

How the token bucket compares to simpler window-based counting

A simpler alternative is the fixed window counter, which just counts requests within a clock-aligned block of time, like "0 to 60 seconds," and resets to zero at the boundary. It is trivial to implement with a single counter and an expiry, but it has a well-known flaw: a client can send its full limit right before the window resets and its full limit again right after, doubling the intended rate in the few seconds spanning the boundary. This is why token bucket and its close relative, the sliding window log, are preferred in production systems where burst-doubling at a boundary is an actual operational risk rather than a theoretical one.

The 429 status code did not exist in the original HTTP/1.1 specification. It was added seventeen years later, in RFC 6585 (2012), specifically because rate limiting had become common enough across the web that clients needed a standardized way to recognize it (IETF, 2012).

Comparing the four main rate-limiting algorithms

Engineering teams choosing a rate-limiting strategy are usually picking among four well-established algorithms, each with a different memory footprint and a different tolerance for burst traffic. The table below summarizes how they actually behave under load, not just how they're described in textbooks.

Algorithm	How it tracks requests	Burst tolerance	Memory cost per client	Common weakness
Token bucket	Refillable token count, one token consumed per request	High — allows stored-up bursts	Low (one counter, one timestamp)	Can allow a large burst if the bucket sat full for a long idle period
Leaky bucket	Fixed-rate outflow queue; requests wait if the queue is full	Low — smooths traffic to a constant rate	Low to moderate (queue plus counter)	Adds latency to bursty-but-legitimate traffic since it enforces a flat rate
Fixed window	Simple counter reset at clock-aligned intervals	Low, but has a boundary flaw	Very low (single counter)	Up to 2x the intended limit right at the window boundary
Sliding window log	Timestamped log of every request in the trailing window	Moderate — precise, no boundary flaw	High (stores individual timestamps)	Memory grows with request volume per client

Token bucket dominates production APIs because it balances low memory cost with realistic burst tolerance, while sliding window log trades memory for precision when boundary accuracy matters more than storage cost.

Leaky bucket, worth explaining on its own, works almost like token bucket in reverse. Instead of tokens accumulating and being spent, incoming requests fill a queue that drains at a fixed, constant rate, like water leaking out of a bucket through a hole of fixed size. If requests arrive faster than the leak rate, they queue up rather than being processed immediately, and if the queue itself is full, new requests get rejected. This produces perfectly smoothed output traffic, which downstream systems appreciate, but it can add latency even to traffic that would otherwise be perfectly legitimate.

Why distributed rate limiting is a genuinely harder problem than single-server limiting

Everything above works cleanly when an API runs on one server holding one counter per client in local memory. The moment that API scales horizontally, running behind a load balancer across multiple server instances, that clean picture breaks. If each server keeps its own independent copy of the rate-limit counters, a client can effectively multiply its allowed rate by the number of servers, since each server only sees a fraction of that client's total traffic and thinks it's well under the limit.

The standard fix is to move the counter out of any individual server's local memory and into a shared, centrally accessible data store that every instance reads from and writes to. Redis, an in-memory key-value data store, is the most common choice for this job because it is fast enough to check on every request without adding meaningful latency, and it supports atomic increment-and-expire operations that avoid race conditions between simultaneous requests. A recent industry breakdown of Redis-based rate limiter design notes that a single Redis instance can realistically handle on the order of 50,000 to 100,000 rate-limit checks per second before it becomes the bottleneck itself, at which point the Redis layer needs its own scaling strategy (OneUptime, 2026).

This introduces problems that don't exist in a single-server design. Two requests from the same client arriving within the same millisecond can land on two different application servers simultaneously; both check the shared Redis counter, both see it under the limit, and both get approved, letting the client briefly exceed its cap by a small margin. Engineering teams handling this at scale often accept a looser effective limit rather than chase perfect precision, treating a stated limit of 1,000 requests per minute as reliably enforcing somewhere between roughly 800 and 1,200 in practice, and tuning the buffer based on how strict the use case demands (OneUptime, 2026).

What happens if the shared rate-limit store goes down

Because the shared counter store, usually Redis, becomes a dependency that every request now passes through, its own availability matters. Teams have to decide explicitly whether to fail open, letting requests through unchecked if the rate limiter itself is unreachable, or fail closed, blocking everything until it recovers. Most public APIs choose to fail open, treating a temporary loss of rate-limit enforcement as a smaller risk than an outage that blocks every legitimate customer at once (OneUptime, 2026).

This is the same tradeoff a physical security checkpoint faces during a system failure: some buildings default to letting people through unchecked rather than trapping everyone outside, accepting a temporary security gap over a hard operational stop. Neither choice is universally correct, it depends on what the rate limiter is actually protecting.

Where rate limits typically get enforced in a request's path

Rate limiting isn't applied at just one point in a system. Different layers enforce it for different reasons, and large APIs often stack more than one of the following:

• At the API gateway layer, before a request ever reaches application code, using infrastructure like Cloudflare, AWS API Gateway, or Nginx's built-in rate-limiting modules. This is the cheapest place to reject abusive traffic because it never touches the actual application servers or database.
• Per API key or authentication token, which lets a provider offer different limits to different customer tiers, a common structure in paid API products where a free tier might get 100 requests per hour and a paid tier gets 10,000.
• Per IP address, useful for unauthenticated endpoints like public search or login pages where no API key exists yet to identify the client.
• Per endpoint, since a computationally expensive endpoint, like one running a complex database aggregation, often needs a stricter limit than a lightweight one, like fetching a static resource. An API that relies on well-tuned indexes to keep that expensive endpoint fast in the first place, as covered in why database indexes speed up queries, still benefits from capping how often it gets hit, since even an indexed query has a real per-request cost at high volume.
• At the CDN edge, geographically close to the requester, which can stop volumetric abuse before it ever reaches origin infrastructure. This overlaps with how CDN caching speeds up websites by serving repeat requests from an edge location instead of round-tripping to the origin server.

Layering these matters because each layer catches a different failure mode. A gateway-level IP limit stops a crude flood attack cheaply, while a per-API-key limit enforces the actual business terms of a paid product, and neither one alone covers both jobs well.

What the client sees when a rate limit is hit, and why the headers matter

When a rate limit is triggered, a well-designed API doesn't just return a bare 429 and go silent. It typically includes a set of standardized headers alongside the status code so client software can react intelligently instead of guessing. The Retry-After header tells the client exactly how many seconds to wait before trying again, a mechanism explicitly described in RFC 6585 (IETF, 2012). Many APIs also add X-RateLimit-Remaining, showing how many requests are left in the current window, and X-RateLimit-Reset, indicating when the window rolls over.

This detail matters more than it might seem. A client library that respects Retry-After backs off automatically and resumes cleanly once the window clears. A client that ignores it, or one built against an API that doesn't send the header at all, tends to retry immediately and repeatedly, which is precisely the behavior that turns a routine throttling event into a self-inflicted denial-of-service problem. Consistent status-code and header behavior is also why how HTTP status codes guide browsers covers status codes as a shared protocol contract rather than a per-API convention: 429 works the same whether the client is a browser fetch call, a mobile app's networking layer, or a third-party integration script.

Does a rate limit apply per request or per user session

It depends entirely on what identifier the API uses to track the client. Per-IP limiting applies to every request from that network address regardless of login state, while per-API-key or per-account limiting tracks a specific authenticated identity across however many requests it sends, independent of which device or IP it comes from.

Rate limiting as a security control, not just a capacity control

Rate limiting isn't only about preventing a server from running out of CPU or memory headroom. It's also a standard defense against a specific and common attack pattern called credential stuffing, where an attacker takes a large list of stolen username-password pairs, usually leaked from an unrelated breach, and automates rapidly trying each combination against a login endpoint, hoping some fraction of users reused the same password elsewhere.

Rate limiting login attempts per IP address or per account slows this kind of automated attack to a pace where it stops being economically viable. If an attacker can only try five login attempts per minute instead of five thousand, testing a million stolen credential pairs shifts from a task measured in minutes to one measured in months, which is enough friction to make most attackers move on to an easier target. This is a case where the rate limit's job isn't fairness or cost control, it's actively slowing down an adversary.

Most rate-limit triggers in production don't come from attackers at all. They come from a client's own retry loop, misfiring, hammering the same endpoint far faster than any human or reasonable automated workflow would.

What people consistently get wrong about rate limiting

A common misconception is that rate limiting exists purely to block malicious attackers. In practice, the majority of real-world rate-limit triggers in production systems come from legitimate but misconfigured clients: a retry loop with no exponential backoff logic, a batch job that fires all its requests simultaneously instead of spacing them out over a queue, or a frontend polling an endpoint far more often than the underlying data actually changes. Distinguishing "malicious" from "misbehaving" traffic at the point of enforcement is often impossible, the rate limiter treats both the same way because from a raw traffic-pattern standpoint, they look nearly identical.

Another mistake is setting limits too aggressively without accounting for legitimate burst patterns. An overly strict fixed-window limit can block a real user who simply opened several browser tabs at once, or a legitimate integration that batches several related calls together for efficiency. This is part of why token bucket algorithms that tolerate reasonable bursts are generally preferred over rigid, unforgiving caps: the goal is shaping traffic to a sustainable average, not punishing every deviation from a perfectly flat line.

A third and more subtle mistake is treating rate limiting as a single global setting rather than something that should vary by endpoint cost and business context. A public read-only search endpoint and an authenticated endpoint that triggers an expensive third-party API call have completely different cost profiles and should almost never share the same limit. Teams that apply one blanket number across an entire API tend to either choke legitimate high-value traffic on cheap endpoints or leave expensive endpoints dangerously exposed.

The takeaway

Rate limiting works by tracking request volume per client over a time window and rejecting or delaying requests that exceed an agreed threshold, most commonly using a token bucket algorithm that permits short, realistic bursts while enforcing a steady average limit, communicated to the client through the standardized 429 status code and headers like Retry-After. At scale, the hard part isn't picking an algorithm, it's coordinating that count across every server instance through a shared store like Redis, accepting a small margin of imprecision as the cost of staying fast. The goal was never to punish traffic. It's to keep one client's request volume, whether from a bug, a cost spike, or an actual attack, from degrading service for everyone else relying on the same API.

---

Read next:

• How HTTP Status Codes Guide Browsers
• How CDN Caching Speeds Up Websites
• Why Database Indexes Speed Up Queries

FAQ

Q: What HTTP status code indicates a client has been rate-limited?

APIs return the 429 "Too Many Requests" status code, formally defined in RFC 6585 (2012), often accompanied by a Retry-After header telling the client how many seconds to wait before sending another request.

Q: What is a token bucket algorithm?

It's a rate-limiting method where each client has a virtual bucket of tokens that refills at a steady rate. Each request consumes one token, allowing short bursts of traffic while enforcing a steady average limit once the bucket is empty.

Q: Why is rate limiting harder across multiple servers?

Each server would otherwise keep its own local counter, letting a client's effective limit multiply by the number of servers. Distributed systems solve this with a shared data store, usually Redis, that every instance checks against instead of counting locally.

Q: Does rate limiting only protect against malicious attacks?

No. Most real-world rate-limit triggers come from legitimate but misconfigured clients, such as retry loops without backoff logic or batch jobs sending requests all at once, not from deliberate attacks.

Q: Why would an API rate-limit login attempts specifically?

Limiting login attempts per IP address or account slows down credential stuffing attacks, where an attacker automates rapid attempts of stolen username-password pairs, making the attack impractically slow and costly to carry out.

Q: What's the difference between token bucket and leaky bucket algorithms?

Token bucket allows bursts by letting unused capacity accumulate as tokens, while leaky bucket smooths traffic to a strictly constant output rate by queuing excess requests, trading burst tolerance for predictable, even throughput.