Rate Limiting and Throttling in FastAPI

Rate limiting is how an API keeps one caller from consuming capacity that belongs to everyone. It bounds what a single client can do in a period of time, so a runaway retry loop, an over-eager integration, or a deliberate abuser degrades their own experience rather than the service.

This topic is part of Async, Background Tasks and Observability. A limiter is the edge defence in front of the async database pool, and it is implemented either as middleware or as a dependency — a choice with consequences this page measures.

The three guides beneath this page cover the practical paths: FastAPI rate limiting with Redis and SlowAPI for adopting a library, per-user token bucket throttling for implementing the algorithm yourself, and rate limit headers and 429 responses for the response clients actually consume.

Three Decisions, In Order

Almost every rate limiting problem traces back to one of three decisions, and they have a natural order because each constrains the next.

Who are you limiting? The key. This is the decision people spend the least time on and regret the most, because a limiter keyed on something the client controls is not a limiter — it is a suggestion. An API key in a header can be rotated. An IP address can be changed, shared by thousands of users behind one corporate NAT, or replaced entirely by your load balancer's address. The authenticated principal is the only key a client cannot trivially change, which is why authenticated and unauthenticated endpoints usually need different limiting strategies rather than the same one applied twice.

How much, over what shape of time? The algorithm. A fixed-window counter is one integer and an expiry; a token bucket is a balance and a timestamp; a sliding window is a list of timestamps. They differ in how they treat bursts and in what they cost to store, and the differences are large enough to be worth understanding before picking.

Where does the check run? The placement. Middleware and dependencies see different traffic, as the measurement below shows, and neither is a superset of the other in the way people assume.

Prerequisites

Authentication resolves before the limiter runs. If you intend to key on the principal, the principal must exist by then. A limiter that runs in middleware ahead of authentication can only see headers, which brings you back to keying on something spoofable.

A shared store is available. In development one process makes an in-memory counter look correct. It will not be correct in production, and this is the single most common way a limiter ships broken.

Client identity survives your proxy. Behind a load balancer, request.client.host is the balancer. Configure Uvicorn's --proxy-headers with --forwarded-allow-ips restricted to your proxy, so forwarded headers are honoured only from a source you trust.

Core Mechanics: The Counter Must Be Shared

A rate limiter is a counter, a key, and a clock, and the counter is where deployments break.

FastAPI in production runs several Uvicorn workers, each a separate OS process with its own memory. A module-level dictionary is therefore not one counter but N counters. A client whose requests are distributed across four workers sends four times the configured limit before anything triggers, and because load balancing is not perfectly even, the effective limit also fluctuates. Every worker's logs say the limiter is working.

Moving the counter to Redis fixes that, and introduces a second problem: the check is now a read-modify-write across a network. Two workers can read the same remaining count, both conclude the request is allowed, and both write back. The limit leaks by roughly the number of concurrent workers — and it leaks precisely when a client is sending fast enough for the requests to overlap, which is when you needed it.

The fix is to make the whole operation atomic in the store rather than in your code. A single INCR with an expiry gives that for a fixed window. Anything more sophisticated needs a Lua script, so refill and deduction execute without interleaving, which is what per-user token bucket throttling builds.

async def enforce_limit(redis: Redis, key: str, limit: int, window: int) -> None:
    # One atomic increment; the expiry is set only on the first hit of a window.
    count = await redis.incr(key)
    if count == 1:
        await redis.expire(key, window)
    if count > limit:
        ttl = await redis.ttl(key)
        raise HTTPException(429, "rate limit exceeded", headers={"Retry-After": str(ttl)})

Note the subtlety in that expiry: it is set after the first increment, so a crash between the two calls leaves a key with no TTL and a client limited forever. SET key 1 EX window NX followed by INCR avoids it, as does doing both in one script — a small illustration of how quickly "just use a counter" acquires edge cases.

Production Implementation: Placement Decides What Is Counted

Middleware and dependencies are usually described as a style preference. They are not: they see different traffic. The app below counts the same requests in both positions.

@app.middleware("http")
async def middleware_limiter(request: Request, call_next):
    if request.url.path.startswith(("/counts", "/reset")):
        return await call_next(request)
    COUNTED["middleware"] += 1     # Counted before routing, before validation.
    return await call_next(request)


async def dependency_limiter() -> None:
    # Counted only once a route MATCHED — but, as the transcript shows, still counted when
    # that route's parameters then fail validation. Unmatched paths never reach here.
    COUNTED["dependency"] += 1


@app.get("/items/{item_id}", dependencies=[Depends(dependency_limiter)])
async def read_item(item_id: int) -> dict[str, int]:
    return {"item_id": item_id}

Three requests — one valid, one that fails validation, one that matches nothing:

$ GET /items/1
200 OK
{
  "item_id": 1
}

$ GET /items/abc
422 Unprocessable Entity
{
  "detail": [
    {
      "type": "int_parsing",
      "loc": [
        "path",
        "item_id"
      ],
      "msg": "Input should be a valid integer, unable to parse string as an integer",
      "input": "abc"
    }
  ]
}

$ GET /nope
404 Not Found
{
  "detail": "Not Found"
}

$ GET /counts
200 OK
{
  "middleware": 3,
  "dependency": 2
}

Three versus two, and the gap is exactly the 404. Two things follow, and the second is not what most people expect.

A dependency limiter cannot see unmatched paths. A scanner probing thousands of random URLs is invisible to it. Every one of those requests still costs you TLS termination, ASGI parsing and routing, so an API whose only limiter is a dependency has no protection against the cheapest kind of abuse. That is the case for a global middleware limit underneath the per-route ones.

A dependency limiter does count requests that fail validation. The 422 was counted. Dependencies run once the route matches, and parameter validation failure does not prevent them. This is worth knowing because it is often assumed to work the other way: a client sending malformed requests will burn its quota, which is usually the behaviour you want — a client in a broken retry loop should be throttled — but it means a client can be limited out of your API without a single request reaching a handler, and your handler-level metrics will show nothing.

The practical arrangement is layered. A broad middleware limit keyed on IP protects against unmatched-path floods and applies before any application code runs. Per-route dependency limits keyed on the authenticated principal express business policy — five exports per hour on the free tier — where the typed user object is available.

def rate_limit(limit: int, window: int):
    async def _dep(request: Request, user: Annotated[User, Depends(current_user)]) -> None:
        await enforce_limit(request.app.state.redis, f"rl:{user.id}", limit, window)
    return _dep


# Expensive endpoint, tighter policy, keyed on a principal the client cannot change.
@router.post("/exports", dependencies=[Depends(rate_limit(limit=5, window=3600))])
async def create_export() -> dict[str, str]:
    return {"status": "queued"}

Comparing the Algorithms

The fixed window's boundary flaw is not theoretical, and the SlowAPI guide measures it admitting exactly twice the configured limit within one second. Whether that matters depends on what the limit protects: for a downstream service with its own hard quota, it matters a great deal.

Token bucket is the default worth reaching for. Its two parameters map onto the two things you actually care about — the burst you will tolerate and the sustained rate you can serve — and it produces an exact Retry-After for free, because the deficit and the refill rate give the wait time directly.

Async and Performance Notes

The limiter runs before everything else on every request, so its own cost is a floor on your latency.

One round trip, not several. A check that issues GET, then INCR, then EXPIRE triples both the latency and the race window. Use one atomic operation or one script.

The client must be async. A synchronous Redis client inside an async def blocks the event loop for the duration of the network call, on every request. This is the blocking-call problem in its most damaging position.

Decide what happens when the store is down. Failing open serves unlimited traffic; failing closed rejects everything and turns a Redis blip into a full outage. Most APIs should fail open with a loud alert, because a limiter is a protection mechanism and not a correctness one — but that must be a decision you made and tested, not a default you discovered during an incident.

Keep keys short-lived. Every key needs a TTL comfortably longer than its window and no longer, or memory grows with your total user count instead of with your active one.

Testing Strategy

Test against the store you deploy. The assertion that matters builds two app instances sharing one Redis and checks the second sees the first's counter. That is what catches the in-memory default, and no single-app test can.

Inject the clock. Refill and expiry are time-dependent, and a test that calls time.sleep(60) will be deleted by someone. Pass a clock in so tests advance it instantly.

Assert the response, not just the status. A 429 without Retry-After is a broken 429. Assert the headers as well, per rate limit headers and 429 responses.

Override the limiter in unrelated tests. Every other test in your suite will eventually trip a limit and fail intermittently. dependency_overrides with a no-op limiter keeps that from happening, and is far better than raising limits in the test config until the flakes stop.

def test_business_logic_without_limits(client):
    app.dependency_overrides[rate_limit_dep] = lambda: None   # Not under test here.
    for _ in range(50):
        assert client.post("/exports").status_code == 200

Failure Modes and Diagnosis

The limit is roughly N times too high. N is your worker count and the counter is in process memory. Diagnose by checking whether the store keyspace is empty while limits still occasionally fire.

Every user is limited at once. You are keying on request.client.host behind a proxy, so all users share the balancer's address. Diagnose by logging the key for a few requests; if they are identical across different users, that is it.

Limits are enforced but clients ignore them. The 429 has no Retry-After, so clients retry immediately and make the overload worse. Check the response headers, not the status code.

Bursts slip through under load. A non-atomic check-then-increment. Diagnose by firing concurrent requests at one key and comparing the admitted count against the limit; a non-atomic implementation admits more.

Legitimate clients complain sporadically. A fixed window rejecting a burst the client considers normal, or a boundary interaction between a global and a per-route limit. Log which limit fired, always — a 429 that does not say which policy rejected it is undiagnosable.

A client evades the limit entirely. The key is client-controlled. Rotating API keys or spoofed forwarded headers both do this. Move to the authenticated principal.

FAQ

Why does rate limiting need a shared store like Redis? Because an API runs many worker processes and often many machines, and an in-memory counter only sees its own process. A client spread across four workers can send roughly four times the limit while every worker believes it is enforcing the policy. A shared store gives all workers one authoritative counter.

What is the difference between token bucket and sliding window? A token bucket allows a burst up to its capacity and then refills at a steady rate, which suits clients that spike occasionally. A sliding window counts requests over a continuously moving period, which is stricter and easier to describe in documentation but rejects bursts a bucket would absorb.

Should rate limiting be middleware or a dependency? Middleware sees every request, including ones that match no route, so it is right for a global default and for protecting against scanners. A dependency sees only requests whose route matched, and has typed access to the authenticated user, so it is right for per-tier policies. Most APIs need both.

Does a dependency-based limiter count requests that fail validation? Yes. Once a route matches, its dependencies run even if the path or query parameters then fail validation and the request returns 422. Only requests that match no route at all bypass a dependency limiter, which is measured on this page.

What should a rate-limited response contain? HTTP 429 with a Retry-After header giving the wait in seconds, plus headers describing the limit, the remaining quota and the reset time. A 429 without Retry-After tells a client it must back off but not for how long, so it guesses, usually badly.

Can clients evade a limit keyed on IP address? Easily, through proxies or a large address pool, and behind a load balancer the IP may not even identify the client. Key on the authenticated principal wherever one exists, and treat IP-based limits as a coarse defence for unauthenticated endpoints only.

Related Reading

• Up to the section: Async, Background Tasks and Observability.
• Adopting a library: FastAPI rate limiting with Redis and SlowAPI — storage strategies, the mandatory Request parameter, and proxy-safe identity.
• Implementing the algorithm: Per-user token bucket throttling — capacity versus refill, lazy refill, and making the update atomic.
• The response clients consume: Rate limit headers and 429 responses — Retry-After, quota headers, and when 503 is more honest.
• Composes with: Middleware Implementation for global placement, Dependency Injection Strategies for per-route policy, and Async Database Sessions for the pool a limiter protects.