Fixing Blocking Calls in Async FastAPI Routes

Key takeaways:

  • The fingerprint is latency that climbs with concurrency while the work itself is unchanged.
  • A measured example went 250ms → 650ms → 1150ms at concurrency 1, 4 and 8; the healthy control stayed flat at 250ms.
  • An event-loop lag heartbeat is the cheapest production detector: healthy under 10ms, blocked 990ms.
  • asyncio debug mode confirms that you blocked but does not name the line.
  • One request can never reveal this, which is why it survives code review and staging.

This is the diagnostic companion to Async Correctness and Concurrency. It assumes you already accept that a blocking call inside async def stalls the worker, and that what you need now is to find which call, in a codebase where nobody wrote time.sleep on purpose.

Latency against concurrency for a healthy and a blocked endpoint Two lines plotted against concurrency of one, four and eight. The healthy endpoint stays flat at about 250 milliseconds. The blocked endpoint rises from 250 to 650 to 1150 milliseconds, a straight climb. The fingerprint: median latency versus concurrency 0ms 600ms 1200ms c=1 c=4 c=8 healthy blocked Both endpoints do exactly 250ms of work. At concurrency 1 they are indistinguishable. The slope is the diagnosis.
Two endpoints, identical work, identical single-request latency. Only the slope separates them, which is why this bug reaches production.

The Problem This Solves

The report is always some version of the same thing: the service is fine, and then at a certain traffic level it is comprehensively not fine. Latency on endpoints that have nothing to do with each other rises together. Nothing is CPU-saturated. The database is bored. Restarting helps for a few minutes.

That pattern — unrelated endpoints degrading in lockstep — is the signature of a shared serialised resource, and in a FastAPI worker the shared resource is the event loop. Somewhere, an async def is calling something that does not yield.

Finding it is the hard part, because the offending line rarely looks like blocking. Nobody writes time.sleep. What they write is requests.get, boto3.client(...).get_object, PIL.Image.open, jwt.decode with an expensive KDF, a psycopg2 cursor left over from before the migration, or a library that opened a socket somewhere three call frames down.

Why It Happens: Nothing Enforces the Promise

Declaring a handler async def asserts that every wait inside it yields. Python does not verify that assertion, and neither does FastAPI. There is no error, no warning, and no runtime check — the coroutine simply runs to completion without ever suspending, and the loop, being cooperative, has no way to preempt it.

This is why the bug is so durable. It passes code review, because the diff looks like ordinary code. It passes tests, because tests issue one request at a time. It passes staging, because staging has no concurrency. It fails only under real traffic, and by then the change that caused it is fifty commits back.

So the diagnostic work is not "read the code carefully." It is measurement.

Step 1: Confirm the Fingerprint

Before hunting for a line, establish that you are actually looking at loop blocking rather than a slow dependency. The test is to hold the work constant and vary concurrency.

There is a measurement trap here that is worth calling out, because getting it wrong produces a clean bill of health for a badly broken endpoint. If you time each request from inside its own coroutine, a blocking handler will report perfect latency — coroutine two does not reach its own perf_counter() call until coroutine one has completely finished, so each one only ever measures its own uninterrupted 250ms. The clock has to start once, before any of them run:

async def latency_at(path: str, concurrency: int) -> dict:
    transport = httpx.ASGITransport(app=app)
    async with httpx.AsyncClient(transport=transport, base_url="http://probe") as client:
        t0 = time.perf_counter()          # ONE start instant, shared by every request

        async def timed() -> float:
            await client.get(path)
            return (time.perf_counter() - t0) * 1000

        latencies = await asyncio.gather(*[timed() for _ in range(concurrency)])
    return {
        "concurrency": concurrency,
        "median_ms": round(statistics.median(latencies)),
        "max_ms": round(max(latencies)),
    }

Run against two endpoints that each take 250ms — one awaiting asyncio.sleep, one calling time.sleep. Real output:

$ GET /probe/latency-fingerprint
200 OK
{
  "/healthy": [
    {
      "concurrency": 1,
      "median_ms": 250,
      "max_ms": 250
    },
    {
      "concurrency": 4,
      "median_ms": 250,
      "max_ms": 250
    },
    {
      "concurrency": 8,
      "median_ms": 250,
      "max_ms": 250
    }
  ],
  "/suspect": [
    {
      "concurrency": 1,
      "median_ms": 250,
      "max_ms": 250
    },
    {
      "concurrency": 4,
      "median_ms": 650,
      "max_ms": 1000
    },
    {
      "concurrency": 8,
      "median_ms": 1150,
      "max_ms": 2000
    }
  ]
}

At concurrency 1 the two endpoints are identical — 250ms each. That single row is the whole reason this bug ships.

From there they diverge completely. The healthy endpoint holds 250ms all the way to concurrency 8, because eight overlapping awaits cost the same wall-clock time as one. The suspect endpoint goes 250 → 650 → 1150, and its worst case reaches 2000ms: eight requests × 250ms, served strictly one after another.

The tell is that the slope is roughly work × concurrency / 2 for the median and work × concurrency for the max. That is a queue, and the queue is the loop.

Step 2: Measure Event-Loop Lag

The fingerprint proves you have a problem but requires you to already suspect an endpoint. Event-loop lag is the detector you can leave running everywhere, including in production, and it costs almost nothing.

The idea is simple: a coroutine sleeps for a known interval in a loop and records how much longer than that interval each iteration actually took. When the loop is free, asyncio.sleep(0.01) returns in a shade over 10ms. When something is blocking, the timer fires late by exactly the amount of time the loop was held.

async def heartbeat(interval: float, out: list[float], stop: asyncio.Event) -> None:
    """Record how much longer than `interval` each sleep actually took.

    The excess is time the loop was not free to wake this coroutine up: event-loop lag.
    """
    while not stop.is_set():
        started = time.perf_counter()
        await asyncio.sleep(interval)
        out.append((time.perf_counter() - started - interval) * 1000)

A 10ms heartbeat running alongside four requests at each endpoint:

$ GET /probe/event-loop-lag
200 OK
{
  "heartbeat_interval_ms": 10,
  "blocking_call_duration_ms": 250,
  "healthy": {
    "path": "/healthy",
    "samples": 30,
    "median_lag_ms": 0.1,
    "max_lag_ms": 0.0
  },
  "suspect": {
    "path": "/suspect",
    "samples": 6,
    "median_lag_ms": 0.1,
    "max_lag_ms": 990.0
  }
}

Three things in that transcript matter.

The maximum is the metric, not the median. Both runs show a median lag of 0.1ms. Blocking is bursty by nature — the loop is perfectly healthy right up until it is frozen — so an average hides it completely. Alert on p99 and max.

990ms of lag from a 250ms call. Four requests each blocked for 250ms, and because the heartbeat could not run at all during any of them, it observed the whole contiguous freeze as a single late wakeup.

The sample count is itself a signal. The healthy run collected 30 heartbeat samples in the same wall-clock window where the blocked run collected 6. A heartbeat that stops producing samples is a loop that stopped running coroutines, so a simple "did we get the expected number of ticks?" check catches blocking even without the timing maths.

In a real service, run this as a task started in the lifespan, export the max lag per interval as a gauge, and alert when it exceeds a small multiple of your interval. Correlating those spikes with the endpoints in flight at the time — which you get for free if you already have structured logging with request IDs — narrows the search from "the service" to "this route".

Step 3: What asyncio Debug Mode Actually Gives You

The commonly-recommended next step is asyncio's built-in debug mode, which warns when a callback runs longer than slow_callback_duration. It is worth knowing exactly what it produces before you rely on it, because the expectation is usually wrong.

loop = asyncio.get_running_loop()
loop.set_debug(True)
loop.slow_callback_duration = 0.1      # warn about any callback that runs longer than 100ms

Running the blocking endpoint under that configuration produces this, captured from a real subprocess run:

$ GET /probe/asyncio-debug-mode
200 OK
{
  "slow_callback_duration_s": 0.1,
  "asyncio_warnings": [
    "WARNING asyncio: Executing <Task finished name='Task-1' coro=<main() done, defined at <path>/debug_run.py:26> result=None created at <path>/runners.py:100> took N seconds"
  ]
}

Note what it names: Task-1, and the coroutine main(). It does not name time.sleep, and it does not name the endpoint. asyncio can see that a callback overran its budget, but the callback it can see is the outermost task, and the blocking statement is buried somewhere inside it.

That makes debug mode a good confirmation and a poor search tool. It answers "am I blocking at all?" definitively, which is genuinely useful when you are not yet sure the loop is the problem. It will not hand you a line number. Treat it as the smoke alarm, not the map.

Two further caveats: debug mode adds real overhead, so it is a development and staging tool rather than something to leave on; and it only fires above slow_callback_duration, so a handler blocking for 30ms in a hot path — which will absolutely wreck a high-throughput service — stays silent at the 100ms default. Lower it to 0.05 or below when hunting.

Step 4: Find the Line

With the route identified, the search space is one function body. Three techniques, cheapest first.

Grep the usual suspects. Inside async def bodies, look for requests., urllib, boto3, open(, .read(), time.sleep, subprocess.run, and any ORM call that is not awaited. A synchronous database driver is by far the most common finding, which is why Async Database Sessions exists as a topic.

Bisect by awaiting. Insert await asyncio.sleep(0) at successive points in the handler and re-run the concurrency test. That statement yields to the loop without doing anything else, so the concurrency measurement improves only for the portion of the body before the yield point. When the numbers stop improving as you move it later, the blocking call sits between the last two positions. Crude, and it works on code you do not understand.

Read what the library actually does. The hardest cases are libraries that look async. An SDK exposing async def methods may be wrapping a synchronous client, and a client that accepts a timeout argument is not thereby non-blocking. When in doubt, put a heartbeat next to a single call to it in isolation.

Once located, the fix is a choice between three options: replace the library with an async equivalent, offload the call to a thread as described in Running Sync Code in a Threadpool, or change the handler's declaration to def and let Starlette thread the whole thing — see async def vs def for which of those to pick.

Verification

The fix is verified the same way the bug was found: hold the work constant and vary concurrency. Turn the fingerprint into a regression test so it cannot come back.

async def test_endpoint_stays_flat_under_concurrency(client):
    """Latency at concurrency 8 must not be materially worse than at concurrency 1."""
    async def batch(n: int) -> float:
        t0 = time.perf_counter()
        await asyncio.gather(*[client.get("/report") for _ in range(n)])
        return time.perf_counter() - t0

    one = await batch(1)
    eight = await batch(8)
    assert eight < one * 2, f"8 concurrent took {eight:.2f}s vs {one:.2f}s for one — still blocking"

The × 2 allowance is deliberate. A correct endpoint should be near × 1, and a blocked one will be near × 8, so the threshold sits in a wide empty gap and the test will not flake on a loaded CI machine.

For production, the durable artefact is the lag gauge from step 2 plus an alert on its maximum. It is the one metric that catches blocking introduced by a dependency upgrade rather than by your own code, which is the case no test in your repository will ever cover.

Trade-offs and When Not To

Do not go hunting without the fingerprint. Latency that is high but flat across concurrency is a slow dependency, not a blocked loop, and offloading it to a thread will not help. Confirm the slope first.

Offloading is a bounded fix. Moving a blocking call into the threadpool takes it off the loop, but the limiter defaults to 40 concurrent calls. If the call is both slow and frequent, you have moved the queue rather than removed it, and the honest fix is a different library or a background queue.

Some blocking is acceptable. A few hundred microseconds of CPU in a handler is not worth restructuring. Blocking matters in proportion to duration times frequency; measure before you refactor a JSON parse.

A lag heartbeat measures the worker it runs in. With multiple workers, one blocked process shows healthy lag in the others while still serving a share of your traffic badly. Label the metric by process, or you will see a suspiciously mild average.

FAQ

What does a blocked event loop look like in metrics? Per-request latency rises roughly linearly with concurrency while the work itself is unchanged. In a measured run, a healthy endpoint stayed at 250ms across concurrency 1, 4 and 8, whereas a blocking one went from 250ms to 650ms to 1150ms with a worst case of 2000ms.

How do I measure event-loop lag? Run a background coroutine that sleeps for a fixed interval in a loop and records how much longer than that interval each iteration actually took. The excess is time the loop was not free to wake it. A healthy loop showed a maximum under 10ms; a blocked one showed 990ms.

Does asyncio debug mode tell me which line is blocking? No. It reports that a callback exceeded slow_callback_duration and names the enclosing task, not the blocking statement. It is a reliable yes-or-no answer to whether you are blocking, and a poor answer to where, so use it to confirm rather than to locate.

Why does my endpoint look fast in development? Because one request at a time cannot reveal the problem. A blocking handler and a correct one take exactly the same time to serve a single request. Only overlapping requests expose the difference, so a manual click-through or a single-request test will always pass.

Is wrapping everything in run_in_threadpool a good fix? It is a correct fix for blocking I/O but a bounded one. The threadpool limiter defaults to 40 concurrent calls, so offloading high volumes of slow work moves the queue rather than removing it. Prefer a truly async library where one exists, and reserve offloading for calls you cannot replace.

Can middleware block the loop too? Yes, and it is worse when it does. Middleware runs on every request, so one synchronous call there stalls the loop once per request rather than only on the affected route. Blocking work in middleware usually shows up as uniform latency growth across every endpoint at once.