When FastAPI BackgroundTasks Silently Fails

Key takeaways:

  • BackgroundTasks runs after the response is sent, so a failing task cannot change the status code the client already received.
  • The exception escapes into the ASGI server, where it is logged by uvicorn.error — and nowhere near your application logger.
  • One raising task cancels every task queued behind it, because Starlette awaits them sequentially in one coroutine.
  • Wrap every task callable in a guard that logs the exception with its arguments, and increment a failure metric.
  • If the work must not be lost, the guard is not enough — you need a durable queue.

You shipped a signup endpoint that sends a welcome email through BackgroundTasks. Support reports that some users never receive it. The API returned 200 every time, there is no error in your dashboards, and the endpoint's own logs look perfectly healthy. This page is part of Background Task Processing, and it covers the specific failure mode where the task raised and nothing told you.

Where a background task exception goes A timeline showing the response leaving with status 200, then the background task running and raising, with the exception travelling to the server error log rather than to the client. time handler runs add_task(...) 200 OK sent client is done task runs raises uvicorn.error logger not your app logger client sees success no error reaches it
The response is committed before the task runs, so the exception can only travel upward into the server, never back to the caller.

Why It Happens

BackgroundTasks is a parameter-shaped wrapper around Starlette's BackgroundTasks object. When your path operation returns, FastAPI attaches that object to the Response as response.background. Starlette's Response.__call__ then does three things in order: it sends the http.response.start message with the status and headers, it sends the http.response.body message with the payload, and only then it awaits self.background().

That ordering is the whole story. By the time your task's first line executes, the ASGI server has already flushed the response. The status code is committed. The body is committed. There is no mechanism by which a later exception could turn the 200 into a 500, because HTTP does not allow retracting a response you have already written to the socket.

So where does the exception go? It propagates out of the Response.__call__ coroutine, up through the router, through your middleware stack, and into the ASGI server. Uvicorn catches it at the top of its protocol handler and logs Exception in ASGI application with a traceback on the uvicorn.error logger. That is a different logger from the one your application code uses, and in a lot of production setups it is either not JSON-formatted, not shipped to the same index, or filtered out by a log level rule. The failure exists but it is not where anyone is looking.

There is a second consequence that surprises people more than the first. BackgroundTasks.__call__ iterates the queued tasks in a plain for loop and awaits each one. An exception in task one means tasks two and three never start. A noisy analytics call that fails intermittently will silently stop your audit writes from happening at all.

Note also that exception handlers registered with @app.exception_handler(...) cannot help here. Those handlers are installed by ExceptionMiddleware and ServerErrorMiddleware, which wrap the request — and by the time the background task runs, the response has already travelled back out through that middleware. The stack that would have caught the error has unwound.

Proving It

Here is a self-contained app with two endpoints: one that queues an unguarded task and one that queues a guarded one. To make the invisible visible, a small pure-ASGI middleware plays the role uvicorn plays — it records the response messages as they leave and catches whatever the application raises afterwards.

import logging
import traceback

from fastapi import BackgroundTasks, FastAPI

TIMELINE: list[str] = []


class ListHandler(logging.Handler):
    """Stand-in for a log backend so the transcript can show what was actually logged."""

    def emit(self, record: logging.LogRecord) -> None:
        line = f"log {record.levelname} {record.name}: {record.getMessage()}"
        if record.exc_info:
            line += " | " + traceback.format_exception_only(record.exc_info[1])[0].strip()
        TIMELINE.append(line)


logger = logging.getLogger("app.tasks")
logger.setLevel(logging.INFO)
logger.addHandler(ListHandler())
logger.propagate = False


class ServerBoundary:
    """Plays the role uvicorn plays: it sees the response leave, then the task explode."""

    def __init__(self, app) -> None:
        self.app = app

    async def __call__(self, scope, receive, send) -> None:
        if scope["type"] != "http" or scope["path"] == "/evidence":
            await self.app(scope, receive, send)
            return

        async def send_wrapper(message) -> None:
            if message["type"] == "http.response.start":
                TIMELINE.append(f"response status {message['status']} sent to client")
            elif message["type"] == "http.response.body" and not message.get("more_body"):
                TIMELINE.append("response body flushed: client request is complete")
            await send(message)

        try:
            await self.app(scope, receive, send_wrapper)
        except Exception as exc:
            # uvicorn logs "Exception in ASGI application" here. The client saw 200 long ago.
            TIMELINE.append(f"server caught {type(exc).__name__} AFTER the response: {exc}")


app = FastAPI()
app.add_middleware(ServerBoundary)

DELIVERED: list[str] = []


async def send_receipt(order_id: str) -> None:
    """The task as most people write it: it can raise, and nothing here handles that."""
    TIMELINE.append(f"task send_receipt({order_id}) started")
    if order_id == "bad":
        raise RuntimeError(f"receipt provider rejected order {order_id}")
    DELIVERED.append(order_id)
    TIMELINE.append(f"task send_receipt({order_id}) finished")


@app.post("/orders/{order_id}/unsafe")
async def create_order_unsafe(order_id: str, tasks: BackgroundTasks) -> dict[str, str]:
    tasks.add_task(send_receipt, order_id)
    return {"order_id": order_id, "status": "created"}

The guarded variant wraps the same callable:

def guarded(fn):
    """Wrap a task so a failure is logged with its traceback instead of vanishing."""

    async def runner(*args, **kwargs) -> None:
        try:
            await fn(*args, **kwargs)
        except Exception:
            logger.exception("background task %s failed args=%r", fn.__name__, args)

    runner.__name__ = f"guarded_{fn.__name__}"
    return runner


@app.post("/orders/{order_id}/safe")
async def create_order_safe(order_id: str, tasks: BackgroundTasks) -> dict[str, str]:
    tasks.add_task(guarded(send_receipt), order_id)
    return {"order_id": order_id, "status": "created"}


@app.get("/evidence")
async def evidence() -> dict[str, object]:
    return {"delivered": DELIVERED, "timeline": TIMELINE}

This is the real output of running that app against a successful order, a failing unguarded order, and a failing guarded order:

$ POST /orders/ok-1/unsafe
200 OK
{
  "order_id": "ok-1",
  "status": "created"
}

$ POST /orders/bad/unsafe
200 OK
{
  "order_id": "bad",
  "status": "created"
}

$ POST /orders/bad/safe
200 OK
{
  "order_id": "bad",
  "status": "created"
}

$ GET /evidence
200 OK
{
  "delivered": [
    "ok-1"
  ],
  "timeline": [
    "response status 200 sent to client",
    "response body flushed: client request is complete",
    "task send_receipt(ok-1) started",
    "task send_receipt(ok-1) finished",
    "response status 200 sent to client",
    "response body flushed: client request is complete",
    "task send_receipt(bad) started",
    "server caught RuntimeError AFTER the response: receipt provider rejected order bad",
    "response status 200 sent to client",
    "response body flushed: client request is complete",
    "task send_receipt(bad) started",
    "log ERROR app.tasks: background task send_receipt failed args=('bad',) | RuntimeError: receipt provider rejected order bad"
  ]
}

Read the timeline carefully. In every case the status line and the body are flushed before the task starts. The failing order still answered 200. The only difference between the second and third requests is where the evidence ended up: in the unguarded case the RuntimeError reached the server boundary, where in production it becomes a uvicorn.error traceback nobody has alerted on; in the guarded case it reached the application logger with the task name and its arguments attached.

The Fix

The decorator above is the minimum. In a real service, make the guard do three jobs: log with context, count the failure, and decide whether the work needs a durable home.

import logging

logger = logging.getLogger("app.tasks")


def background_safe(fn):
    """Decorate task callables so a failure is observable and cannot cancel sibling tasks."""

    async def runner(*args, **kwargs):
        try:
            return await fn(*args, **kwargs)
        except Exception:
            logger.exception(
                "background_task_failed",
                extra={"task": fn.__name__, "args": repr(args)},
            )
            TASK_FAILURES.labels(task=fn.__name__).inc()   # Alert on this, not on the log.

    runner.__name__ = fn.__name__
    return runner


@background_safe
async def send_receipt(order_id: str) -> None:
    ...

Because the guard swallows the exception, the tasks queued behind it still run — that alone fixes a class of bug where a flaky third-party call quietly disabled the rest of your post-response work.

The counter matters more than the log line. A log you have to go looking for is not observability; a task_failures_total counter with a rate alert is. Wiring that up is covered in Prometheus metrics for FastAPI, and giving the log line the same request id as the originating request is covered in correlating logs, traces and errors — worth doing, because a background task failure with no request id is nearly impossible to trace back to the user it affected.

Do not forget the second half of the guard's contract: swallowing the exception means the work is now definitely lost rather than probably lost. That is fine for a metrics ping and wrong for a payment capture.

Verification

Two checks belong in your test suite. The first asserts the endpoint responds successfully even when the task will fail, which documents the behaviour rather than leaving it as a surprise:

def test_response_succeeds_even_when_task_fails(client, caplog):
    response = client.post("/orders/bad/safe")
    assert response.status_code == 200            # The client is never told.
    assert "background_task_failed" in caplog.text  # But the failure is recorded.

The second asserts the guard actually protects siblings:

def test_failing_task_does_not_cancel_the_next_one(client):
    client.post("/orders/bad/two-tasks")
    assert AUDIT_WRITES == ["order bad"]   # Ran despite the earlier task raising.

Note that TestClient re-raises exceptions from background tasks by default, which is why the unguarded version of this test fails loudly in CI while passing silently in production. That asymmetry is itself a useful signal: if a background-task test blows up under TestClient, you have found an unguarded task.

In production, the check is a dashboard panel: request rate for the endpoint next to task_failures_total for its task. If the first is flat and the second is climbing, you are returning success while doing nothing.

Trade-offs and When Not To Use This

Guarding tasks makes failures visible, but it does not make them recoverable. BackgroundTasks has no persistence: the queue lives in a Python object attached to a response in one process. A rolling deploy that terminates the worker between the response and the task drops everything queued, and no amount of exception handling changes that.

Use BackgroundTasks when the work is short, idempotent-by-accident, and genuinely optional — cache warming, a best-effort notification, an analytics ping. Move to a durable queue when losing the work costs money or trust. The comparison of the options is in FastAPI BackgroundTasks vs Celery vs ARQ, and once you are on a queue the next problem is retry safety, covered in retry and idempotency for tasks.

There is also a capacity trade-off. Because the task runs in the same process on the same event loop, a slow task holds that worker's concurrency slot after the client has gone. Under load you can end up with a queue of invisible post-response work that shows up as latency on other requests, with nothing in your request metrics to explain it. If your tasks are not sub-second, they do not belong here regardless of how well you log them.

FAQ

Why does my client get a 200 when the background task raised? Because the response is already sent before the task runs. Starlette returns the response to the ASGI server first, then awaits the background tasks, so by the time your task raises there is no status code left to change and no body left to write.

Where does the exception from a BackgroundTask actually go? It propagates out of the ASGI application into the server. Uvicorn logs it as 'Exception in ASGI application' on the uvicorn.error logger. If that logger is not configured, filtered out, or shipped, the failure is invisible.

Do the remaining background tasks run if one of them raises? No. Starlette awaits the tasks in order in a single coroutine, so the first exception aborts the rest of the list. A failing audit-log task will stop a later email task from ever running.

Should I add a global exception handler to catch background task failures? It will not help. Exception handlers wrap the request/response cycle, and background tasks run after that cycle has completed, so the handler is no longer in the call stack. Wrap the task callable itself instead.

When should I move off BackgroundTasks entirely? As soon as the work must not be lost. BackgroundTasks has no persistence, no retry, and no dead-letter destination, so a deploy or a crash between the response and the task drops the work with no record of it.