Cache Invalidation Patterns in FastAPI

Key takeaways:

  • Deleting the record's own key does not touch the list keys that embed it — the usual stale bug.
  • Versioned prefixes retire an entire group with one increment and no scanning.
  • TTL is not an invalidation strategy; it is the backstop that bounds every mistake above.
  • A version bump leaves the old generation resident until it expires, so budget for two generations.
  • Invalidate synchronously in the write path, and broadcast for other instances separately.

This guide covers the hard half of Caching Strategies: keeping cached data honest. It assumes you already have a working cache-aside setup, such as the one in Redis response caching in FastAPI, and is about what happens to it when the underlying data changes. For caching computed values inside a request rather than whole responses, see caching dependency results.

The Problem This Solves

Caching is easy until something changes. A price update goes out, the endpoint keeps returning the old number, and nothing anywhere raises an error. Support sees an intermittent issue; you see a cache hit. The failure is silent, plausible-looking, and erodes trust in the API in a way an outage does not — an outage is at least honest about being broken.

The interesting failures are almost never the one everybody thinks of. Deleting product:1 when product 1 changes is the obvious step, and teams do it. What gets missed is everything else that embedded product 1.

Why It Happens: Keys Are Derived, Deletion Is Not

A cache key names a computation, not a record. product:1 is "the serialized form of product 1", but product:list is "the serialized form of a query whose result happens to contain product 1", and catalog:search:widget may be a third. When product 1 changes, all three are stale, yet only the first has a name derivable from the record's identity.

This is what makes invalidation genuinely hard rather than merely tedious. The mapping from "record that changed" to "keys now wrong" lives in the head of whoever wrote the caching code. It is not checked by the compiler, not visible in the type system, and not exercised by any test that does not specifically write and then read.

Two strategies exist for coping. Enumerate the derived keys and delete them all — precise, and correct exactly until someone adds a fourth cached query. Or make the whole group unreachable at once by putting a version number in the key prefix, so you never need the mapping. The example below runs both.

Why deleting one key leaves the cache staleA write to product 1 deletes the product 1 key but not the product list key, which still embeds the old copy of product 1. A version bump retires both keys at once by changing the prefix.PATCH product 1name changedv1:product:1deleted, now correctv1:product:listuntouched, now STALEbump to v2both unreachableTTL expirythe last backstop

Prerequisites

  • A cache-aside read path — check the cache, fall through to the database, populate.
  • An async Redis client in production. The example below uses a dictionary so the behaviour can be executed and shown; the key scheme and the deletes are what transfer.

The Fix

This app implements all four strategies against one small dataset, and the frozen clock makes TTL expiry observable rather than theoretical.

CLOCK = {"now": 0.0}                    # Frozen clock; real code uses time.monotonic().
CACHE: dict[str, tuple[object, float]] = {}   # key -> (value, expires_at)
VERSION = {"catalog": 1}


def cache_get(key: str) -> object | None:
    entry = CACHE.get(key)
    if entry is None or entry[1] <= CLOCK["now"]:
        CACHE.pop(key, None)          # Expired entries are indistinguishable from absent ones.
        STATS["misses"] += 1
        return None
    STATS["hits"] += 1
    return entry[0]


def cache_set(key: str, value: object, ttl: float = 300.0) -> None:
    CACHE[key] = (value, CLOCK["now"] + ttl)


def catalog_key(suffix: str) -> str:
    # Versioned prefix: bumping the version retires every key in the group at once.
    return f"catalog:v{VERSION['catalog']}:{suffix}"


@app.get("/products/{product_id}")
async def read_product(product_id: int) -> dict[str, object]:
    key = catalog_key(f"product:{product_id}")
    cached = cache_get(key)
    if cached is not None:
        return {"source": "cache", "key": key, "data": cached}
    value = dict(DB[product_id])
    cache_set(key, value)
    return {"source": "db", "key": key, "data": value}


@app.get("/products")
async def list_products() -> dict[str, object]:
    """A derived key: it embeds product data, so a per-product delete does not touch it."""
    key = catalog_key("product:list")
    cached = cache_get(key)
    if cached is not None:
        return {"source": "cache", "key": key, "data": cached}
    value = [{"id": p["id"], "name": p["name"]} for p in DB.values()]
    cache_set(key, value)
    return {"source": "db", "key": key, "data": value}


@app.patch("/products/{product_id}")
async def update_product(product_id: int, changes: dict) -> dict[str, object]:
    DB[product_id].update(changes)
    key = catalog_key(f"product:{product_id}")
    existed = CACHE.pop(key, None) is not None   # Write-path delete, same transaction.
    return {"updated": product_id, "deleted_key": key, "key_existed": existed}


@app.post("/catalog/bump")
async def bump() -> dict[str, object]:
    VERSION["catalog"] += 1
    return {"version": VERSION["catalog"], "keys_still_in_store": sorted(CACHE)}

The sequence below reads, writes, reads again, then repeats with the list endpoint in play. This is the real transcript:

$ GET /products/1
200 OK
{
  "source": "db",
  "key": "catalog:v1:product:1",
  "data": {
    "id": 1,
    "name": "Widget",
    "price": 10
  }
}

$ GET /products/1
200 OK
{
  "source": "cache",
  "key": "catalog:v1:product:1",
  "data": {
    "id": 1,
    "name": "Widget",
    "price": 10
  }
}

$ PATCH /products/1  {"name": "Widget Pro"}
200 OK
{
  "updated": 1,
  "deleted_key": "catalog:v1:product:1",
  "key_existed": true
}

$ GET /products/1
200 OK
{
  "source": "db",
  "key": "catalog:v1:product:1",
  "data": {
    "id": 1,
    "name": "Widget Pro",
    "price": 10
  }
}

$ GET /products
200 OK
{
  "source": "db",
  "key": "catalog:v1:product:list",
  "data": [
    {
      "id": 1,
      "name": "Widget Pro"
    },
    {
      "id": 2,
      "name": "Gadget"
    }
  ]
}

$ PATCH /products/1  {"name": "Widget Max"}
200 OK
{
  "updated": 1,
  "deleted_key": "catalog:v1:product:1",
  "key_existed": true
}

$ GET /products
200 OK
{
  "source": "cache",
  "key": "catalog:v1:product:list",
  "data": [
    {
      "id": 1,
      "name": "Widget Pro"
    },
    {
      "id": 2,
      "name": "Gadget"
    }
  ]
}

$ POST /catalog/bump
200 OK
{
  "version": 2,
  "keys_still_in_store": [
    "catalog:v1:product:list"
  ]
}

$ GET /products
200 OK
{
  "source": "db",
  "key": "catalog:v2:product:list",
  "data": [
    {
      "id": 1,
      "name": "Widget Max"
    },
    {
      "id": 2,
      "name": "Gadget"
    }
  ]
}

$ POST /advance/301
200 OK
{
  "now": 301.0
}

$ GET /stats
200 OK
{
  "now": 301.0,
  "stats": {
    "hits": 2,
    "misses": 4
  },
  "keys": [
    "catalog:v1:product:list",
    "catalog:v2:product:list"
  ]
}

Reading the failure

The write-path delete works for the direct key. After the first PATCH, the next read reports "source": "db" and the new name. That much is the pattern everybody implements, and it is correct.

Then the same pattern fails. The second PATCH sets the name to Widget Max and again deletes catalog:v1:product:1. The next GET /products returns "source": "cache" with "name": "Widget Pro" — a value that is now wrong in the database and wrong relative to the single-product endpoint. Two endpoints in the same API disagree about the same record. This is the bug in production, and it did not require anyone to forget the invalidation: the invalidation was written, ran, and reported success.

The version bump fixes it without knowing the key. Incrementing to v2 moves the read path to a prefix under which nothing is cached, so the next list read goes to the database and returns Widget Max. Note that no delete was issued and no key was enumerated; the stale entry was made unreachable rather than removed.

Two generations coexist. The final key listing shows catalog:v1:product:list and catalog:v2:product:list both resident. The v1 entry is unreachable but still occupying memory until its TTL fires. That is the real cost of versioned invalidation, and it means peak memory is roughly two generations of the group — plan capacity accordingly, and never omit the TTL.

The clock advanced past the TTL and the keys are still listed. now: 301.0 is past the 300-second expiry, yet both keys appear in the store. Nothing swept them, because expiry is evaluated lazily on read. Redis behaves the same way for most purposes — it samples keys for active expiry, but an untouched key can outlive its TTL in memory. Never treat "the TTL passed" as "the memory came back".

The Repopulation Race

There is a window the transcript above cannot show, because it needs two concurrent requests. A reader misses the cache and fetches from the database; before it writes what it read, a writer updates the row and deletes the key; the reader then writes its pre-update value. The delete happened, and the cache is stale anyway.

Ordering the write path as update the database, then delete the key — rather than delete-then-update — shrinks the window to the duration of the write itself. Closing it entirely requires the cached entry to carry the version it was read at, so a stale writer's entry lands under a prefix nobody will look up. This is a second, quieter argument for versioned keys.

For most APIs the pragmatic answer is a short TTL on keys with high write rates, accepting a bounded staleness window rather than paying for locks on the read path.

Cross-Instance Propagation

Everything so far assumes one shared cache. If each instance also keeps an in-process cache, a delete in one process is invisible to the others, and the version counter is the only thing keeping them consistent — which works, since all instances read the version from the shared store.

Where instances cache the version itself for speed, they need a nudge. Publishing on a Redis channel is enough:

# On a write, tell every instance to drop its local copy.
await redis.publish("invalidate", json.dumps({"group": "catalog"}))

Treat that broadcast as a latency optimisation, not a guarantee. Subscribers miss messages during reconnects, so the local copy of the version still needs its own short TTL. Correctness comes from the TTL; the broadcast only makes convergence fast.

Verification

The test that matters is write-then-read, and it must cover the derived keys, since those are what break:

async def test_update_is_visible_everywhere(client):
    client.get("/products/1")                       # Populate both caches.
    client.get("/products")
    client.patch("/products/1", json={"name": "new"})

    assert client.get("/products/1").json()["data"]["name"] == "new"
    listed = client.get("/products").json()["data"]
    assert listed[0]["name"] == "new"               # The one that actually fails.

Write that second assertion for every list or search endpoint that embeds a cacheable record. It is three lines and it catches the entire class of bug described above.

For a running service, the diagnostic is comparing a cached response against a deliberately uncached one — a ?nocache=1 parameter guarded by an internal header is worth having. If they differ, you have found a missing invalidation and you know exactly which key.

Trade-offs and When Not To

Precise invalidation does not survive growth. Enumerating derived keys is correct on the day it is written and wrong the first time someone adds a cached query without updating the list. Versioned prefixes trade some memory for a scheme that cannot be forgotten.

Group versioning over-invalidates. One product change retires the whole catalog group, including entries that were still perfectly valid. On a write-heavy dataset this collapses your hit rate to nothing. Scope version groups to sets of data that genuinely change together.

Some data should not be cached. If it changes on nearly every read, or if serving a stale value has real consequences — balances, permissions, inventory at checkout — the correct amount of caching is none. Reach for a read replica instead.

TTL alone is a legitimate strategy. For data where staleness measured in seconds is acceptable, skip the invalidation machinery entirely and set a short expiry. The complexity above is worth it only when you need both freshness and a long TTL.

FAQ

Why is cache invalidation considered hard? Because correctness depends on every code path that mutates data also knowing every cache key derived from it, and that mapping lives in people's heads rather than in the type system. A single forgotten write path or one overlooked list key serves stale data indefinitely, and nothing raises an error when it does.

What is versioned-key invalidation? You prefix a group of cache keys with a version number, such as catalog:v7:item:1, and invalidate the whole group by incrementing the version to v8. Every old key becomes unreachable at once and expires on its own TTL, which turns group invalidation into a single atomic increment with no scanning or bulk deletion.

Do orphaned keys from a version bump waste memory? Briefly, yes. The old keys stay resident until their TTL expires, so peak memory is roughly two generations of the group. That is the trade for atomic group invalidation, and it is why every key needs a TTL even when you also invalidate explicitly.

Should invalidation be synchronous with the write or eventual? Delete the affected keys synchronously in the write path so the very next read is correct, then publish an event for other instances to invalidate their own copies. Local correctness is immediate and cross-instance convergence is eventual, which is the strongest guarantee you can get cheaply.

What happens if a read repopulates the cache during a write? It can re-cache the pre-write value, so the delete appears to have been undone. Reading, then writing, then deleting the key after the write commits narrows the window; making the cache entry version-stamped closes it, because a stale writer's entry is no longer reachable.