Optional vs Nullable Fields in Pydantic and FastAPI

Key takeaways:

  • Nullability comes from the annotation (str | None); optionality comes from having a default. They are independent.
  • str | None with no default is required but nullable — omitting it is a 422, sending null is fine.
  • Pydantic v1 gave Optional[x] an implicit None default; v2 does not, which is why migrations produce new missing errors.
  • exclude_unset distinguishes "never sent" from "sent as null"; exclude_none cannot.
  • For PATCH, make every field optional and nullable, then merge with model_dump(exclude_unset=True).

This page belongs to Request Validation Patterns and resolves the single most-conflated pair of concepts at the FastAPI boundary. Every 422 and every 200 below is real output from _verify/examples/val-optional-nullable.py on FastAPI 0.139.2, Pydantic 2.13.4 and Python 3.12.

The two independent axes: required versus optional, and nullable versus not A two-by-two matrix. The columns are whether the field has a default; the rows are whether the annotation admits None. The four cells show the resulting declaration and what happens when the field is omitted or sent as null. no default (required) has a default (optional) str str | None name: str omit => 422 missing null => 422 string_type name: str = "x" omit => 200, uses "x" null => 422 string_type name: str | None omit => 422 missing null => 200, value None name: str | None = None omit => 200, value None null => 200, value None Only model_fields_set tells the bottom-right cell's two paths apart.
Nullability and optionality are separate switches. The bottom-left cell — required but nullable — is the one that surprises people after a v1 migration.

The Problem This Solves

A client sends {"bio": null} to clear a user's bio. Your API returns 200 and the bio is unchanged, because your update code did if patch.bio: user.bio = patch.bio. Another client omits bio entirely and gets 422, because someone wrote bio: str | None expecting that to mean optional. A third writes {"timezone": null} intending "leave it alone" and wipes the field.

All three are the same confusion: "can this be null" and "must this be present" are different questions, and JSON has a third state — absent — that Python's None does not naturally represent.

Why It Happens

Pydantic builds a core schema per field with two independent properties. The annotation decides which values validate: str accepts strings, str | None accepts strings and None. The default decides what happens when the key is absent: if there is one, it is used; if there is not, the field is required and its absence is a missing error.

Nothing links them. str | None says nothing about presence, and = None says nothing about which values are legal — the default is not itself validated by default, which is why x: int = None is accepted at class-definition time and then blows up when you use it.

Pydantic v1 did link them, as a convenience: Optional[str] implied = None. Pydantic v2 removed that special case deliberately, on the grounds that a type should not silently change a field's required-ness. The removal is correct and is also the single largest source of surprise 422s during a v1 to v2 migration — every annotation-only Optional[x] in your codebase became required the day you upgraded.

The Four Cases, With Real Output

One model holding all four combinations:

class FourWays(BaseModel):
    """One model holding all four combinations of required-ness and nullability."""

    required_strict: str                    # required, may not be null
    required_nullable: str | None           # required, MAY be null (no default!)
    optional_strict: str = "default"        # optional, may not be null
    optional_nullable: str | None = None    # optional, may be null


@app.post("/four-ways/")
async def four_ways(payload: FourWays) -> dict[str, Any]:
    return {
        "dumped": payload.model_dump(),
        "fields_set": sorted(payload.model_fields_set),
        "exclude_unset": payload.model_dump(exclude_unset=True),
        "exclude_none": payload.model_dump(exclude_none=True),
    }

Empty body. Only the two fields without defaults are reported:

$ POST /four-ways/  {}
422 Unprocessable Entity
{
  "detail": [
    {
      "type": "missing",
      "loc": [
        "body",
        "required_strict"
      ],
      "msg": "Field required",
      "input": {}
    },
    {
      "type": "missing",
      "loc": [
        "body",
        "required_nullable"
      ],
      "msg": "Field required",
      "input": {}
    }
  ]
}

required_nullable is str | None and it is still required. That is the whole lesson of this page in one transcript. Note also "input": {} — for a missing body field, input is the enclosing object, not null.

The two required fields supplied, one of them as null. Accepted:

$ POST /four-ways/  {"required_strict": "a", "required_nullable": null}
200 OK
{
  "dumped": {
    "required_strict": "a",
    "required_nullable": null,
    "optional_strict": "default",
    "optional_nullable": null
  },
  "fields_set": [
    "required_nullable",
    "required_strict"
  ],
  "exclude_unset": {
    "required_strict": "a",
    "required_nullable": null
  },
  "exclude_none": {
    "required_strict": "a",
    "optional_strict": "default"
  }
}

Look carefully at the last two keys. exclude_unset kept required_nullable: null because the client actually sent it, and dropped optional_nullable because they did not. exclude_none did the opposite: it threw away the deliberate null and kept a default the client never asked for. If you are writing a PATCH handler, exclude_none is almost always the wrong tool.

An explicit null on a non-nullable optional field. Rejected, even though the field has a default:

$ POST /four-ways/  {"required_strict": "a", "required_nullable": "b", "optional_strict": null, "optional_nullable": null}
422 Unprocessable Entity
{
  "detail": [
    {
      "type": "string_type",
      "loc": [
        "body",
        "optional_strict"
      ],
      "msg": "Input should be a valid string",
      "input": null
    }
  ]
}

Having a default does not make None a legal value. optional_strict may be omitted; it may not be nulled. The error type is string_type, not missing — a useful distinction when you are branching on error types in a client.

Setting the optional-nullable field explicitly:

$ POST /four-ways/  {"required_strict": "a", "required_nullable": null, "optional_nullable": "set"}
200 OK
{
  "dumped": {
    "required_strict": "a",
    "required_nullable": null,
    "optional_strict": "default",
    "optional_nullable": "set"
  },
  "fields_set": [
    "optional_nullable",
    "required_nullable",
    "required_strict"
  ],
  "exclude_unset": {
    "required_strict": "a",
    "required_nullable": null,
    "optional_nullable": "set"
  },
  "exclude_none": {
    "required_strict": "a",
    "optional_strict": "default",
    "optional_nullable": "set"
  }
}

fields_set now has three entries. It tracks exactly which keys the client provided — the only place that information survives, since optional_nullable=None from a default and optional_nullable=None from an explicit null are indistinguishable in dumped.

The summary table

DeclarationRequired?null accepted?OmittedExplicit null
x: strYesNo422 missing422 string_type
x: str | NoneYesYes422 missing200, value None
x: str = "d"NoNo200, value "d"422 string_type
x: str | None = NoneNoYes200, value None200, value None

Every cell in that table corresponds to a transcript above or in the example file. The second row is the one people write when they mean the fourth.

Making the intent explicit

A bare required_nullable: str | None reads to most people as "optional". If you genuinely want required-but-nullable, say so:

class Sentinel(BaseModel):
    """Required-but-nullable via Field(...) reads more clearly than a bare annotation."""

    nickname: str | None = Field(...)              # required, null allowed
    avatar_url: str | None = Field(default=None)   # optional, null allowed

$ POST /sentinel/  {"avatar_url": "https://x/y.png"}
422 Unprocessable Entity
{
  "detail": [
    {
      "type": "missing",
      "loc": [
        "body",
        "nickname"
      ],
      "msg": "Field required",
      "input": {
        "avatar_url": "https://x/y.png"
      }
    }
  ]
}

$ POST /sentinel/  {"nickname": null}
200 OK
{
  "dumped": {
    "nickname": null,
    "avatar_url": null
  },
  "fields_set": [
    "nickname"
  ]
}

Field(...) is not decoration. It tells the next reader that the missing default is a decision rather than an oversight, and it survives code review in a way a bare annotation does not. Required-but-nullable is a legitimate contract — "you must tell me the customer's middle name, and null is a valid answer meaning they have none" — but it should look deliberate.

exclude_unset and PATCH Round-Tripping

The canonical use is a partial update where absent means "leave alone" and null means "clear it":

class ProfilePatch(BaseModel):
    """PATCH body: every field optional AND nullable, so absent != explicit null."""

    display_name: str | None = None
    bio: str | None = None
    timezone: str | None = None


STORED: dict[str, Any] = {"display_name": "Ada", "bio": "Engineer", "timezone": "UTC"}


@app.patch("/profile")
async def patch_profile(patch: ProfilePatch) -> dict[str, Any]:
    updates = patch.model_dump(exclude_unset=True)   # only keys the client actually sent
    merged = {**STORED, **updates}
    return {"sent_keys": sorted(updates), "updates": updates, "merged": merged}

Three requests, three different meanings, all real:

$ PATCH /profile  {"bio": "Rewriting my bio"}
200 OK
{
  "sent_keys": [
    "bio"
  ],
  "updates": {
    "bio": "Rewriting my bio"
  },
  "merged": {
    "display_name": "Ada",
    "bio": "Rewriting my bio",
    "timezone": "UTC"
  }
}

$ PATCH /profile  {"bio": null}
200 OK
{
  "sent_keys": [
    "bio"
  ],
  "updates": {
    "bio": null
  },
  "merged": {
    "display_name": "Ada",
    "bio": null,
    "timezone": "UTC"
  }
}

$ PATCH /profile  {}
200 OK
{
  "sent_keys": [],
  "updates": {},
  "merged": {
    "display_name": "Ada",
    "bio": "Engineer",
    "timezone": "UTC"
  }
}

Set, cleared, untouched — three outcomes from one model, and display_name and timezone are never touched in any of them. Swap exclude_unset=True for exclude_none=True and the middle case silently becomes a no-op: the clear-my-bio feature stops working, with no error anywhere. That bug is nearly impossible to find by reading the handler, because the code looks right.

The same mechanism drives model_copy(update=...) and SQLAlchemy partial updates. Whatever the persistence layer, the rule is: compute the update dict with exclude_unset=True, and let absent keys simply not appear.

The limit of this technique

exclude_unset distinguishes two states. Some APIs need three: unset, explicit null, and set to a value — where null itself is a meaningful value distinct from clearing. If None is a legitimate stored value and clearing means something else again, you need a sentinel type rather than None, and a Literal discriminant or a wrapper model to carry it. That is a real design cost; most APIs are better off deciding that null means "clear" and living with two states.

Verification

Assert on model_fields_set in unit tests — it is the property everything else derives from:

def test_absent_and_null_are_distinguishable():
    assert ProfilePatch.model_validate({}).model_fields_set == set()
    assert ProfilePatch.model_validate({"bio": None}).model_fields_set == {"bio"}
    assert ProfilePatch.model_validate({"bio": None}).model_dump(exclude_unset=True) == {"bio": None}

And guard the required-ness contract against accidental change, which is what actually regresses during refactors:

def test_required_fields_are_stable():
    required = {n for n, f in FourWays.model_fields.items() if f.is_required()}
    assert required == {"required_strict", "required_nullable"}

The schema is the other place to look: a required field appears in the required array of the model's JSON Schema, and null acceptance shows up as an anyOf including {"type": "null"}. An OpenAPI diff in CI catches the accidental flip of either, which is the practice recommended in customizing OpenAPI schema generation.

Trade-offs and When Not To

Do not make everything optional and nullable "to be safe". A model where every field can be absent cannot express a create operation — you lose the guarantee that a User has an email, and the check moves into your service layer where it is invisible to clients and to the schema. Use separate models: UserCreate with required fields, UserPatch with everything optional.

Nullable columns are not nullable API fields. A database column that permits NULL is an internal storage decision. Exposing that nullability in the response contract forces every client to handle null forever. If the value is always present in practice, serialize a default and keep the API field non-nullable.

exclude_unset on responses is usually wrong. Omitting keys from a response makes clients defensive about every field. exclude_unset earns its place on the request side and in update payloads sent onward; response shapes should be stable. Where you do need per-endpoint response shaping, excluding fields per endpoint covers the sharper tools.

Adding a required field is a breaking change; adding an optional one is not. This is the single most useful consequence of the distinction. New fields go out with defaults, and are only tightened once client telemetry shows everyone sends them.

FAQ

Does str | None make a field optional in Pydantic v2? No. It makes the field nullable. Optionality comes from having a default. A field annotated str | None with no default is required, and omitting it produces a missing error, though sending an explicit null is accepted.

How do I tell an omitted field from an explicit null in a PATCH body? Declare every field as optional and nullable, then call model_dump(exclude_unset=True). Fields the client never sent are absent from the result, while a field sent as null appears with the value None, which is exactly the distinction a PATCH needs.

Why did my v1 model become required when I migrated to v2? Pydantic v1 gave Optional[x] an implicit default of None. Pydantic v2 removed that, so an annotation-only Optional[x] is now required-but-nullable. Adding = None restores the v1 behaviour, and it is the single most common source of new 422s after a migration.

What is the difference between exclude_unset and exclude_none?exclude_unset drops fields the client never provided, based on model_fields_set. exclude_none drops any field whose value is None regardless of how it got there, so it discards an explicit null the client deliberately sent. For PATCH semantics you want exclude_unset.

Is Field(...) with an ellipsis still meaningful in Pydantic v2? Yes. Field(...) marks a field as required while still letting you attach metadata, and on a nullable annotation it makes the required-but-nullable intent explicit to a reader who would otherwise assume the field was optional.