Managing Environment Variables with Pydantic Settings

Key takeaways:

  • Declare each environment variable as a typed field on a BaseSettings model so it is parsed and validated, not just read as a string.
  • Use env_prefix and extra="forbid" to namespace variables and catch typos at boot.
  • Type credentials as SecretStr so they never leak into logs or tracebacks.
  • Cache the settings accessor with lru_cache so parsing happens exactly once.
  • Override values through real environment variables per environment — no code changes between CI, staging, and production.

This is the hands-on guide under Configuration Management. It assumes you have read that page's argument for treating configuration as a typed, injected dependency.

The Problem This Solves

Raw os.environ["DATABASE_URL"] reads scatter configuration across the codebase, return untyped strings, and only fail when a request happens to touch the missing variable. A typed settings model centralizes every variable in one place, validates all of them at startup, and turns a wrong value into a clear ValidationError instead of a runtime surprise.

Prerequisites

  • pydantic-settings installed (it is a separate package from Pydantic v2).
  • Python 3.11+ for modern typing.
  • A convention for where variables come from: a .env file in development, real environment variables in deployed environments.

Step-by-Step Implementation

1. Define the settings model

# app/config.py
from pydantic import Field, PostgresDsn, RedisDsn, SecretStr
from pydantic_settings import BaseSettings, SettingsConfigDict


class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=".env",
        env_prefix="APP_",       # APP_DATABASE_URL → database_url
        extra="forbid",          # Unknown APP_* variables raise at startup.
        case_sensitive=False,
    )

    environment: str = Field(pattern="^(development|staging|production|test)$")
    database_url: PostgresDsn          # Required; no default.
    redis_url: RedisDsn | None = None  # Optional with an explicit default.
    secret_key: SecretStr = Field(min_length=32)
    request_timeout_seconds: float = Field(default=5.0, gt=0)

2. Cache the accessor

from functools import lru_cache


@lru_cache
def get_settings() -> Settings:
    # Constructed once; raises immediately if a required variable is absent.
    return Settings()

3. Model nested configuration

For grouped settings — a database block, an email block — nest models and use a delimiter:

from pydantic import BaseModel


class DatabaseConfig(BaseModel):
    url: PostgresDsn
    pool_size: int = 10


class Settings(BaseSettings):
    # APP_DB__URL and APP_DB__POOL_SIZE populate the nested model.
    model_config = SettingsConfigDict(env_prefix="APP_", env_nested_delimiter="__")
    db: DatabaseConfig

4. Provide per-environment values

# CI / production set real environment variables — no file required.
export APP_ENVIRONMENT=production
export APP_DATABASE_URL=postgresql://user:pass@db:5432/app
export APP_SECRET_KEY=$(cat /run/secrets/app_secret_key)

Edge Cases and Gotchas

  • Booleans from strings. APP_ENABLE_SIGNUP=0 and =false both parse to False; Pydantic handles the common truthy/falsy spellings, but document the convention so operators are not surprised.
  • Lists and JSON. Complex types are parsed as JSON from the variable value, so a list field expects ["a","b"], not a comma-separated string, unless you add a custom parser.
  • Precedence confusion. A value in the real environment overrides the .env file; if a stale shell variable shadows your file, the file appears to be ignored.

Verification

Add a fast test that asserts a missing required variable fails:

import pytest
from pydantic import ValidationError

from app.config import Settings


def test_missing_database_url_raises(monkeypatch):
    monkeypatch.delenv("APP_DATABASE_URL", raising=False)
    with pytest.raises(ValidationError):
        Settings(_env_file=None)  # No file, no env → required field missing.

You can also print the parsed configuration at boot (with secrets masked) to confirm what the process actually loaded:

python -c "from app.config import get_settings; print(get_settings().model_dump(exclude={'secret_key'}))"