Python Core Technical Specification

Date: 2026-08-08 · Status: Draft 1 · Purpose: the Python binding of docs/09-core-architecture.md. First implementation built in Phase 1; it also hosts the vector generator (docs/08-test-vector-spec.md §7), which is why it comes first.

Library-fact caveat: dependency capabilities below were assessed from documentation knowledge as of early 2026 and were marked [VERIFY] where they had to be re-confirmed against the versions current at implementation time, per the project’s citation-or-flag rule. None remain. Most were resolved in the 2026-09-09 currency sweep, against the versions constraints-ci.txt pins; the last, the PyNaCl row in §2, on 2026-09-22. Each says what was measured and when, and one of them corrected a claim rather than confirming it.

1. Package identity and toolchain

2. Dependencies

Rule: the required dependency set is cryptography alone. Everything else is an extra. This is what keeps the core auditable and keeps FIPS conversations tractable (PRD CL-9: FIPS validation is a property of the build — a deployment swapping in a FIPS-validated OpenSSL underneath pyca is the intended path; document, don’t promise).

3. Module layout

Mirrors docs/09 §1 exactly:

src/fieldseal/
  __init__.py        exports: Fieldseal, FieldContext, errors, providers — and nothing from testing
  api.py             Fieldseal client class
  envelope.py        EnvelopeHeader, parse, serialize, is_ciphertext
  registry.py        SUITES table (frozen dataclasses), allow-list checks
  context.py         FieldContext (frozen dataclass), canonical_context(), aad()
  kdf.py             record_key(), index_key()
  aead/__init__.py   AeadBackend protocol
  aead/gcm.py        suite 0xFF01 backend
  aead/xchacha.py    suite 0xFF02 backend (import guarded by the extra)
  commitment.py      pending spec gap G1 — module exists with NotImplementedError + issue link
  blindindex.py      IDFs, truncate, normalizers (argon2 construction pending G2; its
                     per-column cost and truncate are pinned, spec §7.2, §7.3);
                     IndexDeclaration and validate_index_declaration — the §7.4 band and §7.6
                     cardinality gate, checked once at construction
  keyprovider.py     KeyProvider protocol, StaticKeyProvider, DerivedKeyProvider, EnvelopeKeyProvider
  cache.py           DekCache
  config.py          FieldsealConfig — not yet split out; construction-time validation currently
                     lives in Fieldseal.__init__, and IndexDeclaration in blindindex.py
  errors.py          FieldsealError hierarchy
  testing/__init__.py  encrypt_with_materials — separate subpackage, imported only by tests;
                       inert unless armed: every function raises unless the environment variable
                       FIELDSEAL_TEST_MODE=1 is set at import time (docs/08 §6 arming gate)
py.typed

4. Public API shape

from fieldseal import Fieldseal, FieldContext
from fieldseal.keyprovider import EnvelopeKeyProvider

fs = Fieldseal(
    key_provider=provider,
    allowed_suites={0xFF01},
    write_suite=0xFF01,
    read_mode="strict",
    arm_provisional_suites=False,      # spec §4.8; or FIELDSEAL_ARM_PROVISIONAL_SUITES=1 (docs/14 §4)
    cache=CachePolicy(max_age=timedelta(minutes=10), max_uses=1_000_000, capacity=10_000),
    indexes=[IndexDeclaration(...)],
)

ct: bytes  = fs.encrypt(b"...", ctx)
pt: bytes  = fs.decrypt(ct, ctx)
bx: bytes  = fs.blind_index("...", ctx)   # str or bytes; str is the preferred form
ok: bool   = fs.is_ciphertext(ct)
ct2: bytes = fs.rotate(ct, ctx)
await fs.warm([ctx, ...])          # the only coroutine on the client

# docs/09 §2 configuration reflection
fs.read_mode         # -> str
fs.write_suite       # -> int
fs.allowed_suites    # -> frozenset[int]
fs.provisional_armed # -> bool
fs.indexes           # -> Mapping[str, ValidatedIndex], keyed by index_registry_key(...)

• blind_index takes str | bytes; every other operation takes bytes. This asymmetry predates G16 and is now the normative shape (docs/09 §7.1): normalization is a text operation, encryption is not, so index derivation is the only place where the difference between a string and its encoding changes the answer. Passing str is the preferred form because it is the only one that cannot have lost information already — this core’s bytes path is safe too (str.encode("utf-8") raises on a lone surrogate rather than substituting, unlike JavaScript’s TextEncoder), but that is a property of CPython rather than of the API. What CPython raises is a UnicodeEncodeError, which is outside the §9 taxonomy; the normalizers re-raise it as InvalidArgument so that the refusal carries the same code as the TypeScript core’s and so that on_unindexable can recognise it.
• Index parameters come from the declaration, never from the call. blind_index(value, ctx) takes only the value and the context; the IDF, normalizer, truncation length and on_unindexable policy all come from the IndexDeclaration registered at construction, resolved by (table_uuid, column_uuid, ctx.purpose). This is what gives the §7.4 band and the §7.6 cardinality gate somewhere to run: both ask how many distinct values a column holds, which a per-call argument cannot answer. ctx.purpose must already name the index (ctx.for_index("email-eq")), matching the TypeScript core. The Argon2id cost is one of those parameters: IndexDeclaration(idf="argon2id", argon2=Argon2Params(time_cost=4, memory_kib=65536), …) raises it for that column, Argon2Params is exported from the package for the purpose, and absent means the spec §7.3 minima. Below either minimum, or given on an hmac-sha512 index, it is a ConfigurationError at construction. This core carried the cost as a module constant until #62, which made a raised cost inexpressible here and expressible in TypeScript — the two cores agreeing on every vector, since the vectors pin the minima, and diverging silently on the first column that raised it (docs/09 §7).
• FieldContext is a frozen dataclass with __slots__; adapters build one per column at model-definition time and pass it per call (docs/09 §12). Adapters never set suite_id — the core fills that member itself: config.write_suite on encrypt, and the parsed envelope header on decrypt (docs/09 §3.2 step 4 — a client whose write suite is 0xFF02 must still derive the correct key for a 0xFF01 envelope during mixed-suite reads and rotation).
• All five operations are strictly synchronous and perform no I/O (spec §11.1). warm is async def; a sync convenience warm_blocking() wraps it for WSGI apps (it may do network I/O — it is not in the value path).
• Errors: FieldsealError → UnknownFormatVersion, SuiteNotAllowed, KeyUnavailable, AadMismatch, TagInvalid, CommitmentInvalid, NotCiphertext, ModeViolation (spec §9 code MODE_VIOLATION, added by G6), LengthExceeded (code LENGTH_EXCEEDED, added by G10 — spec §3.5), SuiteProvisional (code SUITE_PROVISIONAL, spec §4.8), and two implementation-local codes docs/09 §9 permits outside §9: ConfigurationError (construction time) and InvalidArgument (an operand refused at the API boundary — an index purpose handed to encrypt, invalid UTF-8 handed to a text normalizer). Each carries .code: str equal to the vector-suite string (docs/09 §9). FieldsealWarning is the spec §10.3 warning for the pass-through modes.
• KeyProvider is spec §8’s interface by name: encryption_key(ctx) -> (key_material, key_id) with purpose routing, and decryption_keys(header) -> Sequence[bytes] returning every currently-valid version in preference order; the client tries each candidate’s commitment in turn (docs/09 §3.2 step 6).
• Configuration reflection (docs/09 §2, added by G18). The five properties above are read-only and report the validated form: indexes returns ValidatedIndex records with the §7.3 Argon2 minima filled in, index_id defaulted to "exact" and on_unindexable to refuse. ValidatedIndex, validate_index_declaration and index_registry_key are exported from the package for the purpose — a caller comparing its own declarations against a client’s registry needs the last two to build the same keys and resolve the same defaults. The mapping is a MappingProxyType over the client’s registry, not the dict: mappingproxy carries no mutating methods at all, which is a stronger guarantee than refusing them, and the ValidatedIndex records are frozen dataclasses, so a caller holding one cannot rewrite the truncation length of a live index. key_provider and the cache are deliberately absent from this surface (docs/09 §2’s carve-out).

5. Security-relevant implementation notes

• Zeroization honesty (docs/09 §8.3): Python bytes are immutable and interned-copyable; true erasure is not achievable in CPython. The cache stores DEKs in bytearray and overwrites on eviction — this narrows, but does not close, the memory-exposure window, and intermediate bytes copies inside pyca calls are outside our control. The module docstring and user docs state this in exactly those terms; claiming more would violate the no-overclaim rule. mlock is not provided (docs/09 §8.3 deviation, documented).
• Which docs/09 §3 erasure steps this binding does not perform (docs/09 §8.3, §3 preamble): neither of them. §3.1 step 13 and §3.2’s zeroize record_key are both no-ops here, because kdf.record_key() returns bytes and CPython offers nothing to overwrite. The bullet above is about the cache, which is a different thing and is not a substitute for saying this: the cache’s bytearray is the long-lived copy and it is erased on eviction; the per-operation record key is short-lived and is not erased at all, on either path. Changing that would mean deriving into a bytearray and passing it to every consumer that currently takes bytes — a wider change than the property buys, given that spec §5.5 already concedes a GC runtime cannot guarantee no copy survives. Recorded rather than fixed, and declared in the conformance report under pinned_decisions.key-material-ownership.
• The spec §7.3 Argon2id salt is a third, and it is the one that matters most (added 2026-09-04, from the PR #111 review). §7.3 forbids Argon2’s K and X, so keying “rests entirely on the salt” (docs/02 line 546): those 16 bytes carry the full strength of the column’s index key, and holding them buys the same offline dictionary attack on that column’s stored indexes as holding the key. The TypeScript core erases its salt; this binding cannot, and the reason is a hard constraint of the pinned backend rather than a preference — argon2-cffi accepts only immutable bytes for salt=, rejecting bytearray and memoryview with TypeError (verified against 25.1.0). Deriving into a bytearray and converting at the call would buy nothing, because the bytes copy handed to the primitive is the exposure. The one mitigation available is taken: the salt is passed inline at the single call site, so no longer-lived reference to it exists. tests/test_blindindex_salt.py pins the backend constraint rather than leaving it as a comment, so a future argon2-cffi that accepts a writable buffer fails a test and forces this decision to be revisited instead of quietly outliving its reason. This is a cross-core divergence in zeroization, not in derived values — the vectors are unaffected and both cores produce identical indexes.
• The core does not erase what the provider returns (docs/09 §8.1). Nothing in this binding could — KeyProvider.decryption_keys is typed Sequence[bytes] and the encryption-key path is bytes — so the rule costs the Python core nothing and is not a deviation. A provider returning a bytearray precisely so that it could be erased is still not erased by the core: under §8.1 that material is the provider’s, and erasing it is the provider’s to schedule.
• Buffer maximum, and which bound binds first (docs/09 §4). CPython bounds a bytes length by sys.maxsize, which is 2⁶³−1 on any 64-bit build — 9,223,372,036,854,775,807, measured 2026-09-19 on CPython 3.14.6 x86-64 and re-checkable on the CI interpreter in one line (python -c "import sys; print(sys.maxsize)"). Spec §3.5’s 2³¹−1 binds first, by a factor of 2³², so on this platform the ceiling-not-a-guarantee clause never engages: a 2³¹-byte plaintext is refused with LENGTH_EXCEEDED at the API boundary, not by an allocator. On a 32-bit build the platform would bind instead — sys.maxsize is 2³¹−1 there, so an operand of 2³¹ bytes is unconstructible and the refusal would come from the allocator rather than from the bound. That is the case spec §3.5’s ceiling-not-a-guarantee clause covers, and no claim is made for it: every CI job runs on a 64-bit Linux runner (ubuntu-24.04), and 32-bit CPython is untested here. This core’s two spec/3.5/length-bound out-of-band report entries (docs/14 §4, run_vectors.py) exercise both sides on lazily-allocated operands and assert the exact code, so the bound is verified directly rather than recorded unverifiable; a runtime that cannot allocate the operand records not-run instead.
• Constant-time compares: all commitment/tag-adjacent comparisons via cryptography.hazmat.primitives.constant_time.bytes_eq; never == on secret-derived values.
• GIL and threading: the client is thread-safe; the cache uses a single threading.Lock around metadata with the singleflight pattern for refresh (docs/09 §8.3). No asyncio primitives in the sync path.
• Fork-safety: secrets is kernel-backed. Docs carry the prefork-server guidance from docs/09 §10 (construct the client after fork in gunicorn post_fork).
• Argon2id blocking cost: 10–100 ms per term (spec §7.3), and the GIL is released for it — measured 2026-09-09 on argon2-cffi 25.1.0 / argon2-cffi-bindings 26.1.0, CPython 3.14.6: one hash_secret_raw at the §7.3 parameters takes 36.9 ms, two on separate threads take 40.0 ms rather than the ~74 ms serialization would cost. This corrects the claim that stood here, which was the opposite — the cost is wall-clock latency on the requesting thread, not a process-wide stall, so a threaded deployment serves other requests through it. It remains a product constraint per query term; it is not a concurrency ceiling.

6. Testing plan

1. Vector harness (tests/vectors/): implements the full contract of docs/08 §5 — manifest hash check, schema validation (jsonschema dev-dependency), both-direction envelope runs, exact error-code mapping, machine-readable report emission (docs/14 §4). Vector path resolved from the repo root so core/python never copies vectors.
2. Unit tests per module, including: codec truncation at every byte offset of a valid envelope (must never panic — always a typed error); allow-list vs registry decoupling (spec §3.4 double-encryption regression case, verification-log defect #6); cache max-age/max-uses/zeroize-on-evict, with use counting per encryption_key return — decryption_keys candidate reads must not deplete max_uses (docs/09 §8.3; mirror the TypeScript providers.test.ts “decrypt-path candidate reads do not deplete §5.5 max-uses” case — this test is written down here before cache.py/EnvelopeKeyProvider exist so the bug fixed in PR #55 is not re-introduced when they land); provider purpose-routing (index purpose must never return the DEK — spec §8).
3. Property tests (hypothesis): parse(serialize(h)) identity; is_ciphertext total on arbitrary bytes (never raises); decrypt(encrypt(p, ctx), ctx) == p for random valid inputs with the real CSPRNG path.
4. Cross-output producer: a pytest-invocable script emitting the cross/ file (docs/08 §4.7) from the production path.
5. Negative import test: import fieldseal must not import fieldseal.testing; enforced by a test asserting "fieldseal.testing" not in sys.modules after a clean import. A second test asserts encrypt_with_materials raises when FIELDSEAL_TEST_MODE is unset (the docs/08 §6 arming gate). The module docstring carries the consequence verbatim: “an implementation that accepts a caller-supplied nonce or seed outside of vector-test mode is non-conformant” (vectors/README.md).

7. Non-goals for the Python core

No Django/SQLAlchemy awareness of any kind (that’s adapters/); no CLI (the vector generator lives in tools/vector-gen/); no PyPy claims until CI covers it.

Divergence recorded 2026-08-22 — the generator does not import fieldseal.testing. This section originally said it would, “deliberately”. Building both showed that to be a mistake: if the generator produces expected values using this core’s code, then this core passing those values is close to tautological, and M1 would certify nothing. tools/vector-gen/ is therefore standalone, and takes a different route to the same primitives — it hand-rolls HKDF from hmac where this core uses pyca/cryptography’s. The two agreeing on the suite is a real cross-check of HKDF — and of HKDF only: the generator and this core share the canonical_context layout, the envelope assembly and the commitment label by construction, written by the same hands from the same sections, so agreement there is agreement of one reading with itself. The same code agreeing with itself would have checked nothing at all; this checks one primitive. What checks the rest is M2 (docs/18).

The cost is honest: HKDF, canonical_context and the envelope layout each exist twice in this repository, and a spec change touches both. That is the price of M1 meaning anything before M2 lands, and M2 — an independently written TypeScript core reproducing these values from the specification alone (docs/11 §6) — remains the real check. Revisit only if the duplication starts drifting rather than being caught.

No async value-path variants either — but as of G9 (issue #9) that is this core’s choice rather than a prohibition: spec §11.1 permits optional async companions, and the Python core declines them because its target frameworks (Django, SQLAlchemy) are the ones that cannot await in the value path at all, so the companions would carry the dual-path vector obligation for no adapter that could use it. Revisit only if a Python adapter credibly claims L4.

View source: docs/10-core-python.md