Python Core Technical Specification

Date: 2026-08-08 · Status: Draft 1 · Purpose: the Python binding of docs/09-core-architecture.md. First implementation built in Phase 1; it also hosts the vector generator (docs/08-test-vector-spec.md §7), which is why it comes first.

Library-fact caveat: dependency capabilities below were assessed from documentation knowledge as of early 2026 and were marked [VERIFY] where they had to be re-confirmed against the versions current at implementation time, per the project’s citation-or-flag rule. None remain. Most were resolved in the 2026-09-09 currency sweep, against the versions constraints-ci.txt pins; the last, the PyNaCl row in §2, on 2026-09-22. Each says what was measured and when, and one of them corrected a claim rather than confirming it.

1. Package identity and toolchain

2. Dependencies

Rule: the required dependency set is cryptography alone. Everything else is an extra. This is what keeps the core auditable and keeps FIPS conversations tractable (PRD CL-9: FIPS validation is a property of the build — a deployment swapping in a FIPS-validated OpenSSL underneath pyca is the intended path; document, don’t promise).

3. Module layout

Follows docs/09 §1’s module list, with deviations from its dependency rule, recorded rather than fixed (found in #256’s review, 2026-10-05). In Python that rule is enforced by review alone, since the language has no package visibility to enforce it with. The rule lets envelope/context/kdf/aead/commitment/blindindex import registry and errors only, and keyprovider/cache import errors only. This core has six edges outside it, written as four clauses: blindindex → kdf (for hkdf_sha512; the Java core takes the same dependency through an injected seam instead, per its S4a entry in docs/07 §7), blindindex → unicode, kdf → context, and keyprovider → cache, context and envelope. unicode/ (the vendored tables, docs/09 §7.1) is not in docs/09 §1’s module list at all. The block below is also out of date: commitment.py and aead/xchacha.py do not exist (the commitment is in kdf.py; 0xFF02 has no backend), and unicode/ is missing. #256 added no edge: the operand predicate is_byte_string lives in blindindex, which api already imports.

src/fieldseal/
  __init__.py        exports: Fieldseal, FieldContext, errors, providers — and nothing from testing
  api.py             Fieldseal client class
  envelope.py        EnvelopeHeader, parse, serialize, is_ciphertext
  registry.py        SUITES table (frozen dataclasses), allow-list checks
  context.py         FieldContext (frozen dataclass), canonical_context(), aad()
  kdf.py             record_key(), index_key()
  aead/__init__.py   AeadBackend protocol
  aead/gcm.py        suite 0xFF01 backend
  aead/xchacha.py    suite 0xFF02 backend (import guarded by the extra)
  commitment.py      pending spec gap G1 — module exists with NotImplementedError + issue link
  blindindex.py      IDFs, truncate, normalizers (argon2 construction pending G2; its
                     per-column cost and truncate are pinned, spec §7.2, §7.3);
                     IndexDeclaration and validate_index_declaration — the §7.4 band and §7.6
                     cardinality gate, checked once at construction
  keyprovider.py     KeyProvider protocol, StaticKeyProvider, DerivedKeyProvider, EnvelopeKeyProvider
  cache.py           DekCache
  config.py          FieldsealConfig — not yet split out; construction-time validation currently
                     lives in Fieldseal.__init__, and IndexDeclaration in blindindex.py
  errors.py          FieldsealError hierarchy
  testing/__init__.py  encrypt_with_materials — separate subpackage, imported only by tests;
                       inert unless armed: every function raises unless the environment variable
                       FIELDSEAL_TEST_MODE=1 is set at import time (docs/08 §6 arming gate)
py.typed

4. Public API shape

from datetime import timedelta

from fieldseal import Fieldseal, FieldContext, IndexDeclaration
from fieldseal.cache import CachePolicy
from fieldseal.keyprovider import EnvelopeKeyProvider

# The §5.5 cache policy is the provider's, not the client's (docs/09 §2, §8.3):
# `Fieldseal.__init__` has no `cache` parameter, and passing one is a TypeError.
provider = EnvelopeKeyProvider(
    wrapper=kms_wrapper,            # any object with `unwrap(wrapped, scope)`
    directory=key_directory,        # anything with `by_scope(scope)`
    cache=CachePolicy(max_age=timedelta(minutes=10), max_uses=1_000_000, capacity=10_000),
)

fs = Fieldseal(
    key_provider=provider,
    allowed_suites={0xFF01},
    write_suite=0xFF01,
    read_mode="strict",
    arm_provisional_suites=False,      # spec §4.8; or FIELDSEAL_ARM_PROVISIONAL_SUITES=1 (docs/14 §4)
    indexes=[IndexDeclaration(...)],
)

ct: bytes  = fs.encrypt(b"...", ctx)
pt: bytes  = fs.decrypt(ct, ctx)
bx: bytes  = fs.blind_index("...", ctx)   # str or bytes; str is the preferred form
ok: bool   = fs.is_ciphertext(ct)
ct2: bytes = fs.rotate(ct, ctx)
await fs.warm([ctx, ...])          # the only coroutine on the client

# docs/09 §2 configuration reflection
fs.read_mode         # -> str
fs.write_suite       # -> int
fs.allowed_suites    # -> frozenset[int]
fs.provisional_armed # -> bool
fs.indexes           # -> Mapping[str, ValidatedIndex], keyed by index_registry_key(...)

• blind_index takes str | bytes; every other operation takes bytes. This asymmetry predates G16 and is now the normative shape (docs/09 §7.1): normalization is a text operation, encryption is not, so index derivation is the only place where the difference between a string and its encoding changes the answer. Passing str is the preferred form because it is the only one that cannot have lost information already — this core’s bytes path is safe too (str.encode("utf-8") raises on a lone surrogate rather than substituting, unlike JavaScript’s TextEncoder), but that is a property of CPython rather than of the API. What CPython raises is a UnicodeEncodeError, which is outside the §9 taxonomy; the normalizers re-raise it as InvalidArgument so that the refusal carries the same code as the TypeScript core’s and so that on_unindexable can recognise it. An operand of any other type is refused, never coerced (#254). Bytes means bytes, bytearray, or a memoryview that is one-dimensional, contiguous and of format 'B' (blindindex.is_byte_string), which is what memoryview(b) gives over either; encrypt, decrypt and rotate take bytes, and blind_index and normalize take bytes or str. Anything else is InvalidArgument, in every read mode. That includes array.array, mmap and objects with __bytes__, which encrypt and blind_index used to accept: wrap the first two in memoryview. It also includes a view of wider items, a strided view and a multi-dimensional one, which are bytes-typed and unusable: the AEAD refused them with a raw error after the provider had been consulted, len() undercounts them, and a normalizer would have indexed their machine-endian bytes. The rule is the format, not each operation’s tolerance, so it also refuses views that worked on dev where an operation could read them: a view over a ctypes byte array (format '<B'), a cast('c') view, and a strided view in decrypt’s pass-through and blind_index. .cast('B') where the view allows it, or bytes(...), is the way through. A released view is refused with its own message. is_ciphertext still answers True for an envelope in some of these views; that is #257. On encrypt and rotate the refusal comes after MODE_VIOLATION and SUITE_PROVISIONAL, so readonly’s answer does not depend on the operand; on blind_index it comes after the declaration is resolved and is never filed in the bucket, because an int is a caller’s error, not a value the pin cannot define. is_ciphertext is the exception: it is total (spec §3.4) and answers False. Two readings, declared in pinned_decisions.api-boundary-order: spec §11.1’s rotate sentence ("rotate of non-envelope input raises NOT_CIPHERTEXT") is read as bounded by its own bytes signature, so rotate(42) is InvalidArgument, not NotCiphertext; and blind_index’s str rests on docs/09 §7.1 (G16 part A), since §11.1’s signature block types it bytes. Before #254 one operand got four answers: decrypt refused it (#251), rotate raised NotCiphertext, encrypt leaked a raw TypeError, and blind_index(42) returned the index of bytes(42), 42 NUL bytes, with no error.
• Index parameters come from the declaration, never from the call. blind_index(value, ctx) takes only the value and the context; the IDF, normalizer, truncation length and on_unindexable policy all come from the IndexDeclaration registered at construction, resolved by (table_uuid, column_uuid, ctx.purpose). This is what gives the §7.4 band and the §7.6 cardinality gate somewhere to run: both ask how many distinct values a column holds, which a per-call argument cannot answer. ctx.purpose must already name the index (ctx.for_index("email-eq")), matching the TypeScript core. The Argon2id cost is one of those parameters: IndexDeclaration(idf="argon2id", argon2=Argon2Params(time_cost=4, memory_kib=65536), …) raises it for that column, Argon2Params is exported from the package for the purpose, and absent means the spec §7.3 minima. Below either minimum, or given on an hmac-sha512 index, it is a ConfigurationError at construction. This core carried the cost as a module constant until #62, which made a raised cost inexpressible here and expressible in TypeScript — the two cores agreeing on every vector, since the vectors pin the minima, and diverging silently on the first column that raised it (docs/09 §7).
• FieldContext is a frozen dataclass with __slots__; adapters build one per column at model-definition time and pass it per call (docs/09 §12). Adapters never set suite_id — the core fills that member itself: config.write_suite on encrypt, and the parsed envelope header on decrypt (docs/09 §3.2 step 4 — a client whose write suite is 0xFF02 must still derive the correct key for a 0xFF01 envelope during mixed-suite reads and rotation).
• All five operations are strictly synchronous and perform no I/O (spec §11.1). warm is async def; a sync convenience warm_blocking() wraps it for WSGI apps (it may do network I/O — it is not in the value path).
• Errors: FieldsealError → UnknownFormatVersion, SuiteNotAllowed, KeyUnavailable, AadMismatch, TagInvalid, CommitmentInvalid, NotCiphertext, ModeViolation (spec §9 code MODE_VIOLATION, added by G6), LengthExceeded (code LENGTH_EXCEEDED, added by G10 — spec §3.5), SuiteProvisional (code SUITE_PROVISIONAL, spec §4.8), and two implementation-local codes docs/09 §9 permits outside §9: ConfigurationError (construction time) and InvalidArgument (an operand refused at the API boundary — an index purpose handed to encrypt, invalid UTF-8 handed to a text normalizer, an operand of the wrong type handed to any operation but is_ciphertext, in every read mode; until #251 and #254 such an operand was coerced with bytes(), which turns an int into that many zero bytes, or leaked a raw TypeError). Each carries .code: str equal to the vector-suite string (docs/09 §9). FieldsealWarning is the spec §10.3 warning for the pass-through modes.
• KeyProvider is spec §8’s interface by name: encryption_key(ctx) -> (key_material, key_id) with purpose routing, and decryption_keys(header) -> Sequence[bytes] returning every currently-valid version in preference order; the client tries each candidate’s commitment in turn (docs/09 §3.2 step 6).
• No client-level cache parameter (docs/09 §2, §8.3). Fieldseal.__init__ takes key_provider, allowed_suites, write_suite, read_mode, indexes and arm_provisional_suites (§2’s on_warning and metrics hooks are not implemented in this binding; see #223); handing it cache= is a TypeError, not a silently ignored argument. The §5.5 policy is EnvelopeKeyProvider’s own required cache option (from fieldseal.cache import CachePolicy), validated in CachePolicy.__post_init__ and raised at that construction. Python has no client-config dictionary to screen, so it has nothing to refuse: the keyword simply does not exist. Every client built on one provider shares that provider’s cache, its use budgets and its capacity, and one client’s warms can evict another’s keys (docs/09 §8.3) — no test in this core pins that yet.
• Configuration reflection (docs/09 §2, added by G18). The five properties above are read-only and report the validated form: indexes returns ValidatedIndex records with the §7.3 Argon2 minima filled in, index_id defaulted to "exact" and on_unindexable to refuse. ValidatedIndex, validate_index_declaration and index_registry_key are exported from the package for the purpose — a caller comparing its own declarations against a client’s registry needs the last two to build the same keys and resolve the same defaults. The mapping is a MappingProxyType over the client’s registry, not the dict: mappingproxy carries no mutating methods at all, which is a stronger guarantee than refusing them, and the ValidatedIndex records are frozen dataclasses, so a caller holding one cannot rewrite the truncation length of a live index. key_provider and the cache are deliberately absent from this surface (docs/09 §2’s carve-out).

5. Security-relevant implementation notes

• Zeroization honesty (docs/09 §8.3): Python bytes are immutable and interned-copyable; true erasure is not achievable in CPython. The cache stores DEKs in bytearray and overwrites on eviction — this narrows, but does not close, the memory-exposure window, and intermediate bytes copies inside pyca calls are outside our control. The module docstring and user docs state this in exactly those terms; claiming more would violate the no-overclaim rule. mlock is not provided (docs/09 §8.3 deviation, documented).
• Which docs/09 §3 erasure steps this binding does not perform (docs/09 §8.3, §3 preamble): neither of them. §3.1 step 13 and §3.2’s zeroize record_key are both no-ops here, because kdf.record_key() returns bytes and CPython offers nothing to overwrite. The bullet above is about the cache, which is a different thing and is not a substitute for saying this: the cache’s bytearray is the long-lived copy and it is erased on eviction; the per-operation record key is short-lived and is not erased at all, on either path. Changing that would mean deriving into a bytearray and passing it to every consumer that currently takes bytes — a wider change than the property buys, given that spec §5.5 already concedes a GC runtime cannot guarantee no copy survives. Recorded rather than fixed, and declared in the conformance report under pinned_decisions.key-material-ownership.
• The spec §7.3 Argon2id salt is a third, and it is the one that matters most (added 2026-09-04, from the PR #111 review). §7.3 forbids Argon2’s K and X, so keying “rests entirely on the salt” (docs/02 line 546): those 16 bytes carry the full strength of the column’s index key, and holding them buys the same offline dictionary attack on that column’s stored indexes as holding the key. The TypeScript core erases its salt; this binding cannot, and the reason is a hard constraint of the pinned backend rather than a preference — argon2-cffi accepts only immutable bytes for salt=, rejecting bytearray and memoryview with TypeError (verified against 25.1.0). Deriving into a bytearray and converting at the call would buy nothing, because the bytes copy handed to the primitive is the exposure. The one mitigation available is taken: the salt is passed inline at the single call site, so no longer-lived reference to it exists. tests/test_blindindex_salt.py pins the backend constraint rather than leaving it as a comment, so a future argon2-cffi that accepts a writable buffer fails a test and forces this decision to be revisited instead of quietly outliving its reason. This is a cross-core divergence in zeroization, not in derived values — the vectors are unaffected and both cores produce identical indexes.
• The core does not erase what the provider returns (docs/09 §8.1). Nothing in this binding could — KeyProvider.decryption_keys is typed Sequence[bytes] and the encryption-key path is bytes — so the rule costs the Python core nothing and is not a deviation. A provider returning a bytearray precisely so that it could be erased is still not erased by the core: under §8.1 that material is the provider’s, and erasing it is the provider’s to schedule.
• Buffer maximum, and which bound binds first (docs/09 §4). CPython bounds a bytes length by sys.maxsize, which is 2⁶³−1 on any 64-bit build — 9,223,372,036,854,775,807, measured 2026-09-19 on CPython 3.14.6 x86-64 and re-checkable on the CI interpreter in one line (python -c "import sys; print(sys.maxsize)"). Spec §3.5’s 2³¹−1 binds first, by a factor of 2³², so on this platform the ceiling-not-a-guarantee clause never engages: a 2³¹-byte plaintext is refused with LENGTH_EXCEEDED at the API boundary, not by an allocator. On a 32-bit build the platform would bind instead — sys.maxsize is 2³¹−1 there, so an operand of 2³¹ bytes is unconstructible and the refusal would come from the allocator rather than from the bound. That is the case spec §3.5’s ceiling-not-a-guarantee clause covers, and no claim is made for it: every CI job runs on a 64-bit Linux runner (ubuntu-24.04), and 32-bit CPython is untested here. This core’s two spec/3.5/length-bound out-of-band report entries (docs/14 §4, run_vectors.py) exercise both sides on lazily-allocated operands and assert the exact code, so the bound is verified directly rather than recorded unverifiable; a runtime that cannot allocate the operand records not-run instead.
• Constant-time compares: all commitment/tag-adjacent comparisons via cryptography.hazmat.primitives.constant_time.bytes_eq; never == on secret-derived values.
• GIL and threading: the client is thread-safe; the cache uses a single threading.Lock around metadata with the singleflight pattern for refresh (docs/09 §8.3). No asyncio primitives in the sync path.
• Fork-safety: secrets is kernel-backed. Docs carry the prefork-server guidance from docs/09 §10 (construct the client after fork in gunicorn post_fork).
• Argon2id blocking cost: 10–100 ms per term (spec §7.3), and the GIL is released for it — measured 2026-09-09 on argon2-cffi 25.1.0 / argon2-cffi-bindings 26.1.0, CPython 3.14.6: one hash_secret_raw at the §7.3 parameters takes 36.9 ms, two on separate threads take 40.0 ms rather than the ~74 ms serialization would cost. This corrects the claim that stood here, which was the opposite — the cost is wall-clock latency on the requesting thread, not a process-wide stall, so a threaded deployment serves other requests through it. It remains a product constraint per query term; it is not a concurrency ceiling.

6. Testing plan

1. Vector harness (tests/vectors/): implements the full contract of docs/08 §5 — manifest hash check, schema validation (jsonschema dev-dependency), both-direction envelope runs, exact error-code mapping, machine-readable report emission (docs/14 §4). Vector path resolved from the repo root so core/python never copies vectors.
2. Unit tests per module, including: codec truncation at every byte offset of a valid envelope (must never panic — always a typed error); allow-list vs registry decoupling (spec §3.4 double-encryption regression case, verification-log defect #6); cache max-age/max-uses/zeroize-on-evict, with use counting per DEK-role encryption_key return and none at all for the index role — decryption_keys candidate reads must not deplete max_uses (docs/09 §8.3; mirror the TypeScript providers.test.ts “decrypt-path candidate reads do not deplete §5.5 max-uses” case — this test is written down here before cache.py/EnvelopeKeyProvider exist so the bug fixed in PR #55 is not re-introduced when they land); and the index role has no use budget — docs/09 §8.3 gives max_uses to DEK-role returns only, because an index key is never AEAD key material and draws no nonce, so the SP 800-38D ceiling the threshold exists for cannot reach it, while max_uses is configurable as low as 1 and index derivation runs on every write and every query; max_age and the capacity LRU still apply to both roles, so the cases are an index key fetched past max_uses and still served, an index key still evicted on max_age, index traffic not spending a DEK’s budget, and a DEK still use-evicted at max_uses — tests/test_providers.py’s test_index_key_returns_do_not_deplete_max_uses and test_index_key_is_still_evicted_on_max_age (both red before the fix), plus test_index_traffic_does_not_spend_the_dek_budget, which is green before it by design — the role has always been part of the cache key, so that case guards the two entries staying separate and bites a cache key that dropped the role; the three are mirrored one-for-one in core/typescript/tests/providers.test.ts; provider purpose-routing (index purpose must never return the DEK — spec §8).
3. Property tests (hypothesis): parse(serialize(h)) identity; is_ciphertext total on arbitrary bytes (never raises); decrypt(encrypt(p, ctx), ctx) == p for random valid inputs with the real CSPRNG path.
4. Cross-output producer: a pytest-invocable script emitting the cross/ file (docs/08 §4.7) from the production path.
5. Negative import test: import fieldseal must not import fieldseal.testing; enforced by a test asserting "fieldseal.testing" not in sys.modules after a clean import. A second test asserts encrypt_with_materials raises when FIELDSEAL_TEST_MODE is unset (the docs/08 §6 arming gate). The module docstring carries the consequence verbatim: “an implementation that accepts a caller-supplied nonce or seed outside of vector-test mode is non-conformant” (vectors/README.md).

7. Non-goals for the Python core

No Django/SQLAlchemy awareness of any kind (that’s adapters/); no CLI (the vector generator lives in tools/vector-gen/); no PyPy claims until CI covers it.

Divergence recorded 2026-08-22 — the generator does not import fieldseal.testing. This section originally said it would, “deliberately”. Building both showed that to be a mistake: if the generator produces expected values using this core’s code, then this core passing those values is close to tautological, and M1 would certify nothing. tools/vector-gen/ is therefore standalone, and takes a different route to the same primitives — it hand-rolls HKDF from hmac where this core uses pyca/cryptography’s. The two agreeing on the suite is a real cross-check of HKDF — and of HKDF only: the generator and this core share the canonical_context layout, the envelope assembly and the commitment label by construction, written by the same hands from the same sections, so agreement there is agreement of one reading with itself. The same code agreeing with itself would have checked nothing at all; this checks one primitive. What checks the rest is M2 (docs/18).

The cost is honest: HKDF, canonical_context and the envelope layout each exist twice in this repository, and a spec change touches both. That is the price of M1 meaning anything before M2 lands, and M2 — an independently written TypeScript core reproducing these values from the specification alone (docs/11 §6) — remains the real check. Revisit only if the duplication starts drifting rather than being caught.

No async value-path variants either — but as of G9 (issue #9) that is this core’s choice rather than a prohibition: spec §11.1 permits optional async companions, and the Python core declines them because its target frameworks (Django, SQLAlchemy) are the ones that cannot await in the value path at all, so the companions would carry the dual-path vector obligation for no adapter that could use it. Revisit only if a Python adapter credibly claims L4.

View source: docs/10-core-python.md