Work Experience Creed Essays Contact
Back to work APPLICABILITY-FIRST REGULATORY RESEARCH, SELF-FUNDED DEMONSTRATION

Which FAR Applies

After the Revolutionary FAR Overhaul, "what does FAR Part 13 say?" has no single answer. The codified regulation, the overhaul text an agency adopted through its own class deviation, and a pending proposed rule can all answer it, and which one applies depends on the agency and the date. Most tools blend them into one fluent reply. This system resolves which text governs first, retrieves only from that text, and abstains when applicability or evidence is insufficient. It is a self-funded demonstration, not performed under contract, using public data only, and not legal or contracting advice.

Architecture

pipeline.txt
public sources → acquisition policy → raw immutable objects (sha256)
        │
        ▼
parsers + independent source checks → normalized model
        │
        ▼
validator → release cut → dump → restore → compare → frozen release

question + agency + as-of date
        │
        ▼
serving gate → resolver → approved editions only
        │
        ▼
lexical retrieval + exact citation → structured generator
        │
        ▼
deterministic verification → support check → answer | abstain
Resolve before you retrieve

Retrieval quality is downstream of applicability. A perfect retriever over the wrong edition returns a confident, well-cited, wrong answer. So the resolver takes an agency, a date and a part, and returns one of five named states: governing, ambiguous, uncovered, underspecified or invalid. Retrieval is filtered to the approved editions inside the same SQL statement that ranks, so it cannot override the resolver. Proposed rules are visible to the reader but can never govern.

Authority, text and applicability are separate

Stream, edition and adoption are different tables with different meanings, so "what the words say", "when they said it" and "who is bound by them" never collapse into one field. Uncertainty is a product state rather than a hidden one: ambiguity is shown side by side, and gaps are declared instead of filled.

Releases are evidence, not rows

Raw source bytes are stored with a sha256 and retrieval time, and parsing re-runs from the stored bytes. A release is cut, dumped, restored into a fresh database and compared against an independently computed inventory before it is frozen. Serving activation then proves that the database the resolver reads is the frozen artifact, and the resolver checks again on exit. Release ids come from a committed registry, because a local database sequence once reissued an id that already belonged to a valid release.

Fail closed on the answer path

The Part 13 Ask slice is a server-rendered web app and JSON API. Generation sits behind a structured provider boundary, and every claim goes through deterministic citation checks and a separate support verifier. Unsupported claims are dropped, and if no core claim survives the system abstains. With no model configured it refuses rather than falling back to an extractive mode that had scored 0 of 3 correct abstentions.

The failures that changed the design

An independent adversarial audit confirmed 19 findings (16 High, 3 Medium), all accepted, and releases 0001 and 0002 were withdrawn the same day rather than patched. The central finding was round-trip blindness: the parser and the check meant to catch its omissions read the same elements, so 305 files with empty nodes matched an empty reference and passed. The response was an independent source oracle, bidirectional completeness checks, mutation catalogues and evidence-bound releases. A check that shares the producer's assumptions is correlated, not independent.

An evaluation that found a non-model defect

The Part 13 bootstrap eval has 35 questions. Human review of its labels showed the resolver treating a missing adoption row as "no adoption", so release 0006, which holds zero adoption rows, returned legacy text for GSA and DoD while their signed deviations adopt the overhaul text. That surfaced before any model-backed run existed. The fix, a proposed ADR, lets the resolver fall back to codified text only when a human-declared review window proves the agency was checked through the query date, and otherwise returns uncovered. Absence of evidence is not evidence of absence.

Stack

Python 3.11+, PostgreSQL 16 full-text search, FastAPI, Jinja2, pydantic, httpx, lxml, pytest, Anthropic Messages API for the answer model. No vector store, search service or frontend framework.

Status

Working prototype, built and verified by one person with agent-assisted implementation; the verification gates and audits were kept outside the implementing agent's own claims. Release 0006 is frozen but unpublished and not independently audited. The last audited release had zero High findings. 1,131 tests pass in normal, reversed and random order. The adoption-coverage fix is in open review, no eval question is human-verified yet, there is no model-backed benchmark, and nothing is deployed publicly. Dense retrieval is not implemented because no measured failure justifies it.

Source is private and nothing is deployed publicly.

Back to work