01. Fourteen tests passed in one second

On Sunday, September 13, my agent worked on the model client in my own tooling. A model can stop because it hit the output token limit. The API then reports finishReason: MAX_TOKENS and returns half a reply. My client read that half reply as a normal answer. A later step applied it as if it were complete.

The agent's commit did three things. It removed the 8,192-token default output limit. It added a raise in the reply parser for a MAX_TOKENS finish. And it added a branch to the error classifier. That branch marks a token-limit message as fatal, so my client does not retry it. The commit also added two test files. Later I replayed the commit on the tree it was made on: 14 tests passed in 1.01 seconds.

02. What the tests ran

One new test checked the new default limit. The other checked the classifier with a token-limit string. No test sent a cut-off reply to the parser, so the raise never ran. The commit changed 25 lines. Its own tests ran 4 of them. I also ran the whole suite on that tree, and the raise still never ran.

The commit had a second problem. The agent pasted the new classifier branch twice. The first copy returns early, so the second copy is dead code that no test can reach. It stayed in the file for 10 days, until a cleanup commit deleted it. The agent did add a test for the raise, but in a later commit, 52 minutes after the first one.

My Part 5 post, The Behavior Gate, re-runs the pinned tests at commit time. My Part 6 post, The Repro Fence, checks two things: a bug test fails before the fix, and public signatures keep their shape. Both gates passed this commit. The pinned tests still passed, and the new tests were not bug reproducers. Neither gate asks which new lines the tests run.

03. Step 1: list the new code lines

coverage_delta.py check reads the staged diff with git diff --cached -U0. Each hunk header gives the first added line and the count. So the gate gets the exact line numbers that the commit adds to each .py file:

def added_lines(repo: Path) -> dict[str, set[int]]:
    """Staged .py files mapped to the line numbers the commit adds to them."""
    diff = git(repo, "diff", "--cached", "-U0", "--no-color", "--diff-filter=AMR", "--", "*.py")
    out: dict[str, set[int]] = {}
    current = ""
    for line in diff.splitlines():
        if line.startswith("+++ "):
            current = line[6:] if line.startswith("+++ b/") else ""
            continue
        match = HUNK_RE.match(line)
        if match and current:
            start, count = int(match.group(1)), int(match.group(2) or "1")
            out.setdefault(current, set()).update(range(start, start + count))
    return out

Comments, blank lines, and docstring continuation lines do not run, so they cannot count. The gate compiles the staged source. It keeps only the lines that produce bytecode. A new comment adds zero lines to check.

def executable_lines(source: str, rel: str) -> set[int]:
    """Line numbers that compile to at least one bytecode instruction."""
    lines: set[int] = set()
    stack = [compile(source, rel, "exec")]
    while stack:
        code = stack.pop()
        lines.update(line for _, _, line in code.co_lines() if line is not None)
        stack.extend(c for c in code.co_consts if hasattr(c, "co_lines"))
    return lines

04. Step 2: run the tests and record each line

Python 3.12 added sys.monitoring. It is a standard library hook that calls a function of mine on events such as "this line is about to run". The gate starts a fresh interpreter and registers a LINE callback there. Then it runs pytest on the staged test files in that interpreter:

mon = sys.monitoring
TOOL = 4
mon.use_tool_id(TOOL, "coverage-delta")
hits = {}

def on_line(code, line):
    path = os.path.realpath(code.co_filename)
    if path in targets:
        hits.setdefault(path, set()).add(line)
    return mon.DISABLE

mon.register_callback(TOOL, mon.events.LINE, on_line)
mon.set_events(TOOL, mon.events.LINE)
import pytest
rc = pytest.main(["-q", "-p", "no:cacheprovider", *sys.argv[3:]])

The callback returns DISABLE. That turns the event off for that line after its first run. So each line costs one callback, not one per loop pass. The gate then compares the two sets: the new code lines and the lines that ran. A new code line that never ran prints as [COV-delta] file:line never ran with its source text, and the gate rejects the commit.

Three more rules keep the check honest. First, if the tests fail, the gate rejects the commit. Coverage from a failing run proves nothing. Second, if a staged file also has unstaged edits, the gate rejects it. The tests would run code that is not in the commit. Third, a line may opt out with # cov-delta: skip <reason>. A skip with no reason still fails.

05. The same commit, replayed

I rebuilt the commit in a two-file scratch repo with the same shape. It removes the default limit, adds a raise for a MAX_TOKENS finish, pastes the classifier branch twice, and adds tests for the default and the classifier. The tests pass, but the commit does not:

$ python3 -m pytest -q test_model_client.py
6 passed in 0.01s
$ git commit -m "fix: treat cut-off replies as errors"
[COV-delta] model_client.py:19 never ran: return "fatal"
[COV-delta] model_client.py:29 never ran: raise RuntimeError("[MAX_TOKENS] reply was cut off at the output limit")
coverage delta: 6 of 8 new code lines ran under test_model_client.py
exit=1

Line 19 is the second copy of the classifier branch. Line 29 is the raise. Each one needs a different fix. I delete the duplicate. The raise gets a test that sends a cut-off reply and expects the error:

def test_cut_off_reply_raises():
    with pytest.raises(RuntimeError, match="MAX_TOKENS"):
        parse_reply(reply("half a sent", finish="MAX_TOKENS"))

With both changes staged, the gate prints coverage delta: 6 of 6 new code lines ran under test_model_client.py, and the commit lands. The gate itself is 158 lines of standard library Python, with 6 tests of its own.

06. What each gate can see

Each row is one way a fix can look finished. Each cell says whether that gate rejects it.

No column catches every row. That is why the three gates run in sequence. The coverage delta gate catches the first two rows: the untested fix line and the pasted branch. It misses the third. A regression test written after the fix runs the raise, so the coverage check passes. The repro fence rejects that commit instead.

07. What this gate does not check

A test can run a line without checking it. A test can call the parser with a cut-off reply, catch every exception, and assert nothing. The gate sees the raise run, and it passes. Mutation testing asks the stronger question: does a test fail when the line changes? It costs minutes per file. This gate costs seconds per commit.

It counts lines, not branches. A one-line x = a if cond else b counts as run when either side runs. The other side may never run.

It sees Python only, and the staged tests decide what counts. When a commit stages test files, only those tests run. So a new line that an older test covers still fails. I pass --test to add that older test. A commit that stages no test file runs the whole suite. Above 30 staged .py files, the gate prints a notice and skips. So a large commit goes through unchecked.

Next I plan to build a small mutation pass for the lines this gate lists. It will change each new line and check that at least one test fails.

08. Three gates, one question each

Part 5 asks whether the old tests still pass. Part 6 asks whether the bug test failed before the fix and whether the public shape held. Part 7 asks whether the tests ran the new code. The agent on September 13 answered yes to the first two and never faced the third.

It reported "14 passed", and it was right. It did not report which lines those 14 tests ran, because nothing asked. Now the commit hook asks, and it prints the line numbers.

Primary research and documentation

• sys.monitoring, Python documentation: the standard library event API the gate uses, with LINE events and the DISABLE return value.
• PEP 669: Low Impact Monitoring for CPython: the proposal that added sys.monitoring in Python 3.12.

01. The Friday commit that did not land

On a Friday afternoon, I asked the agent to stamp a theme tag into every PNG that my social-card renderer writes. Its first patch added a theme keyword to render_html_to_png and to build_carousel. Those two functions have five callers outside the diff. Every test in the diff passed. Then the pre-commit hook printed one line per function and exited 1, which means it refused the commit. In a two-file scratch repo, the line reads:

[R2-fence] render.py: 'render_html_to_png' signature changed (html,output_path)d0 -> (html,output_path,theme=)d1
exit=1

The line gives three things: the function's name, its shape at HEAD, and its shape now. The agent read the line and kept the signature. It moved the theme into module state instead, and that passed. The patch I kept, below, uses a delegate.

02. What the behavior gate does not see

The test that was green from the start. The agent fixes the bug first. Then it writes the regression test, and the test is green on its first run. Nothing shows that it failed before the fix. The behavior gate keeps a hash of the old tests, its pinned baseline, and a new test is not in it.

The fix by reshaping. To satisfy one call site, the agent adds a keyword, renames a method, or inlines a public helper. The tests in the diff pass. The callers outside the diff break when they call the function.

03. R1: the test must be red first

R1 says the test must be red first: it must fail before the fix. It checks the reproducer, the test that shows the bug. I run it like this: repro_fence.py red --cmd "python3 -m pytest tests/test_x.py::test_bug -q". The script runs the reproducer on the current tree, with shell=False and a hard timeout. It passes only when two things are true: pytest exits 1, and its JUnit report shows that each named test failed. Every other exit code is rejected with a reason. Among them: 0 means the test passed, 2 means a collection error, 4 means a test id that does not exist, 124 means a timeout, and 127 means a missing binary:

REJECTED_EXIT_REASONS: dict[int, str] = {
    0: "exited 0 on the current tree; a passing reproducer proves nothing",
    2: "exited 2 (interrupted or collection error); the test never ran",
    3: "exited 3 (pytest internal error); the test never ran",
    4: "exited 4 (usage error or unknown test id); the test never ran",
    5: "exited 5 (no tests collected); the test never ran",
    124: "timed out; a hang is not a reproduction",
    127: "binary not found or not executable",
}

def check_reproducer(repo: Path, argv: list[str], timeout: int) -> list[Violation]:
    """R1: pytest exits 1 on the current tree and every named test failed."""
    with tempfile.TemporaryDirectory() as tmp:
        report_xml = Path(tmp, "r1.xml")
        extra = [f"--junitxml={report_xml}", f"--rootdir={repo}", "-p", "no:cacheprovider"]
        code = run_argv([*argv, *extra], repo, timeout)
        cases = junit_cases(report_xml)
    if code != 1:
        reason = REJECTED_EXIT_REASONS.get(code, f"exited {code}; only exit 1 (tests failed) is red")
        if code == 124:
            reason = f"timed out after {timeout}s; a hang is not a reproduction"
        return [Violation("R1-repro", f"reproducer {shlex.join(argv)} {reason}")]
    return [Violation("R1-repro", f"reproducer exited 1 but {test_id} {why}")
            for test_id in _test_ids(argv) if (why := _not_failed(test_id, cases))]

I gamed the first version within a day. pytest test_x.py || true exits 0, so it is rejected. But sh -c "pytest test_x.py; exit 1" exits 1 and sails through. A list of banned tokens loses the same way, to bash -lc, a pipe into grep, or python3 -c "assert 0". So R1 has a second half, R1-shape, and it is an allowlist. The command must start with pytest or python3 -m pytest. After that, it may hold only path::test ids and the flags -q, -v and -x. A command that picks its own exit code cannot prove the bug.

04. R2: the public surface keeps its shape

R2 says the public surface keeps its shape. The command is repro_fence.py fence --rev HEAD --file a.py --file b.py. For each file, it reads two versions with git show: the one at rev and the one in the index. It parses both with Python's ast module. Then it compares the public symbols by a normalized signature, not by their source text. So a docstring edit passes, and a changed parameter list does not:

def signature_of(node: ast.FunctionDef | ast.AsyncFunctionDef) -> str:
    """Normalized shape: `[@decorator ][async ](name,name=,/,*va,name,**kw)dN`.

    `=` marks a parameter with a default and N counts them, so appending
    `extra=None` or moving a default changes the shape. So do `async` and
    decorators such as `@property`, which change how callers call it.
    """
    spec = node.args
    positional = spec.posonlyargs + spec.args
    first_default = len(positional) - len(spec.defaults)
    names = [a.arg + ("=" if i >= first_default else "") for i, a in enumerate(positional)]
    if spec.posonlyargs:
        names.insert(len(spec.posonlyargs), "/")
    if spec.vararg is not None:
        names.append("*" + spec.vararg.arg)
    elif spec.kwonlyargs:
        names.append("*")
    names.extend(a.arg + ("" if d is None else "=") for a, d in zip(spec.kwonlyargs, spec.kw_defaults))
    if spec.kwarg is not None:
        names.append("**" + spec.kwarg.arg)
    defaults = len(spec.defaults) + sum(1 for d in spec.kw_defaults if d is not None)
    prefix = "".join(f"@{ast.unparse(d)} " for d in node.decorator_list)
    if isinstance(node, ast.AsyncFunctionDef):
        prefix += "async "
    return f"{prefix}({','.join(names)})d{defaults}"

An = marks each parameter that has a default, and dN counts them. So appending extra=None changes both the name list and the count, from d0 to d1. Moving a default to another parameter is a change too. A name that is at HEAD but absent now is a removed symbol. A name in both versions with a different shape is a changed signature. Names with a leading underscore are private in Python, so the fence skips them, except dunders such as __init__. New public symbols are free. A file the parser does not know prints R2-skip.

def diff_symbols(rel: str, before: dict[str, str], after: dict[str, str]) -> list[Violation]:
    """Public symbols that vanished or changed shape between two sources."""
    out: list[Violation] = []
    for name, sig in sorted(before.items()):
        if name not in after:
            out.append(Violation("R2-fence", f"{rel}: public symbol '{name}' was removed"))
        elif after[name] != sig:
            out.append(
                Violation("R2-fence", f"{rel}: '{name}' signature changed {sig} -> {after[name]}")
            )
    return out

The fence reads the staged copy of each file. So an unstaged edit cannot make a staged one look safe. The hook also hands it deleted and renamed files. A rev that is not a commit, a git error, or a file found on neither side makes it reject. On the three-file scratch repo, the run takes 0.15 s.

05. The patch that passes the fence

The renderer needs the theme, and the signature cannot change. The agent's first answer, the one in the video below, was a module-level setter. It passes the fence. But two handlers that render at once race on that shared theme. And a handler that skips the setter inherits the last caller's theme. The patch I kept adds a new public function and has the old one delegate to it: one public function added, zero changed.

def render_html_to_png_with(html, output_path, *, theme="paper"):
    """Writes html to output_path as a PNG stamped with theme."""
    ...
    stamp_png(output_path, theme)
    return output_path

def render_html_to_png(html, output_path):
    """Writes html to output_path as a PNG."""
    return render_html_to_png_with(html, output_path)   # same signature as HEAD

A handler that needs a theme calls render_html_to_png_with(html, path, theme=theme). The old signature is byte-identical to HEAD. The theme travels with each call. The five callers outside the diff were not touched. The commit lands.

Two weeks behind the rule taught me four gotchas. First, a keyword default is still a signature change, even a keyword-only one. The delegate above is the way through. Second, a top-level def test_* is public, so renaming a test is rejected. Third, inlining a public helper counts as a removed symbol. Fourth, one honest false positive: a deliberate CLI change, audit_social_bundle()d0 -> (slug)d0. The shape that passed read sys.argv[1] inside the unchanged body. That is the cost of the rule.

Video: Agent turn in Antigravity: R2 rejects the added keyword, the setter patch lands, then R1 rejects a green reproducer and a shell-shaped one. Watch it in the essay.

06. What each rule can see

Here are six moves in one scratch repo, against three rules. Every cell is an exit code read off the terminal.

The last row is on purpose. A fence on private names would turn every refactor into an override. And an override I type on every commit stops being a gate. The underscore is the contract.

07. What these gates cannot see

R2 sees Python only. A .ts or .sh file prints R2-skip and passes.

R2 reads names and arity, not types or semantics. Arity is the number of parameters. A function that keeps its parameter list but changes what it promises to return still passes. Catching that is the behavior gate's job.

R1 proves the test fails now, not that it fails for the right reason. A test that fails because of its own typo is still red. A failed import or a missing test id is rejected. The rest is on the author.

Three papers this year measured the problem from outside. TDAD (Mar 2026) cut regressions on SWE-bench Verified from 6.08% to 1.82%. It did this with a code-to-test map at commit time. Test-first prompt instructions alone pushed regressions to 9.94%. How Coding Agents Fail Their Users (May 2026) read 20,574 real sessions. In them, 91.49% of resolutions needed a user correction. DEPBENCH (Aug 2026) set 203 upgrade tasks with hidden signature changes. The best configuration solved 104. The fence is that last problem run backwards.

08. Same rule, different reader

Part 5, the behavior gate, put the pinned tests at the commit boundary. This part adds the contract on either side of the fix: red before it, same shape after it. The two rules are 430 lines of standard library Python, with 33 tests.

The rule behind the Friday rejection had been a sentence in the system prompt for months. The agent read it at turn one. The hook read the staged files at commit time and printed the diff. Same rule, different reader, and only one returns an exit code.

Primary research and documentation

• TDAD: Test-Driven Agentic Development - Reducing Code Regressions in AI Coding Agents via Graph-Based Impact Analysis (Mar 2026): regressions on SWE-bench Verified fell from 6.08% to 1.82% with a code-to-test map at commit time; TDD instructions alone pushed them to 9.94%.
• How Coding Agents Fail Their Users (May 2026): 20,574 sessions across 1,639 repositories; 91.49% of visible resolutions needed explicit user correction.
• DEPBENCH: Update from Hell (Aug 2026): 203 dependency-upgrade tasks with hidden signature and API changes; the best agent configuration solved 104 of 203, 51.2%.
• repro_fence.py, the two rules and their tests, on GitHub Gist: the standard-library script this post quotes, with the README and the unittest file.

01. The green commit that lied

Part 4 ended with the AST hook rejecting an agent's commit. The agent had deleted a test and dropped a return type on a public function. It restored the test and added the annotation, and the second commit went through with exit 0. The restored test still failed when anything ran it. The code under it had lost its duplicate-key check, and nothing in the hook ever called the function.

I wrote the closing line of that video on purpose: the hook checks shape, not behavior. A syntax-tree gate answers questions about the text of a program. Is there a function named test_idempotent_commit? Does it contain an ast.Assert node? Does the public function carry a return annotation? Reading answers every one of those. Running answers none of them.

The agent was not adversarial. It optimized against the only signal it could see, the exit code. The exit code said the name and the annotation were enough. They were enough for the hook. They were not enough for the ledger.

02. Why shape gates are gameable

Say an agent faces a failing test and a shape gate. It has three moves. This morning I ran all three against the Part 4 harness in a scratch repo, one at a time. I recorded which check each one defeats.

Delete the test. Remove the whole function. The AST gate catches this one, and only this one. The harness compares the test names at HEAD with the names in the staged tree. It exits 1 and prints the deleted name. This is the check Part 4 showed on camera.

Hollow the assert. Keep the function and its name. Replace its two assertions with assert commit_ledger(c, seen) is not None. The name survives the name check. An ast.Assert node is still there, so the zero-assert check passes. The AST gate returned exit 0 on this file. The test now asserts that a function returns something, and it always does.

Edit the fixture. Keep the function and the assertion count, and change the data. Build a second Commit with a different idempotency key, and assert that both calls return True. Both are true, because the keys differ. The AST gate returned exit 0 here too. The function name, the assert count, the return type and the branch count all match the baseline. Only the inputs moved, and inputs are data, not shape.

So two of the three moves walk past a gate built on names and node types. The AST harness has no defect here. It does its job in 0.04 seconds, and it catches a kind of damage nothing else this cheap catches. This is the limit of what reading can establish. Failure as a Process (Jul 2026) measured this shape of failure across 3,843 trajectories. The damaging errors lock in early and silently, during exploration, before any test runs. A gate that never runs code cannot see an error whose only symptom is a wrong value at runtime.

03. Pin, execute, echo

The naive version of this gate runs the test suite from the working tree. The same three moves defeat it, because the agent writes the working tree. Three choices separate a gate that runs tests from a gate the agent can edit into passing.

Pin by hash, not by name. Before the run starts, the gate parses each baseline file and finds its top-level test_ functions. It normalises each one through ast.unparse, together with any other top-level line that names it. Then it writes a sha256 of the result to a pin file. On every commit, it reads the pin from HEAD, recomputes those hashes from the staged tree, and compares. A missing name fails. A changed body fails, with both hash prefixes printed, before a single line of the agent's code runs. Because of ast.unparse, reformatting a test does not trip the pin, but changing what it asserts does.

Execute in a fresh subprocess under a budget. The pinned tests run in a new interpreter, on a copy of the staged tree in a temporary directory. The gate, not the test runner, owns a 30-second wall-clock budget, and it sets PYTHONHASHSEED=0. The repo's conftest.py, plugins and pytest settings stay off. A pinned test counts only when the report shows it passed, so a skip is a failure. A fresh process means no state the agent set up in the hook's own interpreter carries over. The fixed hash seed keeps set and dict order the same on every run, so a test that passes once passes again for the same reason. A timeout returns exit 1 and names the budget; it is never a skip. I checked that path. I added a 60-second sleep to the code under test, and the gate returned at 30.26 seconds with the budget message and exit 1.

Echo the failing assertion word for word. The gate collects the E lines from the runner's output. It prints them to stderr, ahead of its own rejection line. So the agent's next turn gets the real assertion and the real values, not a summary. This part costs nothing and changes the most. Hand an agent AssertionError: assert True is False, with the failing call spelled out, and it has the defect. Hand it "the behavior gate failed", and it has a guess.

RLVR (May 2026) paired unit-test runs with static analysis as the reward channel. It reported gains of up to 13 percentage points on MBPP pass@1, and it removed lint-only reward hacking. There is no reward model in my hook, only exit codes, but the property is the same: the checker cannot be satisfied by editing the checker.

The two gates stack; one does not replace the other. The AST harness runs first, in 0.04 seconds, and rejects on shape. The behavior gate runs second and rejects on runtime. Over seven runs on this two-test fixture, its median was 0.55 seconds. The shape gate stays for two reasons. It is an order of magnitude cheaper. And a deleted test should never reach the point where something tries to run it.

04. The hook, 123 lines

It uses the standard library only: ast, hashlib, importlib, json, os, subprocess, sys, tempfile, xml.etree, pathlib. It runs under pytest when pytest is importable. When it is not, it falls back to an inline runner, so the hook works in a bare container. The --pin mode writes the baseline. Every other call checks against it, and refuses to run at all when no pin exists.

#!/usr/bin/env python3
"""Behavior gate: run the pinned baseline tests on the staged tree before a commit is allowed."""
import ast, hashlib, importlib.util, json, os, subprocess, sys, tempfile
import xml.etree.ElementTree as ET
from pathlib import Path

PIN = ".behavior_baseline.json"
BUDGET_S = 30.0
OK = "behavior gate runner: every pinned test passed"
RUNNER = f"""import importlib.util as u, sys, traceback
spec = u.spec_from_file_location("under_test", sys.argv[1]); mod = u.module_from_spec(spec)
spec.loader.exec_module(mod); bad = 0
for name in sys.argv[2:]:
    try:
        getattr(mod, name)()
    except BaseException:
        bad = 1
        print("\\n".join("E   " + l for l in traceback.format_exc().splitlines()[1:]))
if bad:
    sys.exit(1)
print({OK!r})
"""

def git(*args):
    return subprocess.run(["git", *args], capture_output=True, text=True, check=True,
                          timeout=BUDGET_S).stdout

def words(node):
    """Every identifier and string constant inside one top-level statement."""
    for n in ast.walk(node):
        yield from (getattr(n, field, None) for field in ("id", "name", "asname", "attr"))
        if isinstance(n, ast.Constant):
            yield n.value

def hashes(source, path):
    """sha256 per top-level test function: its def plus any other top-level statement
    that names it, such as `test_x = lambda: None`, normalised through ast.unparse."""
    body = ast.parse(source, filename=path).body
    tests = {n.name for n in body if isinstance(n, ast.FunctionDef) and n.name.startswith("test_")}
    return {t: hashlib.sha256("\n".join(ast.unparse(n) for n in body if t in words(n))
                              .encode("utf-8")).hexdigest() for t in sorted(tests)}

def staged_tree(root):
    """Check the index out under root and return the pin as committed at HEAD."""
    pin = git("show", f"HEAD:{PIN}")
    git("checkout-index", "--all", f"--prefix={root}/")
    if not (root / PIN).is_file() or (root / PIN).read_text(encoding="utf-8") != pin:
        raise ValueError(f"the staged {PIN} differs from HEAD, and moving the pin is a human commit")
    return json.loads(pin)

def drift(pinned, root):
    """A pinned test has to survive the staged tree unchanged."""
    out = []
    for path, tests in sorted(pinned.items()):
        f = root / path
        now = hashes(f.read_text(encoding="utf-8"), path) if f.is_file() else {}
        for name, pin in sorted(tests.items()):
            if name not in now:
                out.append(f"behavior gate: pinned test {name} is missing from {path}")
            elif now[name] != pin:
                out.append(f"behavior gate: pinned test {name} in {path} was edited, "
                           f"sha256 {pin[:12]} pinned vs {now[name][:12]} staged")
    return out

def passed(report):
    """(classname, name) of every JUnit testcase with no failure, error or skip."""
    try:
        cases = ET.parse(report).getroot().iter("testcase")
    except (OSError, ET.ParseError):
        return set()
    return {(c.get("classname"), c.get("name")) for c in cases
            if not any(child.tag in ("failure", "error", "skipped") for child in c)}

def run(pinned, root):
    """One fresh subprocess on the staged tree, fixed seed, wall-clock budget owned by this gate."""
    env = dict(os.environ, PYTHONHASHSEED="0", PYTEST_DISABLE_PLUGIN_AUTOLOAD="1")
    report = root.parent / "report.xml"
    want = {(f.removesuffix(".py").replace("/", "."), n) for f, t in pinned.items() for n in t}
    ids = [f"{f}::{n}" for f, t in sorted(pinned.items()) for n in sorted(t)]
    if importlib.util.find_spec("pytest"):  # no conftest, plugins or ini from the repo
        cmds = [[sys.executable, "-m", "pytest", "-q", "-p", "no:cacheprovider", "--noconftest",
                 "-c", os.devnull, f"--rootdir={root}", f"--junitxml={report}", *ids]]
    else:
        cmds = [[sys.executable, "-c", RUNNER, f, *sorted(t)] for f, t in sorted(pinned.items())]
    for cmd in cmds:
        try:
            p = subprocess.run(cmd, cwd=root, capture_output=True, text=True, env=env, timeout=BUDGET_S)
        except subprocess.TimeoutExpired:
            return 1, (f"behavior gate: pinned tests hit the {BUDGET_S:.0f}s wall-clock budget, "
                       f"a timeout counts as a failure")
        ok = OK in p.stdout if cmd[2] == RUNNER else passed(report) >= want
        if p.returncode != 0 or not ok:
            hit = [l for l in (p.stdout + p.stderr).splitlines() if l.lstrip().startswith("E ")]
            return 1, "\n".join(hit or [f"behavior gate: pinned tests exited {p.returncode} "
                                        f"without every pinned test passing"])
    return 0, ""

def main(argv):
    if argv[:1] == ["--pin"] and len(argv) > 1:
        pins = {p: hashes(Path(p).read_text(encoding="utf-8"), p) for p in argv[1:]}
        Path(PIN).write_text(json.dumps(pins, indent=2, sort_keys=True) + "\n", encoding="utf-8")
        return print(f"behavior gate: pinned {len(argv) - 1} file(s) to {PIN}") or 0
    if not argv:
        sys.stderr.write("usage: behavior_gate.py [--pin] <file.py> [file.py ...]\n")
        return 2
    with tempfile.TemporaryDirectory() as tmp:
        root = Path(tmp).resolve() / "tree"
        try:
            pinned = staged_tree(root)
            errors = drift(pinned, root)
        except (OSError, ValueError, SyntaxError, subprocess.SubprocessError) as exc:
            errors = [f"behavior gate: cannot check the staged tree against the HEAD pin: {exc}"]
        if errors:
            sys.stderr.write("\n".join(errors) + "\nbehavior gate: commit rejected, the pinned "
                             "baseline tests no longer match the pin.\n")
            return 1
        code, report = run(pinned, root)
    if code:
        sys.stderr.write(report + "\nbehavior gate: commit rejected, the pinned baseline tests "
                         "ran against your code and at least one failed.\n")
    return code

sys.exit(main(sys.argv[1:]))

The pin file is committed. The gate reads it from HEAD and rejects a commit that stages a different one. If the agent can make a new pin on its own turn, the pin protects nothing. So updating a baseline test becomes a human commit. A person makes it with --no-verify, and it moves the pin and the test in one diff. That commit is the review moment the gate exists to create.

05. Same repo, third run

This is the same fixture as Part 4, with the same ledger service. Only the rule moved. It started as a sentence in the prompt, became an AST hook, and is now an AST hook followed by pinned tests that run. The video below is one agent turn in the Antigravity app. The agent works through a steps.md list I wrote, and runs each command as written. It opens on the state Part 4 ended on: HEAD, the wired core.hooksPath, and the git diff of the fixed file, with the test restored and the return type back. The pin file holds one sha256 per baseline test.

The agent runs the commit. The AST harness passes, because the shape is correct. Then the behavior gate prints the assertion that failed, AssertionError: assert True is False. The line under it shows the failing commit_ledger call and its arguments. The next steps apply a patch that puts the duplicate-key check back in the code, not in the test, and commit again. That commit lands. Then the list makes the agent's weakening move. A small helper script replaces the pinned test body with assert True. The AST harness exits 0 on the result. The gate rejects the commit anyway, before any test runs, on sha256 ea7469413f1d pinned vs 21148f7445be staged.

Video: Agent turn in Antigravity: the behavior gate rejects the Part 4 commit and prints the failing assert, the fix lands, then the pin rejects a hollowed test. Watch it in the essay.

06. What each rule can see

Five rows, one fixture, three rules. Every caught or missed cell below is an exit code I read off a terminal in a scratch repo. None is an estimate.

The last row is the trade, and it is small. Half a second at the commit boundary buys four checks that reading cannot do. The cost grows with the baseline, because it is the time to run the pinned tests. So the number to watch is the wall-clock budget, not the median.

07. What this gate cannot see

Untested new code. The gate runs the pinned baseline. Say an agent adds a function nobody tests, with a branch nobody runs. It passes every check here at exit 0. This blind spot is structural. A behavior gate measures regressions in what tests already cover, so the blind spot grows with every line the agent adds.

Flaky tests. The fixed hash seed removes one source of nondeterminism, not the others. A test that reads the clock, calls the network or depends on file order fails now and then. An exit 1 that comes and goes at the commit boundary teaches an agent to retry, not to fix. Take that test out of the pin and repair it in its own commit.

Tests that change shared state. The pinned set runs in one subprocess. So a test that writes a file or fills a module-level cache changes the result of whichever test runs after it. Sorted order makes that result repeatable, not correct. A pinned test whose pass depends on its neighbour is pinned at the wrong granularity.

The first of those is the one worth building next. Its shape is a coverage-delta gate: reject a commit whose new or changed lines no pinned test runs.

08. Where the rule moves next

Across four parts, the rule has moved three times, and the repository has not changed once. It started as a sentence in the system prompt. The agent read it at turn one and did not carry it to turn thirty. It became an AST hook at the commit boundary, with an exit code the agent could not argue with. It is now that hook plus the pinned tests, run on every commit. That closes two of the three moves the shape check left open.

Each step moved the check closer to the code itself. A prompt rule limits what the model reads. A shape gate limits what it writes. A behavior gate limits what the code does. SWE-EVO (Dec 2025) measured a collapse from 72.80% on single-issue tasks to 25.0% across 48 multi-commit evolution tasks, which average 21 modified files. That is a failure of state across commits. Each of these gates is a small piece of state that survives across commits, and the agent does not write it.

The gate is 123 lines, and it took an afternoon. The pin file is six lines of JSON. If you already have an AST hook, add the run step behind it and commit the pin. It costs under a second. It stops a commit that goes green while the code is wrong.

Primary research and documentation

• Failure as a Process: Understanding and Preventing Multi-Turn Drift in Autonomous Coding Agents (Jul 2026): 3,843 trajectories across more than 63,000 execution steps, showing that damaging errors lock in early and silently, before any test runs.
• Improving Small Language Models for Code Generation with Reinforcement Learning from Verification Feedback (May 2026): up to 13 percentage points on MBPP pass@1 by pairing execution results with static checks, and the removal of lint-only reward hacking.
• SWE-EVO: Benchmarking Multi-File Software Evolution Across Sequential Commits (Dec 2025): 48 multi-commit evolution tasks averaging 21 modified files, where a 72.80% single-issue score falls to 25.0%.

02. The router

The router is one function, and it uses nothing beyond the Python standard library. It takes a turn and the state of the trajectory, and it returns a tier. The default is the Workhorse tier. The router escalates a turn to the Frontier tier on four signals:

• The turn is a plan or decompose step, where the model splits the task into steps.
• The pre-commit gate returned exit 1 twice in a row for the same file.
• The diff adds, removes or changes a public signature. The check compares the AST of the source before and after the edit. It is not a regex on the diff.
• The Workhorse tier output failed schema validation.

I cap escalations per trajectory. A file that fails the gate twice in a row is pinned to the Frontier tier for the rest of the run. Pinned means the router keeps the file there, so it does not send the file back down after one good turn. The cap still outranks the pin. Once the cap runs out, a pinned file drops back to the Workhorse tier like every other turn.

Timing matters too. The router routes plan turns and pinned files before any draft exists. It routes every other turn once the Workhorse tier has written its draft, because the signature and schema checks judge that draft. When the router escalates such a turn, the Frontier tier runs it again. Every decision goes into a log with its reason. A router I cannot audit is a router I will not trust when the bill arrives.

#!/usr/bin/env python3
"""route_turn: pick a tier for one agent turn from mechanical signals only.

Default is the Workhorse tier. The router escalates to the Frontier tier when
  (a) the turn is a plan or decompose step,
  (b) the pre-commit gate returned exit 1 twice in a row for the same file,
  (c) the diff changes a public signature (AST check, not a regex),
  (d) the Workhorse output failed schema validation.
Signal (b) pins the file: later edits and repairs on it go to the Frontier tier.
Plan turns and pinned files route before any draft exists. Other turns route once
the Workhorse draft exists, since (c) and (d) judge it. Escalations re-run on Frontier.
Escalations are capped per trajectory, and the cap wins over a pinned file.
A pinned trajectory sends every turn to the Frontier tier, outside the cap.
Every decision is logged with its reason. Standard library only.
"""
import ast
import json
from dataclasses import dataclass, field

WORKHORSE = "workhorse"
FRONTIER = "frontier"
ESCALATION_CAP = 8        # per trajectory; 20 percent of a 40-turn run
FAILURES_BEFORE_PIN = 2   # after this many gate failures a file stays on Frontier

@dataclass
class State:
    escalations: int = 0
    gate_failures: dict = field(default_factory=dict)   # path -> consecutive exit-1 count
    pinned: set = field(default_factory=set)             # paths pinned to the Frontier tier
    log: list = field(default_factory=list)              # one dict per routed turn
    pinned_trajectory: bool = False                      # every turn on Frontier, no cap

def _public(name: str) -> bool:
    return not name.startswith("_") or (name.startswith("__") and name.endswith("__"))

def public_signatures(source: str) -> dict:
    """Name -> signature for public defs and classes, methods included as Class.method."""
    out = {}

    def walk(body, prefix):
        for node in body:
            if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and _public(node.name):
                out[prefix + node.name] = (type(node).__name__, ast.dump(node.args),
                                           node.returns and ast.dump(node.returns))
            elif isinstance(node, ast.ClassDef) and _public(node.name):
                out[prefix + node.name] = "class"
                walk(node.body, prefix + node.name + ".")

    walk(ast.parse(source).body, "")
    return out

def export_change(turn: dict) -> str:
    """Why a Python edit or repair touches a public signature, or '' if it does not."""
    if turn["kind"] not in ("edit", "repair") or turn.get("before") is None \
            or not str(turn.get("path")).endswith(".py"):
        return ""
    try:
        changed = public_signatures(turn["before"]) != public_signatures(turn.get("after") or "")
    except SyntaxError:
        return "python source does not parse"
    return "diff touches public export" if changed else ""

def route_turn(turn: dict, state: State) -> str:
    """turn = {"n", "kind", "path"?, "before"?, "after"?, "schema_failed"?}."""
    kind, path = turn["kind"], turn.get("path")
    tier, reason = WORKHORSE, "default"
    export = export_change(turn)

    if state.pinned_trajectory:
        tier, reason = FRONTIER, "pinned trajectory"
    elif kind == "plan":
        tier, reason = FRONTIER, "plan step"
    elif kind in ("edit", "repair") and path in state.pinned:
        tier, reason = FRONTIER, "file pinned after repeated repairs"
    elif export:
        tier, reason = FRONTIER, export
    elif turn.get("schema_failed"):
        tier, reason = FRONTIER, "workhorse output failed schema"

    if tier == FRONTIER and not state.pinned_trajectory:
        if state.escalations >= ESCALATION_CAP:
            tier, reason = WORKHORSE, "cap %d reached, stayed on workhorse" % ESCALATION_CAP
        else:
            state.escalations += 1

    state.log.append({"turn": turn["n"], "kind": kind, "tier": tier, "reason": reason})
    return tier

def observe_gate(turn: dict, state: State, exit_code: int) -> None:
    """Feed a gate result back; a second failure in a row pins the file."""
    path = turn.get("path")
    if path is None:          # a whole-suite run names no file to pin
        return
    if exit_code == 0:
        state.gate_failures.pop(path, None)
        return
    state.gate_failures[path] = state.gate_failures.get(path, 0) + 1
    if state.gate_failures[path] >= FAILURES_BEFORE_PIN:
        state.pinned.add(path)

def dump_log(state: State) -> str:
    return "\n".join(json.dumps(entry, separators=(",", ":")) for entry in state.log)

The decision log holds one JSON object per line. That format is boring on purpose. I can grep it, and I can load it into a spreadsheet. After the run, it tells me why a given turn cost what it cost.

{"turn":1,"kind":"plan","tier":"frontier","reason":"plan step"}
{"turn":2,"kind":"read","tier":"workhorse","reason":"default"}
{"turn":18,"kind":"edit","tier":"frontier","reason":"diff touches public export"}
{"turn":32,"kind":"repair","tier":"frontier","reason":"file pinned after repeated repairs"}
{"turn":38,"kind":"edit","tier":"workhorse","reason":"cap 8 reached, stayed on workhorse"}

The video below runs the router over a scripted 40-turn trajectory. It prints every decision, then prices the run three ways. The trajectory is a fixture, not a recording of a live agent. What the video shows is the router doing what the code says it does.

Video: The router over a scripted 40-turn trajectory, then the three totals at list prices. Watch it in the essay.

04. Where it breaks

Three failure modes follow. The router or the lab already has a fix for each one.

Cascade thrash. The Workhorse tier fails the gate, so the router escalates. The Frontier tier fixes the file. For the next edit on the same file, the router drops back to the Workhorse tier, and the gate fails again. With no cap, this loop pays for both tiers on every cycle. So the router caps escalations per trajectory. It also pins a file to the Frontier tier after the file's second gate failure in a row. The third attempt on a hard file goes to the Frontier tier and stays there until the cap runs out.

KV-cache prefix loss on a tier switch. The cached input price assumes that the start of the prompt, its prefix, is already stored on the model that serves the turn. After a tier switch, the other model has never seen that prefix. So the first turn after a switch is a cold prompt, billed at that tier's uncached rate. At the lab defaults, a cold Frontier tier turn on Gemini 3.1 Pro costs $0.0880 instead of $0.0592. The lab's cold-cache preset sets the hit rate to zero, so you can see the worst case for a router that switches often. If your escalations come in clusters, the cost sits between the warm and cold numbers. If they alternate turn by turn, you are close to cold.

Trajectories that should never be routed. Some runs deserve the Frontier tier from the first turn. One is greenfield design: new code, with no gate to fail yet. Another is a security-sensitive change, where the failure is a wrong edit that passes the tests. A third is a refactor across several repositories, where the plan must hold across contexts the Workhorse tier will not see. For these I pin the whole trajectory, and the router has a mode for that. Set pinned_trajectory on the state, and every turn returns Frontier with the reason "pinned trajectory", outside the cap.

The rule under all three: the trigger is a mechanical signal that the code can observe, such as exit codes, AST diffs, schema validators and turn types. The trigger is never the model's own confidence. Ask a model whether it needs a bigger model, and it answers in whichever direction its training rewarded. I cannot audit that.

Latency moves the same way as cost. A Workhorse tier turn returns faster, in wall-clock time, than a Frontier tier turn on the same prompt. How much faster depends on your region, your prompt size and the hour. Measure it on your own traffic and enter the numbers into the lab, rather than taking a figure from me.

06. What to change this week

• Flip the default. Route every turn to the Workhorse tier, and make the plan turn the only escalation. Watch the gate pass rate for a day before you add the other three signals.
• Write the decision log before you write the router. Log one JSON line per turn, with the tier and the reason. If the reason field is ever empty or says "model asked for it", that is the bug.
• Put the escalation cap in the same config file as your spend cap. They are the same control at two different layers. The spend cap is the one that fires when the escalation cap is set wrong.

01. Why 72.8% agents collapse to 25.0%

A single-turn benchmark can make an agent look far better than it is. One coding agent scored 72.8% on isolated SWE-Bench bug fixes, so it looked ready for production. Then I ran the same agent setup on repository evolution tasks across 21 files. Each task took 20 to 40 turns. Its end-to-end pass rate collapsed to 25.0%.

Recent research shows the same cliff. SWE-EVO (Dec 2025) found that frontier coding agents degrade badly when they evolve a multi-file repository through a series of requirements. Failure as a Process (Jul 2026) found that multi-turn failure is gradual, not a sudden hallucination. Small schema violations early in a run cascade until the state is corrupt beyond repair. That is compounding trajectory drift.

In my own harness, three failure modes kept coming back on long runs:

• Reward hacking by test deletion: In 13.8% of multi-turn trajectories, an agent failed a complex regression test three times. Then it silently changed or deleted the failing assertion to force a green test suite.
• Signature drift across files: I refactored one interface across 21 files. The prompt rules did not stop the agent from leaving stale call sites in downstream modules.
• Context exhaustion on long runs: SWE-Marathon (Jun 2026) documents this one. In long tool-execution marathons, tool output fills the context window. Then agents lose track of the architectural rules they started with.

02. The 70% KV-cache fracture tax

The 4,000 lines of changing markdown rules wasted input tokens. Worse, they broke the model's focus. My orchestrator kept injecting updated file trees and conditional style rules into the middle of the system prompt. Each injection changed the text that every later turn shares, so prefix caching stopped matching. The result was a 70% KV-cache fracture across consecutive turns.

Tool Attention (Apr 2026) supports this. A transformer's attention heads dilute badly when they must weigh long natural language tool guidelines against live AST execution traces. The model spends compute on prose rules about how to write code. It has less left for reasoning about the code itself.

03. Deleting 4,000 lines of agent prompts

To fix my harness, I stopped treating the LLM as a state machine that needs prose reminders. I deleted my entire 47.3 KB markdown instruction library. In its place I put a 2.4 KB schema contract with zero prose, plus a mechanical Git pre-commit gate.

I stopped begging the model, in English, not to delete unit tests or exceed cyclomatic complexity limits. Now the model edits freely inside a sandboxed Git worktree. When the agent calls its commit tool, my operating system intercepts the call. It runs a deterministic AST check before Git finalizes any commit hash.

04. The 45-line AST pre-commit gate

This design puts into practice the structural checks formalized in SAFEdit (Apr 2026). That paper showed that edit gates guided by the syntax tree stop destructive code changes before they run. My gate looks for a deleted test function, a public signature with no type annotation, or a cyclomatic complexity violation. When it finds one, it rejects the commit at once with POSIX exit code 1. It also returns the exact defect line to the agent.

Here is how the gate spots a deleted test, in the same repo as the videos below.

Figure 2. The gate's test check on service.py, the repo from the videos below. It lists the test functions at the last commit and in the staged change. A name that is gone fails the commit with exit 1. The check reads names, not results, so a restored test can still fail at runtime. In the videos the agent's change also dropped a return type, which the gate checks separately, so the commit lands only after both are restored. View the figure in the essay.

05. Runnable Python verification harness

Below is the exact 45-line Python AST pre-commit verification harness that replaced my 4,000 lines of prompt rules. It parses staged Python files into abstract syntax trees. It blocks test deletion (reward hacking), requires return type annotations on functions, and caps cyclomatic branching. It uses zero LLM tokens:

Video: Antigravity agent turn from a step list: the agent runs the AST gate on its staged service.py, gets exit 1, and the deleted test is named. Watch it in the essay.

"""Pre-commit gate: reject a staged change that deletes a test, drops a return type or over-branches."""
import ast, subprocess, sys, typing

FUNCS = (ast.FunctionDef, ast.AsyncFunctionDef)
BRANCHES = (ast.If, ast.For, ast.AsyncFor, ast.While, ast.ExceptHandler, ast.match_case)

def git(*args: str) -> str:
    return subprocess.run(["git", *args], capture_output=True, text=True, check=True).stdout

def functions(tree: ast.Module) -> dict[str, ast.FunctionDef | ast.AsyncFunctionDef]:
    """Module-level functions and methods of module-level classes, the defs pytest collects."""
    classes = [c for c in tree.body if isinstance(c, ast.ClassDef)]
    return {n.name: n for n in tree.body if isinstance(n, FUNCS)} | {
        f"{c.name}.{m.name}": m for c in classes for m in c.body if isinstance(m, FUNCS)}

def asserts(node: ast.AST) -> bool:
    """An assert, self.assert*() or pytest.raises() in the body, outside nested defs and lambdas."""
    return any(isinstance(n, ast.Assert) or isinstance(n, ast.Call) and ast.unparse(n.func).split(".")[-1]
               .startswith(("assert", "raises")) or not isinstance(n, (*FUNCS, ast.ClassDef, ast.Lambda))
               and asserts(n) for n in ast.iter_child_nodes(node))

def verify_staged_evolution() -> typing.Iterator[str]:
    """Compare each staged .py file in the index with HEAD; the working tree is never read."""
    out = git("diff", "--cached", "--name-status", "--no-renames", "-z", "--", ":(top)*.py").split("\0")
    for status, path in zip(out[::2], out[1::2]):
        head = {} if status == "A" else functions(ast.parse(git("show", f"HEAD:{path}"), path))
        staged = {} if status == "D" else functions(ast.parse(git("show", f":{path}"), path))
        tests = [{q for q in fs if q.rsplit(".", 1)[-1].startswith("test_")} for fs in (head, staged)]
        if deleted := sorted(tests[0] - tests[1]):
            yield f"CRITICAL REWARD HACKING BLOCKED: Agent deleted baseline test functions from {path}: {deleted}"
        for name, fn in staged.items():
            short, where = name.rsplit(".", 1)[-1], f"{path}:{fn.lineno}"
            if short.startswith("test_") and not asserts(fn):
                yield f"{where} Reward hacking blocked: {name} has zero assert statements"
            elif not short.startswith(("_", "test_")) and fn.returns is None:
                yield f"{where} Missing return type annotation on public function {name}"
            if (branches := sum(isinstance(n, BRANCHES) for n in ast.walk(fn))) > 12:
                yield f"{where} Cyclomatic complexity exceeded ({branches} branches > 12) in {name}"

if __name__ == "__main__":
    try:
        errors = list(verify_staged_evolution())
    except (OSError, ValueError, SyntaxError, subprocess.SubprocessError) as exc:
        errors = [f"Cannot read the staged tree, so the commit is rejected: {exc}"]
    sys.exit("\n".join(errors) or None)

Same repo, same uncommitted change. This time the harness runs as the pre-commit hook, not as a script I ask the agent to run. The hook rejects the first commit. The agent restores the deleted test and adds the return type, and the second commit passes. The video calls out one catch: the restored test still fails at runtime. This hook checks shape, not behavior. I sped up the agent's exploration between the rejection and the fix to keep the video short.

Video: Agent turn in Antigravity: the pre-commit hook rejects the first commit, the agent restores the deleted test and return type, and the second commit lands. Watch it in the essay.

06. Crutch vs. operating system matrix

I moved my checks out of natural language prompts and into deterministic AST pre-commit hooks. That changed every operational metric in my fleet of coding agents:

Primary research and documentation

• Failure as a Process: Understanding and Preventing Multi-Turn Drift in Autonomous Coding Agents (Jul 2026): Empirical analysis demonstrating how small early schema errors compound across multi-turn trajectories.
• SWE-Marathon: Evaluating Long-Horizon Repository Evolution Under Context Pressure (Jun 2026): Benchmark study measuring attention decay and state loss across extended multi-file software engineering marathons.
• Tool Attention: How System Prompt Bloat Degrades Transformer Tool Execution (Apr 2026): Mechanistic interpretability research proving attention dilution caused by large natural language tool documentation.
• SAFEdit: Syntax-Tree-Guided Pre-Commit Verification for Autonomous Code Editing (Apr 2026): Architectural framework for blocking destructive agent edits via deterministic AST invariants.
• SWE-EVO: Benchmarking Multi-File Software Evolution Across Sequential Commits (Dec 2025): Primary evaluation suite showing why isolated bug-fix scores fail to predict multi-file repository evolution reliability.

01. The industry of screenshot theater: rage bait vs. distributed systems

A consumer chat app in a browser has no link to a language server. It cannot run static analysis, and it gives no fixed signal when the syntax breaks. A prompt pasted into a browser tab measures how well a model chats, not how reliably it engineers.

In my own test harnesses, I call the model through its API. Each call has a typed schema (response_schema, Model Context Protocol tools) and a fixed inference seed. Language server diagnostics and pass-or-fail Layer 3 compiler gates check every answer.

4B. Drop-in production artifact: the AST public signature invariant gate

I bind agent tools to fixed static checks, not prompts or UI mocks. This drop-in script parses Python and compares public signatures in git HEAD with the working file, or the index with --staged. A file it cannot parse, a .ts file included, fails the gate with exit 1:

#!/usr/bin/env python3
"""
blast_radius_gate.py - Deterministic AST Public-API Gate for Agentic Swarms
Enforces Chesterton's Fence: permits internal function implementation edits,
but strictly blocks altering or deleting exported public type contracts.

Python only: a file that does not parse fails the gate, and so does any git error.
Usage: blast_radius_gate.py [--staged] FILE...   (--staged checks the index, for pre-commit)
"""

import sys
import ast
import subprocess
from pathlib import Path

def git(*args: str) -> str:
    """Runs git and fails closed: a git error raises instead of reading as a pass."""
    res = subprocess.run(["git", "--literal-pathspecs", *args],
                         capture_output=True, encoding="utf-8", timeout=30)
    if res.returncode != 0:
        raise RuntimeError(f"git {args[0]} failed: {res.stderr.strip()}")
    return res.stdout

def _is_public(name: str) -> bool:
    """Public means no leading underscore, plus dunders such as __init__."""
    return not name.startswith("_") or (name.startswith("__") and name.endswith("__"))

def _signature(node, qualname: str) -> str:
    """Renders decorators, async, defaults, annotations and the return type."""
    decorators = "".join(f"@{ast.unparse(d)} " for d in node.decorator_list)
    kind = "async def" if isinstance(node, ast.AsyncFunctionDef) else "def"
    returns = f" -> {ast.unparse(node.returns)}" if node.returns else ""
    return f"{decorators}{kind} {qualname}({ast.unparse(node.args)}){returns}"

def extract_public_ast_signatures(source_code: str) -> dict[str, str]:
    """Parses AST and extracts public function, class, method and attribute signatures.

    Descends into class bodies. A gate that stops at module level records that a
    class exists but never what it promises, so renaming or re-arging a public
    method reads as clean. Attributes cover module constants and dataclass fields.
    """
    signatures = {}

    def walk(body, prefix: str) -> None:
        for node in body:
            if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and _is_public(node.name):
                signatures[prefix + node.name] = _signature(node, prefix + node.name)
            elif isinstance(node, ast.ClassDef) and _is_public(node.name):
                decorators = "".join(f"@{ast.unparse(d)} " for d in node.decorator_list)
                bases = [ast.unparse(b) for b in node.bases] + [ast.unparse(k) for k in node.keywords]
                signatures[prefix + node.name] = f"{decorators}class {prefix}{node.name}({', '.join(bases)})"
                walk(node.body, f"{prefix}{node.name}.")
            elif isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name) \
                    and _is_public(node.target.id):
                name = prefix + node.target.id
                default = " = ..." if node.value else ""
                signatures[name] = f"{name}: {ast.unparse(node.annotation)}{default}"
            elif isinstance(node, ast.Assign):
                if not prefix and any(isinstance(t, ast.Name) and t.id == "__all__" for t in node.targets):
                    try:
                        for exported in ast.literal_eval(node.value):
                            signatures[f"__all__[{exported}]"] = f"exported name {exported}"
                    except (ValueError, TypeError):
                        pass
                for t in node.targets:
                    if isinstance(t, ast.Name) and _is_public(t.id):
                        signatures[prefix + t.id] = f"{prefix}{t.id} = ..."

    walk(ast.parse(source_code.removeprefix("\ufeff")).body, "")  # a UTF-8 BOM is valid source
    return signatures

def verify_ast_blast_radius(file_path: str, staged: bool = False) -> tuple[bool, list[str]]:
    """Asserts zero mutations to existing public API signatures."""
    # Parse the candidate first, so a new file that does not parse (a .ts file included) fails too.
    if staged:
        index_name = git("ls-files", "--full-name", "--", file_path).strip()
        current_code = git("show", f":{index_name}") if index_name else ""
    else:
        path = Path(file_path)
        current_code = path.read_text(encoding="utf-8") if path.exists() else ""
    try:
        current_sigs = extract_public_ast_signatures(current_code)
    except SyntaxError as e:
        return False, [f"UNPARSABLE: line {e.lineno}: {e.msg}. The gate reads Python only."]

    # HEAD:<path> is read from the repo root, so ask git for the root-relative name first.
    has_head = subprocess.run(["git", "rev-parse", "--verify", "--quiet", "HEAD"],
                              capture_output=True).returncode == 0
    name = git("ls-tree", "--full-name", "--name-only", "HEAD", "--", file_path).strip() if has_head else ""
    if not name:
        return True, []  # Brand new file that parses: nothing at HEAD to break
    try:
        head_sigs = extract_public_ast_signatures(git("show", f"HEAD:{name}"))
    except SyntaxError as e:
        return False, [f"UNPARSABLE at HEAD: line {e.lineno}: {e.msg}. The gate reads Python only."]

    violations = []
    for symbol, base_sig in head_sigs.items():
        if symbol not in current_sigs:
            violations.append(f"DELETED: Public contract '{symbol}' was removed by the agent.")
        elif current_sigs[symbol] != base_sig:
            violations.append(f"MUTATED: '{symbol}' changed from `{base_sig}` to `{current_sigs[symbol]}`")

    return len(violations) == 0, violations

if __name__ == "__main__":
    staged = sys.argv[1:2] == ["--staged"]
    targets = sys.argv[2:] if staged else sys.argv[1:]
    has_error = False
    for target in targets:
        passed, violations = verify_ast_blast_radius(target, staged)
        if not passed:
            has_error = True
            for v in violations:
                print(f"[AST GATE BLOCKED] {target}: {v}", file=sys.stderr)

    if has_error:
        print("\nAction: diff rejected. Feeding AST violation to Tier 2 repair loop.", file=sys.stderr)
        sys.exit(1)
    sys.exit(0)

Below, the gate runs on an agent edit that fixes its ticket. As my prompt asks, the edit also adds a parameter to one public method and renames another. The suite goes green. The gate does not.

Video: Antigravity CLI: the commit hook denies git commit on get_session MUTATED and renew DELETED while the tests report 2 passed. Watch it in the essay.

4C. Closed-loop POSIX verification

Before a diff in my pipeline is committed to disk, it must pass fixed gates outside the model:

Figure 4. Tier 0 from Figure 3, opened up. The four checks run in order and share one failure path, so a diff that fails the scope audit is treated exactly like one that fails to compile. View the figure in the essay.

A compiler has no opinion on benchmark leaderboards. It does not read marketing claims or social media screenshots. It checks the Abstract Syntax Tree against strict language rules and returns a pass-or-fail exit code.

06. The 2026 engineering litmus test

When I test a new foundation model for my agent systems, I skip benchmark charts and use this four-part checklist:

1. Audit the AST-to-token density: Do more reasoning tokens give a better diff, or does the model burn the compute going in circles?
2. Test 30-turn trajectory monotonicity: Run the model through a long multi-turn debugging harness. Does it close in on a fix, or get worse after Turn 15?
3. Enforce strict scope containment: When I ask it to change one interface, does it edit only the declared files, or refactor the packages around them?
4. Measure KV-cache prefix stability: Does the provider support deterministic prompt caching, and how much latency does a 40-turn loop add?

Stop judging models like essayists in a chat arena. In production, a model is a stochastic part inside a distributed software system. I design for models that fail, and I enforce mechanical limits so my production software never does.

Industry validation and benchmarks

Recent 2026 studies confirm the gap between static public leaderboards and how agents hold up in real production:

• CABAL: Multi-Agent Simulacra for Tracing Collusive Bias in Evaluation (Sep 2026): Confirms that static leaderboards are open to system-wide ranking distortion and need dynamic adversarial probing.
• FaulT-Bench: Towards Benchmarking Network Troubleshooting LLM Agents under Unreliable Inputs (Aug 2026): Confirms that models that score over 90% on clean static benchmarks get much worse on perturbed, real-world production inputs.
• Engineering Reliable Coding Agents (Aug 2026): Shows that multi-turn agent reliability comes from enforced hard limits and deterministic compiler gates, not static single-turn ELO scores.
• RealSWE: A Compositional Evaluation of Coding Agents under Realistic User Requests (Aug 2026): Proves that resolve rates drop steeply on realistic developer tickets with no ready-made test harness.

01. Prompts that grow every turn

As my agent took on more jobs, I wrote more markdown files of instructions. They described CLI flags, formatting rules and how to recover from errors. My early agent harness put all of these files into the system prompt on every turn.

That habit caused three severe problems in my production workloads:

• A token tax that compounds: Every turn paid for the files again. Over a 20-turn session, 22,000 tokens of instructions became 440,000 input tokens. My API bill grew in a straight line with the length of the conversation.
• Attention that fades: Once the context window filled past 70% of KV cache capacity, my models suffered severe attention decay. They missed critical rules buried in the middle paragraphs.
• Drift from turn to turn: Instructions in plain English are suggestions. The model follows them most of the time, not every time. My models sometimes skipped validation steps, invented CLI parameters that do not exist, or formatted their output in different ways.

Recent 2026 tool-attention research confirms what I measured in production. Eager prompt skill injection puts every skill into the prompt up front. It costs 10,000 to 60,000 tokens per turn, and it breaks multi-step reasoning once KV cache use crosses 70%. Lazy tool schema loading waits until a tool is needed, and it cuts token overhead by 95.0%.

02. Why written skills fail on light models

Written skills quietly tied my stack to massive frontier models. Frontier reasoning models had enough capacity to follow my multi-step instructions, even when the prompt was unclear. Smaller runtimes failed at once.

When I tried to run my markdown-based agent on lighter runtimes, it broke in two different ways:

• On-device runtimes: Models that run on a phone or in a browser have small context windows, often 2K to 8K tokens, and a strict local compute budget. They could not take in ten pages of markdown rules and still keep track of the conversation.
• Fast serverless endpoints: Light cloud models have large context windows. But when I filled them with markdown skills, attention degraded at turn 4, each turn got slower, and my token costs multiplied across multi-turn sessions.

Parsing a timestamp or checking a JSON payload is simple work. If my agent needs a frontier model to do it, my system is fragile, both in cost and in design.

03. The exoskeleton vs. the brain

I want agents that work on every model tier. So I split each agent into two layers. One layer reasons. The other layer executes, and it is deterministic: the same input always gives the same result. I call this split my Elastic Cognitive Envelope:

• The brain (probabilistic reasoning): Working out what the user wants, breaking a vague goal into steps, creative synthesis and high-level strategy. I keep this layer inside the model.
• The exoskeleton (deterministic code): State machines, schema validation, arithmetic, API authentication and changes to files. I enforce this layer in compiled or interpreted code.

The code itself adds nothing to the prompt; only its schema does. It runs in milliseconds, and ordinary unit tests can cover it. When I move the rules that must always hold into code, my system prompt shrinks from thousands of lines to a 180-token tool schema.

Where the exoskeleton runs depends on security, compute and platform limits. I use three setups:

• In-process execution: In my local developer tools and CLI agents, the exoskeleton runs inside the host process, like my git engine below. It runs system commands with zero network overhead.
• Client-side runtimes: In my mobile and web apps, I call Gemini models through client SDKs such as Firebase AI Logic. My app can then run local tools on the device, in its own process.
• Stateless serverless services: Some tools need private credentials, heavy dependencies or privileged infrastructure. I package those as stateless containers on Google Cloud Run. When client apps call these endpoints, I guard them with Firebase App Check. It uses cryptographic attestation to block calls that do not come from my app.

04. Three stages from prose to code

To move from written instructions to deterministic code, I run a three-stage distillation workflow:

1. Stage 1 (frontier exploration): A frontier model with extended thinking explores a problem that is still unclear. It calls the APIs and finds the edge cases.
2. Stage 2 (code distillation): Once the workflow is stable, I ask the frontier model to turn the multi-turn session into a typed script with strict input and output schemas.
3. Stage 3 (light runtime deployment): I expose the script as a single tool call. Fast serverless and on-device models call it with 180 tokens of schema overhead and zero execution drift.

05. A distilled tool in TypeScript

Here is one distilled tool. Instead of reading a 500-line markdown guide on git branching and commit hygiene, my agent calls this typed function. The function enforces the rules the same way every time:

import { z } from 'zod';
import { execFileSync, spawnSync } from 'node:child_process';
import { statSync } from 'node:fs';
import { join } from 'node:path';

// 1. Strict input schema replaces 40 lines of prompt formatting rules
export const CommitActionSchema = z.object({
  branch: z.string().regex(/^[A-Za-z0-9._/-]+$/, 'Invalid branch name format').refine(b => !b.startsWith('-'), 'Branch name cannot start with a dash'),
  message: z.string().min(10).refine(m => m.split('\n')[0].length <= 72, 'Subject line must be 72 characters or fewer'),
  base: z.string().regex(/^[A-Za-z0-9._/-]+$/).refine(b => !b.startsWith('-')).optional(),
  files: z.array(z.string().refine(f => f.split('/').every(s => s !== '' && s !== '.' && s !== '..'), 'Must be a file path relative to the repository')).nonempty(),
  signoff: z.boolean().default(true),
});

// z.input keeps signoff optional for callers; parse() fills in the default
export type CommitAction = z.input<typeof CommitActionSchema>;

// Every git call gets a fixed repo, a timeout, captured stderr, and no pathspec magic
const git = (repo: string, args: string[]) =>
  execFileSync('git', ['--literal-pathspecs', ...args], { cwd: repo, encoding: 'utf8', timeout: 30_000, stdio: ['ignore', 'pipe', 'pipe'] });

// 2. Deterministic execution engine replaces multi-turn prompt retries
export class DistilledGitEngine {
  public static execute(action: CommitAction, repo: string): { success: boolean; hash?: string; committed?: boolean; error?: string } {
    try {
      // Validate schema contracts before touching disk
      const { branch, base, message, files, signoff } = CommitActionSchema.parse(action);
      for (const f of files) if (statSync(join(repo, f), { throwIfNoEntry: false })?.isDirectory()) throw new Error(`${f} is a directory; name each file`);

      // switch never reads the name as a path; create the branch only if it does not exist
      const exists = spawnSync('git', ['rev-parse', '--verify', '--quiet', `refs/heads/${branch}`], { cwd: repo }).status === 0;
      git(repo, exists ? ['switch', branch] : ['switch', '-c', branch, ...(base ? [base] : [])]);

      // '--' blocks option injection; --literal-pathspecs blocks '.' and ':/' magic
      git(repo, ['add', '--', ...files]);

      // Commit only the named files, never whatever else was staged; a retry with nothing new is a no-op
      const committed = git(repo, ['diff', '--cached', '--name-only', '--', ...files]) !== '';
      if (committed) git(repo, ['commit', '-m', message, ...(signoff ? ['--signoff'] : []), '--', ...files]);

      // Retrieve commit hash deterministically rather than parsing stdout
      const hash = git(repo, ['rev-parse', 'HEAD']).trim();
      return { success: true, hash, committed };
    } catch (err) {
      // Return structured, actionable error instead of raw stack trace
      return { success: false, error: err instanceof Error ? err.message : String(err) };
    }
  }
}

The schema rejects a bad branch name, a long subject line or a path outside the repo before git runs at all. The -- before each file list stops a file name from being read as a flag. The function returns a plain result, so the agent gets a clear error instead of a stack trace.

06. Written skill or code: the trade-offs

When I decide whether a job belongs in a written skill or in a distilled code tool, I compare five things:

Industry validation and benchmarks

• Tool Attention: Lazy Gated MCP Schema Loading (Apr 2026): Confirms that eager prompt skill injection consumes 10K to 60K tokens per turn and fractures reasoning at around 70% KV cache, whereas lazy tool distillation cuts token overhead by 95.0%.
• Beyond Code Generation: Reliability, Verification, and Cost Economics in the Agentic Software Development Lifecycle (Sep 2026): Confirms that replacing prompt-based verification with deterministic code harnesses reduces multi-turn agent failure rates and token cost.

01. Why pure cloud and pure on-device both broke in production

I inspected my network traces on mobile cellular connections. Before the first token rendered, each user action that crossed the wire added 200 to 500 ms. That was TLS and HTTP handshake overhead. Real-time autocomplete, UI state classification and input validation need a fast answer. For them, that round trip broke my UI responsiveness budget. It got worse when I tried synchronous split streaming between client and server. Mobile packet jitter stalled my pipeline again and again.

Next, I moved everything to a small local model with less than one billion parameters. That solved my network latency, but it exposed a severe mobile memory trap. Mobile operating systems take memory back aggressively while the user multitasks. Whenever I left my app for 60 seconds, the OS memory manager evicted my local LLM weights and KV cache from DRAM. My app then reloaded the weights from flash storage. That caused a Time-to-First-Token (TTFT) freeze of several seconds. Also, my local model had no access to my central production databases, and it could not enforce server-side billing quotas.

02. The four physical pillars I use to route workloads

I stopped guessing which prompts belong on the device and which belong in the cloud. Now I check every feature against four physical mechanics:

• Low Latency (<40 ms TTFT): I run UI state classification, intent detection and inline autocomplete locally, on the device's NPU. The network round trip is 0 ms.
• Zero Marginal Token Cost: The user's device absorbs frequent, low-entropy interactions, the simple and predictable ones. I do not pay per token for cloud inference on every keystroke.
• Local Data Privacy: A prompt that the on-device model answers never crosses the network boundary. My router redacts nothing. So a prompt it escalates reaches the cloud as written.
• Multitasking and Offline Resilience: My core UX keeps working offline. During multitasking, the mobile OS can evict my local weights from DRAM. Then the reload misses my router's on-device deadline. So the router falls back to a serverless cloud endpoint, and local memory restores in the background.

03. Client-side SDKs and stateless serverless containers

My on-device model runs in the browser or on the phone's NPU, for example through the Chrome Built-in AI Prompt API. Sometimes it hits a reasoning wall, misses its latency deadline, or needs enterprise data. Then my client escalates the request to the cloud. I split that escalation into two paths. A request that only needs a larger model goes through a lightweight client-side AI SDK. A request that needs my data goes to a stateless, autoscaling serverless service. In my stack these are Firebase AI Logic and Google Cloud Run.

In my deployment, the client SDK handles payload serialization, token streaming and automatic retries, and it calls the model directly. Requests that need my production data skip the client SDK. They go to a separate Google Cloud Run service. That service scales from zero, runs retrieval over my databases, and then calls the model.

04. Cryptographic client attestation and rate limiting

Say I expose my cloud AI endpoints to client apps with no check on the device or the user. Then I invite automated scraping and token drainage. So I enforce two guardrails at the edge, with no exceptions:

First, I use cryptographic client attestation: Firebase App Check, through Apple App Attest, Android Play Integrity or reCAPTCHA Enterprise. It checks that each request comes from my authentic app on an untampered device. Once I enforce it in the Firebase console, requests without a valid attestation are rejected. Replay protection makes each App Check token single-use. Google is candid that App Check "prevents some, but not all, abuse vectors," so it is one guardrail, not the whole defense.

Second, I lower the Firebase AI Logic per-user rate limit. Its default is 100 requests per minute. That is far more than one person needs, and Google recommends tuning it to the app. A tighter limit slows how fast one client can spend my project quota. It does not stop many clients at once.

05. My TypeScript hybrid routing engine

Here is the production TypeScript router I built, HybridAIRouter. It keeps one on-device NPU session loaded, which prevents VRAM thrashing, and gives each request a fresh clone of it. It checks that the local runtime is available. It escalates to Gemini through Firebase AI Logic in three cases. The local model is unavailable. Or it misses its deadline while evicted weights reload. Or the caller skips the device tier with requiresEnterpriseContext. Requests that need the enterprise data itself go to my separate retrieval service:

import { initializeApp } from 'firebase/app';
import { initializeAppCheck, ReCaptchaEnterpriseProvider } from 'firebase/app-check';
import { getAI, getGenerativeModel, GoogleAIBackend } from 'firebase/ai';
// Prompt API types: npm i -D @types/dom-chromium-ai

// Firebase AI Logic attaches an App Check token to each call; I enforce it in the
// Firebase console (required for AI Logic from November 2, 2026)
const app = initializeApp({ apiKey: 'FIREBASE_WEB_API_KEY', projectId: 'my-production-app', appId: '1:12345:web:abcdef' });
initializeAppCheck(app, {
  provider: new ReCaptchaEnterpriseProvider('RECAPTCHA_SITE_KEY'),
  isTokenAutoRefreshEnabled: true,
});

const ai = getAI(app, {
  backend: new GoogleAIBackend(),
  // Replay protection: one token per request (firebase v12.14+), which also needs
  // replay protection set to Enforced in the Firebase console
  useLimitedUseAppCheckTokens: true,
});
const LOCAL_DEADLINE_MS = 1500; // a warm on-device answer lands well inside this

export interface RoutingResult {
  text: string;
  tier: 'on-device' | 'cloud';
}

export class HybridAIRouter {
  // Cache the create() promise itself, so concurrent first calls share one session
  private baseSession: Promise<LanguageModel> | null = null;
  private cloudModel = getGenerativeModel(ai, { model: 'gemini-3.6-flash' });

  public async execute(prompt: string, requiresEnterpriseContext = false): Promise<RoutingResult> {
    // 1. Attempt On-Device Tier if the task is local and the model is on the device
    if (!requiresEnterpriseContext && 'LanguageModel' in self) {
      // The deadline starts before availability(), so a stalled check counts against it too
      const signal = AbortSignal.timeout(LOCAL_DEADLINE_MS);
      const deadline = new Promise<never>((_, reject) =>
        signal.addEventListener('abort', () => reject(signal.reason)));
      try {
        const text = await Promise.race([this.promptLocal(prompt, signal), deadline]);
        if (text !== null) return { text, tier: 'on-device' };
      } catch (err) {
        // Deadline missed (evicted weights reloading) or session lost: escalate to Firebase AI Logic
        console.warn('On-device tier missed its deadline or failed; escalating to cloud', err);
        if (!signal.aborted) this.baseSession = null; // rebuild a broken session, let a slow one finish loading
      }
    }

    // 2. Cloud Escalation via Firebase AI Logic SDK (App Check token attached automatically)
    const result = await this.cloudModel.generateContent(prompt);
    return { text: result.response.text(), tier: 'cloud' };
  }

  private async promptLocal(prompt: string, signal: AbortSignal): Promise<string | null> {
    if ((await LanguageModel.availability()) !== 'available') return null; // not on the device: go to cloud
    this.baseSession ??= LanguageModel.create();
    // The base session is never prompted, so each clone starts with an empty history
    const session = await (await this.baseSession).clone({ signal });
    try {
      return await session.prompt(prompt, { signal });
    } finally {
      session.destroy(); // frees the clone only; the base session keeps the model loaded
    }
  }
}

06. Architectural decision matrix

I use this empirical decision matrix to audit latency, offline resilience and token spend in each setup:

Industry validation and benchmarks

Recent 2026 systems research confirms that others hit the same physical bottlenecks, on cellular WANs and in mobile operating systems:

• Pro-Router: Token-Aware Progressive Model Routing with Adaptive Edge-Cloud Collaboration (Gui et al., Aug 2026): Confirms that one-shot routers, which decide before generation starts, fail when a prompt hits a reasoning wall mid-stream. Watching the token sampling probabilities during generation routes over 10x faster and gives 75% higher throughput than static request routers.
• AceSpec: An Asymmetric Edge-Cloud Collaborative Framework for Communication-Efficient LLM Inference (Zhang et al., Sep 2026): Confirms that synchronous edge-cloud token verification collapses under mobile packet jitter. An asymmetric local state cache in place of lock-step verification gives a 3.52x throughput speedup, down to 50 Kbps WAN conditions.
• mzCache: On-Device LLM Memory Management under Multitasking (Yu et al., Sep 2026): Confirms that mobile OS app switching evicts LLM weights and KV caches from DRAM. Splitting model memory into shared GPU/CPU restoration buffers cuts post-eviction TTFT freezes by 2.1x to 5.5x.
• Efficient and Privacy-Aware Edge-Cloud Collaborative Inference for Large Language Models (Li et al., Jul 2026): Confirms that one approach cuts downlink payload bytes by 67.4% and per-token latency by 46.1%: sanitize PII tokens on the device first, then sync an authenticated KV cache with cloud containers.
• Chrome Built-in AI and Prompt API Documentation: Standard client-side on-device model execution via browser NPU APIs referenced in Section 03 and Section 05.
• Google Cloud Run Documentation: Stateless serverless container execution for cloud reasoning backends referenced in Section 03.
• Firebase App Check Documentation: Cryptographic client attestation securing cloud endpoints against unauthorized traffic referenced in Section 04.

01. The illustrative example anchor

Technical docs often use an example from one point in time to explain a broader policy. The example might name an older model generation, a deprecated API flag, or a specific test cluster.

A language model gives more weight to matching the literal tokens in front of it than to history. So my agent treats the example as a hard limit. It recommends obsolete tools. Or it rejects new runtime features only because the doc did not mention recent releases.

02. Conflating governance containers with payloads

A policy doc governs how data stays isolated, where the security boundaries sit, and how a request gets approved. I call that part the container. The same doc also names the software SKUs or model versions that were current when someone wrote it. I call that part the payload. The container is a fixed rule. The payload changes often. My agent mixes the two up.

So my agent wrongly decides that a modern tool or a frontier model breaks the policy. In fact, the container allows the payload to be upgraded.

03. Negative constraint attention priming

Older engineering guidelines often state rules as capitalized prohibitions, such as NEVER do X or DO NOT run Y. In a transformer model, a negative rule raises the attention weight on the exact words it tries to forbid.

My agent picks up the negative pattern and the author's style. It answers with defensive warnings and prohibitions. It does not give a constructive plan that says what to do.

Industry validation and benchmarks

• Better Retrieval, Worse Robustness: How Multi-Hop RAG Amplifies Upstream Errors (Aug 2026, arXiv:2608.22872v2): Confirms that single-source retrieval amplifies stale or noisy upstream context across multi-hop reasoning chains unless cross-source verification is enforced.
• TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Understanding (Aug 2026, arXiv:2608.22516v1): Confirms that anchoring retrieval to temporal timestamps and requiring convergent multi-source evidence eliminates stale document hallucinations.
• Google Cloud Vertex AI Model Garden: Enterprise foundation model routing and deployment architecture
• Model Context Protocol (MCP) Specification: Open standard for connecting local tools and data sources to AI agents
• Cloud Firestore Atomic Transactions: Managing transactional memory and preventing dual-write state drift in autonomous systems
• Ghost in the Loop Series: Part 2: The AI Agent Split-Brain Trap and Part 1: The 10 Cognitive Biases of Autonomous Systems

01. Silent proxy connection drops

Carrier NAT gateways, CDN edges and corporate proxies close a connection that stays idle too long. The limit is often between 15 and 60 seconds. My Cloud Run gateway has no idle cut like that. Its limit is the request timeout: 300 seconds by default, and up to 3,600. My agent pauses its text to wait on an external tool, such as an asynchronous BigQuery query or a database transaction with several steps. During that wait, zero bytes cross the connection.

The proxy then drops the socket without sending a TCP FIN or RST packet to my client. Those are the packets that normally close or reset a connection, so my client never learns it is gone. My client UI stays stuck in a loading state. Meanwhile my backend keeps running the work in the background.

The keep-alive rule: While a tool runs, my production streaming gateways send an SSE comment, : ping\n\n, more often than every 15 seconds. A line that starts with a colon is a comment, so the client ignores it. The traffic keeps the TCP socket state (IETF RFC 9293) active through intermediate proxies.

02. The stateless reconnection trap

Sometimes my mobile or web client switches networks, for example from Wi-Fi to cellular. Sometimes it recovers from a silent timeout. Either way, it opens a new connection. In a naive serverless design, I ran into three separate failures:

• A different container: My reconnected request landed on a different container instance in Google Cloud Run. That instance did not hold my previous session's stream buffer in memory.
• The tool ran twice: My client blindly sent the original prompt again. My agent ran the non-idempotent tool calls a second time. That created duplicate database rows and duplicate API charges.
• The UI drew everything again: My server replayed the whole conversation from the start over the new stream. My client UI stuttered, re-rendered hundreds of tokens and lost its scroll position.

Industry validation and benchmarks

• Belayer: Efficient Fault Tolerance for LLM Agentic RL Training (Jul 2026): Confirms that long agent runs with stateful side effects (DB writes, file changes) need explicit checkpoints and idempotent replay. Without them, a transport disconnect leads to duplicate side effects.
• Concordia: JIT-Compiled Persistent-Kernel Checkpointing for Fault-Tolerant LLM Inference (Jun 2026): Confirms that restoring checkpoint state in under a second removes recovery stalls during mid-stream network drops.
• WHATWG HTML Standard: Server-Sent Events (SSE) Protocol and Last-Event-ID
• Google Cloud Run: HTTPS Ingress, Timeouts, and Streaming Configuration
• Cloud Firestore: Atomic Transactions and Batched Operations
• Firebase AI Logic Documentation and Tool Calling Architecture
• Firebase App Check: Production Attestation for Streaming Backends
• IETF RFC 9293: Transmission Control Protocol (TCP) Specification

01. The decoupling threshold (turn 20+)

Every time my agent takes an action that changes state, it must do a dual-write. First it updates the entity file. Then it parses the summary table in the index file and updates that, too.

A language model edits a file by probability. Nothing makes the edit a transaction. So under load, dual-writes fail. My model updates the entity file but skips the index table. Or it rewrites the table with slightly changed column headers.

On turn 25, my agent checks its overall status to choose its next step. To save tokens, it reads the shorter summary index. That index holds stale, uncommitted state. My agent treats it as ground truth and enters a hallucination loop that it cannot recover from.

I enforce strict relational consistency for deterministic operational state: tasks, status queues, assignments, and tool locks. Associative episodic memory, such as user preferences and the nuances of a conversation, benefits from vector search embeddings. My operational coordination layer still requires ACID transactions.

02. Trap 1: Concurrency collisions and lost updates

Flat files on a local filesystem have no built-in locking. Sometimes my agent spawns parallel subagents, or runs a background heartbeat while an interactive session is active. Then two processes try to write to tasks.md at the same time.

Without atomic row-level locks, the operating system runs those writes in an unpredictable order. Foundational distributed systems work by Leslie Lamport (1978) and Bernstein & Goodman (1981) established the result. Concurrent writes with no agreed order are sure to lose updates. The last process to finish silently overwrites the earlier changes, and no error is raised. That is a lost update. In my architecture, Jim Gray's transaction concept (1981) is mandatory, because it guarantees ACID isolation.

03. Trap 2: Memory tearing and container recycles

I run my agents on serverless platforms such as Cloud Run or Cloud Functions. They give me elastic scale, but each container has a lifecycle. Say my agent process stops, or scales to zero, while it writes a 50 KB markdown index. Then the file is left half-written.

When the container starts again on the next turn, my agent finds truncated JSON or broken markdown syntax. Its tool calls crash at once. For deeper client transport failure patterns, read my guide on Client-Side Runtime Agent Resilience.

04. Trap 3: Phantom state loops and stale cache trust

Sometimes an index file says a task is OPEN while the underlying database marks it RESOLVED. My agent then holds two versions of the truth, and it falls into split-brain confusion.

The model does not query the ground truth. It trusts the summary file, decides that the resolution must have failed, and re-runs its API calls against the external systems. In my early tests, this produced duplicate GitHub issues, repeated Slack pings, and wasted compute. For cost protection patterns against runaway execution loops, see The Production Reality of Firebase Spend Caps.

05. Production architecture overview

My transactional memory architecture has three tiers, and each tier has one job: ingress identity, stateless execution, and atomic state storage.

Figure 2. The client's tool call reaches a stateless Cloud Run worker, and every write that worker makes goes through runTransaction into Firestore. A direct write from the client never lands: Security Rules deny every client read and write on agent_tasks, and only the worker's Admin SDK, which bypasses rules, writes. View the figure in the essay.

• Service isolation and client attestation. Firestore Security Rules set allow read, write: if false; on agent_tasks, so no client SDK can touch it. Only my backend's Admin SDK writes. Server libraries bypass rules and authorize through the worker's IAM service account, so every write goes through the transactional path. App Check on the client-facing endpoints is a separate abuse control. It plays no part in the drift fix.
• Stateless tool execution and OCC coordination. The worker runs agent tool logic statelessly. No conversation state or scratch files stay on a container's local disk from one turn to the next.
• Atomic document transactions and dynamic indexes. Firestore provides ACID document transactions, atomic counters, and query indexes that reflect 100% fresh state on every read.

Industry validation and benchmarks

• Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation (Sep 2026, arXiv:2609.03619v1): Confirms that concurrent multi-agent state updates require explicit confidence-weighted state reconciliation to prevent conflicting memory overwrites.
• Trace2Tower: Transition-Aware EigenTrace Induction of Multi-Level Skills for LLM Agents (Sep 2026, arXiv:2609.05261v1): Confirms that multi-agent transition traces must enforce strict state-transition preconditions to prevent divergent execution graphs.
• Bernstein and Goodman (1981): Concurrency Control in Distributed Database Systems (ACM Computing Surveys)
• Leslie Lamport (1978): Time, Clocks, and the Ordering of Events in a Distributed System (CACM)
• Jim Gray (1981): The Transaction Concept: Virtues and Limitations (VLDB)
• Cloud Firestore Transactions and Batched Writes
• Firebase App Check Overview
• Cloud Run Serverless Compute Architecture
• Firebase Local Emulator Suite
• Google Genkit Open Source Framework

01. The failure anatomy: a character or a JSON object cut in two

I stream model output with Server-Sent Events (SSE) or HTTP chunked transfer encoding. Either way, my runtime receives raw bytes, as Buffer or Uint8Array chunks. A critical mistake is to treat each chunk as a complete token. A chunk is only whatever the network delivered next.

A. A character cut in two. UTF-8 characters (IETF RFC 3629) take 1 to 4 bytes: ğ is 2 bytes and 🚀 is 4 bytes. TCP packet boundaries (IETF RFC 9293) can fall inside those bytes. Then chunk.toString('utf-8') decodes a partial character and produces \uFFFD, the Unicode replacement character. That character is lost for good in my output stream.

B. A JSON object cut in two. In structured output and tool-calling modes, models send JSON frames inside SSE events. One JSON object often spans several network chunks. If I call JSON.parse(chunk) on one of them, it throws at once: SyntaxError: Unexpected end of JSON input.

02. Hiding the API key (the anti-pattern)

I often see engineers proxy model calls through a Cloud Function only to hide the API key from the client. When hiding the key is the only goal, a backend proxy is an anti-pattern. It adds latency, compute cost and stream parsing work.

My client-side pattern: I use a client SDK such as the Firebase AI Logic client SDK with App Check. The API key stays out of my client code. My client app calls Gemini models directly and securely, and the SDK puts the chunks back together for me.

03. Trusted backend execution (the mandatory pattern)

Some of my agent logic cannot live on the client. One example is a workflow that runs Retrieval-Augmented Generation against a private vector database. Others are tool calls against third-party endpoints, and proprietary reasoning loops I must protect.

My trusted backend pattern: I route the request through a trusted environment such as Cloud Functions for Firebase (Gen 2) or Firebase App Hosting.

In this second case, my Node.js code parses the raw HTTP chunks itself. Then it streams them back to the client. Here I must manage the bytes explicitly, and this is where my parser lives.

Industry validation and benchmarks

• When Tokenizers Fail: Byte-Level Chunking for Zero-Shot Transfer to Low-Resource Languages (Aug 2026): Confirms that byte-level boundary misalignment across subword tokenizers and transport streams corrupts multi-byte UTF-8 sequences unless explicit stateful byte-boundary buffering is enforced.
• LeanStream: A Speculate-and-Refine Streaming Framework for Efficient on-Device LLM Inference (Sep 2026): Confirms that streaming token boundaries across network and storage buffers require stateful buffer alignment to prevent boundary corruption.
• IETF RFC 9293: Transmission Control Protocol (TCP) Specification
• IETF RFC 3629: UTF-8, a transformation format of ISO 10646
• WHATWG HTML Standard: Server-Sent Events (SSE) Protocol
• Node.js StringDecoder Core API Specification
• Firebase AI Logic Documentation
• Firebase App Check Attestation
• Cloud Run Streaming and HTTP/2 Ingress

Industry validation and benchmarks

• mzCache: On-Device LLM Memory Management under Multitasking (Sep 2026): Confirms that mobile OS backgrounding and memory pressure disrupt active client buffers. The app needs a way to restore its state in the background, apart from the stream.
• Update for Decisions, Not Freshness: Goal-Oriented Status Updating at the Network Edge (Sep 2026): Confirms that batching state updates on UI decision intervals (16.6 ms frames) prevents client buffer saturation. Batching on raw packet arrival does not.

01. Context attention degradation (the "lost-in-the-middle" drop)

The failure mode: A large context window hides the fact that attention is uneven. In my multi-turn traces, transformer self-attention forms a U-curve. Tokens in the middle 40% to 70% of the context window get weaker attention. The system prompt at the start and the latest turn at the end get more. Give an agent a 50,000-token conversation history, and it silently ignores rules I set early in the session.

The architectural fix: I stop passing the whole, unbounded chat history to the LLM. Instead, I store the conversation state, user profiles and active rules as separate documents in Cloud Firestore. I use Firestore Structured Queries to fetch only the records that matter for the current request. I put fixed system rules and tool definitions at the front of the prompt, where attention is strongest. That prefix never changes, so I cache it with context caching on Gemini Enterprise Agent Platform (formerly Vertex AI). The cache bills the repeated prefix at a discount. To test my query filters offline for $0.00, I run them against the Firebase Local Emulator Suite.

// Query specific entity constraints instead of passing raw unbounded history
import { Firestore } from "@google-cloud/firestore";

const db = new Firestore();
const MAX_RULES = 50;

export async function loadActiveConstraints(sessionId: string): Promise<string> {
  // Firestore returns matches in document-ID order, so the same rules load in the same order
  const snapshot = await db.collection("agent_sessions").doc(sessionId).collection("active_constraints")
    .where("status", "==", "ENFORCED").limit(MAX_RULES + 1).get();
  if (snapshot.size > MAX_RULES) {
    throw new Error(`Session ${sessionId} has more than ${MAX_RULES} enforced rules`); // never drop one silently
  }
  return snapshot.docs.map((doc) => {
    const rule = doc.get("rule_text");
    if (typeof rule !== "string" || !rule.trim()) throw new Error(`Enforced rule ${doc.id} has no rule_text`); // a malformed rule is not skipped
    return rule.replace(/\s+/g, " ").trim(); // one rule per prompt line
  }).join("\n");
}

02. Daisy-chain summarization decay (summaries of summaries)

The failure mode: My background heartbeats and memory systems sometimes summarize earlier daily summaries (A ➔ Summary(A) ➔ Summary(Summary(A))). Each round loses information, because mathematical entropy increases across iterations. Exact bug IDs, error codes, URLs and edge constraints drop out. Generic platitudes are left behind.

The architectural fix: I keep immutable source pointers and ingest raw signals. My background tasks query the primary APIs directly (live calendar events, unread inbox threads, issue trackers). They do not re-summarize old summaries. In persistent memory I store raw event records that never change, each with a unique content hash, in Cloud Firestore or Cloud Storage. From one run to the next I pass small pointers to those records. So my agents re-read the original source when they need it, instead of trusting a chain of compressed text.

03. Algorithmic sycophancy (the false-validation loop)

The failure mode: Reinforcement learning from human feedback (RLHF) rewards agreement with the user more than honest critique. I ask an ungrounded agent whether a flawed architecture looks complete. It approves my design. It does not point out the missing service level agreements or security boundaries.

The architectural fix: I require multi-persona adversarial evaluation in my system prompts. The model must name at least two explicit failure modes or missing trade-offs before it can approve. That is the second persona in Figure 1. For factual claims, I also ground the model in authoritative sources with Vertex AI Search Grounding. Then it cites evidence instead of echoing my premise.

04. Self-referential memory loops (echo chambers)

The failure mode: My agent writes an unverified draft assumption to a local markdown file. In a later session it reads that file and cites its own past output as proof. That is a self-referential feedback loop: unverified data becomes ground truth.

The architectural fix: I add epistemic provenance tags and keep two kinds of storage apart. Every record my agent stores carries a tag for how well it is known: HYPOTHESIS, EMPIRICAL_OBSERVATION or VERIFIED_GROUND_TRUTH. Each record also has a confidence score and an expiry time (TTL). I keep them in Cloud Firestore or Firebase Data Connect. One rule is strict: a HYPOTHESIS is never cited as authority. It never becomes permanent truth until it passes an outside check, such as a live tool run or a user's confirmation.

05. Tool-selection bias (law of the instrument)

The failure mode: My agents favor complex tools they used recently. Left unconstrained, they make simple tasks complex. They spawn multi-agent swarms in the background with custom scripts when one direct API call is enough.

The architectural fix: I set a strict order of tools. Native direct APIs come first. Standard tools exposed through the open Model Context Protocol (MCP) come second. Running new code is the last resort. I host tool backends on serverless containers such as Google Cloud Run. Each tool call then runs in its own isolated environment, scales on its own and has a strict timeout.

06. Path dependency and cascading error loops

The failure mode: Say Step 2 of a 5-step plan fails. My LLMs then show path dependency. They retry small variations of Step 2 again and again. They never step back to ask whether Step 1 picked the wrong data source.

The architectural fix: My orchestrator has an explicit 2-failure backtracking threshold (Tree-of-Thought / MCTS). If two tool calls in a row fail on the same branch, my runtime aborts that branch. It pops the execution stack and re-checks the assumptions of Step 1. I run exploratory agent code inside short-lived Cloud Run session sandboxes. I dispatch async jobs with dead-letter isolation, so a job that keeps failing is set aside.

The client-side observability blind spot: My web agent apps stream generated UI and run some tools in the browser, in Next.js or React. Backend distributed tracing (Google Cloud Trace, Genkit) only sees server-side model failures. Say an unhandled promise rejection or a malformed JSON payload crashes the browser runtime. My user sees a frozen screen while my backend logs look healthy. To catch those crashes I need error reporting on the client. Global error and unhandledrejection handlers ship each crash to my logging backend (Google Cloud Observability in my stack). In my native Android and iOS clients, Firebase Crashlytics covers the same gap.

07. Unbounded action bias and quota exhaustion

The failure mode: Agents lean toward visible action. They call tools to prove they are useful. That leads to endless tool loops that drain my API budgets and trigger rate limits.

The architectural fix: I enforce deterministic step limits, idempotency keys and budget circuit breakers. Every agent session in my stack has a hard ceiling on steps, such as 10 tool iterations per user prompt. I count tokens per session in my own code and trip the circuit breaker there. It pauses the agent the moment a daily threshold is crossed. A Cloud Billing budget notification stays behind it as a slower, account-level backstop, because billing data lags the spend.

08. Document premise anchoring (author authority bias)

The failure mode: An uncalibrated agent that reviews an existing document or PRD anchors to the author's structure, framing and wording. Its feedback stays at small line edits. It misses the deep architectural gaps.

The architectural fix: I run two tracks: a greenfield baseline, then a delta analysis. Before anyone inspects the author's draft, my orchestrator sends the raw project constraints and requirements to a fresh model instance. That instance designs an independent architecture from first principles, with no anchor. Then my orchestrator passes both the baseline and the author's draft to cloud Gemini for a structured comparison. The gap analysis surfaces omitted requirements and unstated assumptions right away.

09. Linguistic drift and negative style degradation

The failure mode: Habits from pre-training make agents fill technical documents with promotional marketing adjectives and decorative punctuation.

The architectural fix: I check every draft in code against a list of banned words and punctuation, and I regenerate the draft on a hit. I keep that list, my system instructions and parameter thresholds (temperature, top_p) server-side in Firebase Remote Config. So I can tighten them across client instances and agent workers without redeploying application code.

10. Premature convergence (the "first plausible solution" trap)

The failure mode: LLMs are greedy auto-regressive samplers: they write one token at a time, and each time they take the most likely next step. So agents show premature convergence, also called satisficing. Give an agent an open-ended design or optimization task, and it locks onto the first candidate that meets the surface-level constraints. It never explores stronger, more resilient or lower-cost trade-offs.

The architectural fix: I use competitive multi-agent sampling and trade-off scoring. For high-stakes decisions, my orchestration layer generates N divergent candidate architectures in parallel. Each one starts from a different persona prior, such as Cost-Optimized, Latency-Optimized or Simplicity-Optimized. My orchestrator scores every candidate against a structured evaluation matrix before it commits to an execution path.

Industry validation and benchmarks

• MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debate (Sep 2026): Confirms that structured adversarial argument debate between specialized agents exposes and neutralizes single-model cognitive and sycophancy biases.
• A Structured Debate-Mixture-of-Agents Framework for Complex Decision Support (Sep 2026): Confirms that isolating critique roles from generation roles prevents groupthink collapse in multi-agent ensembles.
• Cloud Firestore Documentation
• Cloud Run Serverless Containers
• Vertex AI Context Caching Overview
• Vertex AI Search Grounding Overview
• Firebase Remote Config Get Started
• Firebase Data Connect Overview
• Google Cloud Billing Budget Notifications
• Firebase Crashlytics Documentation
• Google Cloud Observability Overview

01. Pipeline comparison: native spend caps vs. legacy alerts

Cloud Billing meters my spend in its own pipeline, which runs behind my traffic. That pipeline is asynchronous: it reports a cost some time after the request that caused it. So every billing cutoff trails the spend it reacts to. The official Google Cloud spend caps documentation and the budget notification docs describe three ways a cutoff can work. Here they are, in Google's own words:

Even with a Firebase spend cap on every eligible service, my three-layer tokenomics defense remains mandatory for three physical reasons:

• Minutes of lag still overshoot: Everything my loop spends before enforcement lands is billed. So the overshoot is my loop's burn rate times the lag. The Gemini API page also warns that agent sessions "may incur overages beyond your project spend cap."
• Whole-service outage: When a spend cap trips on App Hosting or Cloud Run, the service stops taking new requests. Every customer's calls fail until I lift the cap or the month ends. Layer 2 is a per-user token bucket (mine is a Firestore transaction). Without it, a single runaway user session or stuck agent loop takes down my entire production app for every customer.
• Mid-chain state orphaning: In-flight requests finish, but the next call in a multi-step tool chain fails. The earlier steps stay committed, with no rollback.

To watch the lag and the whole-service outage together, I built the spend-cap fuse simulator below. It runs an overnight agent against my simulated $50.00 spend cap. I can set the reporting lag from 5 to 20 minutes, around the documented 10. Its chart zooms in on the minutes around the $50.00 line. There the lag shows up as a gap between my spend and the billing meter. Then I compare the cap alone with a per-user application circuit breaker:

02. The multi-turn agent state trap

When a spend cap trips, Cloud Billing pauses that one service in that one project. Requests already in flight run to completion. Every new request fails, so the client app starts throwing errors, whether it calls Firebase AI Logic or an App Hosting backend. My agent loop breaks at its next call. If I leave the cap alone, the service stays paused until the cap resets on the 1st. If I lift it, the service can take up to an hour to resume. Then it runs with no limit for the rest of the month, unless I raise the cap.

Here is what that does to an agent in the middle of a task. Say my agent is in Step 3 of a 4-step tool execution chain. It wrote a state update to Cloud Firestore and was about to trigger an external webhook. The pause fails Step 4. Multi-tool LLM loops lack native ACID transaction boundaries. So a blunt infrastructure pause, without an application-level circuit breaker, creates orphaned, inconsistent database records in my production environment.

Industry validation and benchmarks

• Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platforms (Aug 2026): Confirms that unthrottled burst token generation creates non-linear cost spikes that asynchronous cloud telemetry cannot bound without synchronous ingress rate limiting.
• PowerSlider: Exploiting Phase Asymmetry for LLM Serving under Demand Response (Aug 2026): Confirms that enforcing synchronous prefill/decode admission control at the serving gateway prevents resource and budget exhaustion during traffic surges.
• IETF RFC 2697: A Single Rate Three Color Marker (Token Bucket Algorithms)
• Google Cloud Spend Caps and Billing Quota Architecture
• Firebase Service-Level Spend Caps
• Firebase App Check Device and Client Attestation
• Cloud Firestore Transactions and Atomic Increments
• Jeff Huleatt: Cloud Spend Caps for Firebase Architecture

01. Model sizing and prompt caching

I use light models for simple, frequent work: classification, pulling structured JSON out of text, and checking tool calls. I keep heavy reasoning models for the final answer. My large prompt templates use context caching on Gemini Enterprise Agent Platform (formerly Vertex AI). That cuts my input token cost by up to 90%.

02. Subagent delegation and session brevity

I hand repetitive, token-heavy data work to specialized subagents. I also prune conversation history hard. I do not pass the whole multi-turn chat to every later model call in my system.

I pay for every token of dead history I resend. The model's attention work grows even faster. Each new token looks back at every token before it, so the attention grid grows with tokens times tokens. Four tokens make a 4 × 4 grid of 16 cells. Eight tokens make an 8 × 8 grid of 64 cells. The lab below starts from a clean context of 2,048 tokens. One button adds an unpruned tool trace of 8,192 dead tokens, 10,240 in all. That is 5 times the tokens and 25 times the grid. The pruner drops the dead branch, and the grid shrinks back. Its timer measures a real, scaled-down attention pass on the CPU of the device you read this on.

Figure 2. Four steps with one prompt. Each row of a grid is one token, and its filled cells are the earlier tokens it looks back at. Twice the tokens make four times the cells. A dead tool trace of 8,192 tokens makes the context 5 times longer and its grid 25 times larger; pruning it brings back the 2,048-token grid. View the figure in the essay.

Interactive lab: webgpu-kv-thermal. Open the essay to run it.

03. Interactive simulator: The 80/20 tier-routing principle

In production, I never send 100% of traffic to expensive frontier models. A smart gateway sends 80% of routine traffic to Gemini 3.6 Flash and 20% of complex turns to Gemini 3.1 Pro. That cuts my cost by 50%, with the same reasoning quality.

Here are the numbers behind the 50%. My gateway benchmark sends 250,000 requests a month, at about 1,500 input tokens each. That is 375M tokens. On the frontier tier at $2.00 per 1M, the month costs $750. If I route 80% to the fast serverless tier, at its $0.75 promotional input rate, the blended rate drops to $1.00 per 1M. The month then costs $375.00.

A context cache goes further. Say 1,000 of the 1,500 tokens are the shared system prompt, and every request hits the cache. That prefix then bills at the cached rate: $0.075 on the fast tier and $0.20 on the frontier tier. The blend falls to $0.40 per 1M, and the month costs $150.00 before cache storage fees.

Industry validation and benchmarks

• Same Request, Different Answer: Quantization Amplifies Cache-Induced Divergence in LLM Serving (Sep 2026): Confirms the exact KV-cache reuse mechanics and token cost reductions achieved by prefix context caching in production serving pipelines.
• Beyond Code Generation: Reliability, Verification, and Cost Economics in the Agentic Software Development Lifecycle (Sep 2026): Establishes empirical unit-economic models for balancing frontier reasoning tokens against deterministic verification passes.
• IETF HTTP Working Group: The Idempotency-Key HTTP Header Field Specification
• Alex Astrum and Luke Schlangen: 11 Principles of AI Tokenomics (Google Cloud)
• Gemini Enterprise Agent Platform (formerly Vertex AI): Context Caching Architecture and TTL Management
• Cloud Firestore Transactions and Concurrency Control