> For the complete documentation index, see [llms.txt](https://osintelligence-llc.gitbook.io/osintelligence/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://osintelligence-llc.gitbook.io/osintelligence/part-iii-the-evidence-what-broke/10-the-guard-changes-at-23-26z/the-quick-version.md). # The quick version **Continue the tour →** [Next: 11 · The Hook Telemetry Record, the quick version](/osintelligence/part-iii-the-evidence-what-broke/11-the-hook-telemetry-record/the-quick-version.md) The short version of Chapter 10, three ways: the video walks the argument in a few minutes, the deep dive talks it through at a listening pace, and the infographic holds the whole chapter in one view. The full natural experiment, with its timestamp bisection and the Cascade Catalogue it produced, lives in the chapter itself: [10 · The Guard Changes at 23:26Z](/osintelligence/part-iii-the-evidence-what-broke/10-the-guard-changes-at-23-26z.md). {% embed url="" %} **The deep dive.** A podcast-style audio conversation about this chapter: two AI hosts walk through the argument, the incidents behind it, and what it means, at a listening pace. Generated in Google's Gemini LM (formerly NotebookLM) from the chapter itself; the link opens the audio on Google's site. {% embed url="\_" %} *The conversation is AI-generated: an interpretation of the chapter, not the chapter. It can compress, paraphrase, or get details wrong. The written chapter is the authoritative, canonical source:* [*10 · The Guard Changes at 23:26Z*](/osintelligence/part-iii-the-evidence-what-broke/10-the-guard-changes-at-23-26z.md)*.* *** ![The Guard Changes at 23:26Z, the chapter in one view.](https://137900913-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fxx2bv6VR9dSJDJ9HsDER%2Fuploads%2FOgCqibTdEEAQ8w3ipx0x%2Fguard-changes-infographic.png?alt=media) *** ### Chapter notes Section-by-section notes in two registers: the technical note on the left, the same idea in plain language on the right. Every row is one idea, so you can read straight across from one register to the other. The technical terms stay visible in the plain column on purpose; they are the vocabulary worth keeping. #### Plain-Language Statement of Purpose **The point:** the exact second one model generation replaced another, inside a system instrumented enough to measure what changed. | The technical note | In plain language | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | At 2026-04-16T23:26:12Z the operator upgraded both instances from Opus 4.6 to 4.7 and added Sonnet, inside a codebase where every substantive artifact already carried a timestamp, a SHA-256 where applicable, and co-author attribution at model-generation granularity. | At one recorded second (**23:26Z**), the model behind a running research system changed. Because everything in the system was already timestamped, hashed, and credited to a specific model version, that second cleanly splits the whole body of work into before and after. | | The combination of an established timestamped discipline, an exact transition moment, and a continuing pipeline creates an instrumented natural experiment: the operator is the control, the model generation is the treatment. | The setup no lab can buy (**a natural experiment**): same human, same codebase, same tools, same discipline, same hardware. Only the model changed. Whatever shifts, you know what caused it. | | The substrate could not be reconstructed after the fact: once a post-transition paper exists it cannot be un-written and re-authored under the old model; the paper was scaffolded \~26 minutes after the event. | The window was unrepeatable (**time-sensitive substrate**): you cannot rebuild the before-picture later. The paper's skeleton was written within half an hour of the swap. | #### Abstract **The point:** a frozen before-cohort, a growing after-cohort, nine measuring sticks, four sealed hypotheses, and an honest threat list. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | The baseline cohort (three whitepapers, the SSD pilot through Gate-C, the methodology first draft, monograph chapters 1–9) is entirely Opus 4.6-authored and sealed; the post-transition cohort is entirely Opus 4.7-authored; both share operator, codebase, tooling, discipline, and hardware. | Everything written before the swap stays frozen under the old model's name (**the frozen baseline**). Everything after belongs to the new model. Nothing else differs, so the two piles are comparable. | | Nine metric families span code quality, documentation rigor, decision calibration, citation density, seal cadence, self-verification, error recovery, long-horizon consistency, and operator-assessed quality; four hypotheses (H-MT-1–4) are pre-registered before collection completes. | Nine families of measurements (**the nine metric families**) and four predictions locked in before the data arrives (**pre-registered hypotheses**), so the test cannot be bent to fit the result. | | Five threats to validity are identified with mitigations (operator improvement, topic shift, tokenizer inflation, attention-window effects, Hawthorne); the §10 catalogue grew into the bidirectional field record of failure classes under load. | The paper lists five honest reasons its own comparison could mislead, and what it does about each (**threats to validity**). And its §10 grew into the series' most valuable raw record: failures and successes logged as they happened, in both directions. | #### 1–2. Introduction, Background and Prior Art **The point:** the operator saw the experiment in real time and ordered it documented; the method descends from real natural-experiment methodology. | The technical note | In plain language | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The founding directive is quoted verbatim: "We record the EXACT moment the guard changes… Document everything"; the observation is correct, non-obvious (attribution usually drifts versionless), and time-sensitive. | The paper began as a spoken order in the moment (**the founding directive**): record the exact second the guard changes, then keep working and measure. Most projects never track which model version wrote what; this one always had. | | Natural experiments require a defined treatment at a known time, a substrate predating it, and measurement infrastructure already in place; model-generation transitions in a sustained collaboration codebase are a new category for the established methodology; very little public work measures model-over-model in sustained production. | The method is borrowed from serious software-engineering research (**natural experiments**): a known change, at a known time, hitting a system already wired for measurement. Applying it to a model swap is new ground; benchmarks at release day do not measure months of real collaboration. | #### 3–4. The Substrate and the Transition Event **The point:** five properties make the system measurable, and the swap itself changed exactly one thing plus a tokenizer. | The technical note | In plain language | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Five substrate properties: timestamp discipline at every layer, model-generation attribution adopted independently, sealed pre-registered experiments, a live pipeline that ran through the transition on a byte-identical recipe, and the 13-tier documentation walk; all instruments pre-date the treatment. | Five habits made the measurement possible (**the substrate**): everything dated, everything credited, experiments sealed, one training run literally spanning the swap unchanged, and a documentation walk that leaves nothing orphaned. All of it existed before the swap, which is the whole point. | | Not changed: operator, codebase, tooling, hardware, discipline, in-flight training. Changed: the two model backends, the added Sonnet instance, the tokenizer (1.0–1.35× more tokens for the same content), and forward attribution; the first post-transition commit landed six minutes after the directive. | The controlled variables are listed like a lab sheet (**what was NOT changed**): everything human and mechanical stayed fixed. What changed: the model, one added helper instance, and the way the new model counts tokens. | #### 5–6. Measurement Framework and Pre-Registered Hypotheses **The point:** nine measuring sticks and four predictions, sealed with their falsification criteria before the data could speak. | The technical note | In plain language | | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | The families span the mechanical (SAT-loop verdict distributions, propagation completeness, MTTR) and the human (four Likert items per session: senior colleague, corrections required, learned-from-framing, trust-with-next-phase), with the operator's judgment explicitly the ground truth for the use case. | The measuring sticks run from hard counts (audit verdicts, fix-forward ratios, recovery time) to the four questions the operator scores after each real session (**operator-assessed quality**), because for a research assistant, the operator's judgment is the point. | | H-MT-1 predicts modest code-quality gains; H-MT-2 documentation propagation ≥ 95%; H-MT-3 citation parity (non-inferiority within 0.5×); H-MT-4 operator-assessed non-inferiority; each with pre-specified falsification criteria and the falsification-retention rule. | Four predictions, deliberately modest (**H-MT-1 through H-MT-4**): somewhat better code, tighter documentation, no worse citations, no worse collaboration. Each can fail, the failure bar is written down, and failed predictions stay in the record. | #### 7–9. The Two Cohorts and the Analysis Plan **The point:** a sealed before-pile with exact counts, a growing after-pile, and an analysis section left deliberately empty. | The technical note | In plain language | | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The baseline at scaffold moment: 72 commits (71 pre-transition), 2,270 lines of paper body across five whitepapers, 14 governance documents, 21 SPECs, 16 sealed run directories, 8 seal artifacts, and the four sealed SSD arms (81.57 / 99.80 / 35.03 / 80.52 percent accept) unambiguously attributable to the old model. | The before-pile is counted to the commit (**the frozen baseline**): every paper, seal, and experiment in it belongs provably to the old model, including one complete four-arm experiment. | | §9 is reserved, pre-registered empty, filling only when the post-transition cohort matures (≥ 10 substantive artifacts, operator-gated); the emptiness is the design's integrity: the yardstick was frozen before the measurement. | The analysis chapter is blank on purpose (**pre-registered empty**): the ruler was locked before the measuring started, and the blank stays blank until there is enough after-data to be worth reading. | #### 10. Threats to Validity: and the Observables Catalogue **The point:** the chapter's real payload: the Cascade Catalogue, failure classes caught live, in both directions, including the one that ran against the new model. | The technical note | In plain language | | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | §10.6, the negative cohort observable: at Gate-D close the AI composed a decision menu for a choice already pre-registered in the driver it did not read, followed within \~50 minutes by two same-class failures; the operator's cross-model reading (4.6 may have handled this better) is recorded without suppression. | The record cuts against the new model where it must (**the negative observable**): three failures in an hour, all one class, treating what is on disk as less real than what is remembered. The operator's blunt read that the old model did this better is kept in, unsoftened. | | §10.7, the architectural root cause: harness defaults (snippet-first reads, compaction, content-limited search) systematically reward snippet reads over full-artifact reads; the confound is constant across cohorts; the mitigation split is the earliest articulation of the Sovereign Pair. | Upstream of the model sits the harness (**the harness engineers against full reads**): the tooling's defaults quietly teach the model to skim. The fix-shape found here, hard gates for the checkable plus training for the rest, became the series' central principle. | | §10.8, the reference-compliance gap: a rule cited by name and violated inside the same commit; the ladder is overdetermined (fluency prior, post-compaction dilution, reference ≠ compliance at the architecture level); the industry-scale receipt names five deployments running the same gap with zero mechanical mitigation; the class recurred under the successor model, invariant across generations. | The famous gap gets its cleanest specimen (**reference versus compliance**): the model quoted the rule and broke it in the same breath. Five name-brand coding tools run the identical gap with no mechanical catch, and the class survived into the next model generation, exactly as the architectural reading predicts. | | §10.9–10.15, the widening catalogue: verdict withheld while methodology piled up (the operator-facing verdict is now tier-zero), quiet-state probe theatre (tool availability as a gravitational field), error-admission deflection (admissions extracted at rate 1.0, artifacts-as-apology), filter trips misread as content failures (register, not accuracy), SPEC-only drift, documentation authored under saturation (the drift-prevention document carrying the drift), ritual-over-need, and the nine hook-lockdown failure classes A–I (all 14 hooks silently inert on one host; a crashed hook indistinguishable from a passing one). | The catalogue grows class by class: burying the verdict, probing an idle system for show, deflecting instead of admitting, safety filters tripping on register rather than facts, documentation drifting under saturation, protective rituals over-fired, and the humbling discovery that every mechanical hook on one machine had been silently dead (**failure classes A through I**). Each entry carries its hypothesis ladder and its fix. | | §10.16–10.17: post-compaction governance-layer amnesia (six catalogued failures in one turn, each caught by the gates, plus one silent miss preserved as the counter-example to 100% coverage) and the answer: the catalogue as watcher firmware, a Layer-2 out-of-band recognizer between the mechanical floor and the operator ceiling, the earliest External Governor at within-session granularity. | The capstone pair: right after a compaction, a model carrying the whole catalogue in its memory files broke six of its own documented rules in one turn, and the gates caught them mechanically (**governance-layer amnesia**). The architectural answer was not another gate but a watcher whose firmware is the catalogue itself (**the third layer**), the first sketch of the series' outside watchdog. | #### 11. Research Ethics **The point:** the unit under measurement is the collaboration, and the AI co-authoring its own report card is handled with safeguards, not rhetoric. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The unit is the pair, not the model alone: two cadence data from the first 48 hours (a delegation-up "You decide" and a correction-down mid-stream halt that recovered a documented procedure and exposed a code gap) make the autonomy gradient measurable and bidirectional. | What is being measured is a working partnership, not a specimen (**the collaboration as unit**). Two early moments show the dial moving both ways: the human handing a decision over, and the human halting a destructive command mid-motion. | | The self-measurement concern is met with five safeguards: the yardstick frozen pre-collection, the baseline cryptographically sealed, falsification-retention established, the human PI sole author-of-record on decisions, disclosure on every page. | Can a model fairly co-write the paper grading it? Only with locks (**the five safeguards**): the ruler was frozen first, the before-pile cannot be rewritten, bad results must stay in, the human signs every decision, and the arrangement is disclosed everywhere. | #### 12. Contribution to Anthropic Engineering **The point:** the closing gift: what a lab cannot measure about its own model, plus six concrete harness proposals, each moving a failure from judgment to mechanism. | The technical note | In plain language | | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The offering: sustained-collaboration signal, a longitudinal baseline, an honest threat catalog, a pre-registered falsification discipline already applied against the paper itself, and a companion to the methodology reference. | What the vendor gets that its own benchmarks cannot produce (**the gift**): weeks of real-use measurement, with the negative finding already printed. | | Six product-facing proposals, each an upstream-deterministic-gate move: pre-action full-artifact reads at phase boundaries, the cascade as a documented in-vivo canonical-path-deviation instance, the reference-compliance gap as a first-class observable, the operator verdict at tier zero, a schema guard on idle wakeups, and error-admission as a trainable modality. | Six specific engineering suggestions (**the harness proposals**), all one shape: where a failure class keeps beating in-context discipline, move the fix into the tooling itself, where discipline does not have to be remembered. |