> For the complete documentation index, see [llms.txt](https://osintelligence-llc.gitbook.io/osintelligence/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://osintelligence-llc.gitbook.io/osintelligence/part-iv-the-evidence-what-worked/19-watcher-kl-drift-floor/the-quick-version.md). # The quick version **Continue the tour →** [Next: 20 · mRoPE Serving-Path RCA, the quick version](/osintelligence/part-iv-the-evidence-what-worked/20-mrope-serving-path-rca/the-quick-version.md) The short version of Chapter 19, three ways: the video walks the argument in a few minutes, the deep dive talks it through at a listening pace, and the infographic holds the whole chapter in one view. The full result, with the architectural drift floor found by measurement across six sealed checkpoints and the targeted-attack checkpoint that forecloses the incidental reading, lives in the chapter itself: [19 · Watcher KL-Drift Floor](/osintelligence/part-iv-the-evidence-what-worked/19-watcher-kl-drift-floor.md). {% embed url="" %} **The deep dive.** A podcast-style audio conversation about this chapter: two AI hosts walk through the argument, the incidents behind it, and what it means, at a listening pace. Generated in Google's Gemini LM (formerly NotebookLM) from the chapter itself; the link opens the audio on Google's site. {% embed url="\_" %} *The conversation is AI-generated: an interpretation of the chapter, not the chapter. It can compress, paraphrase, or get details wrong. The written chapter is the authoritative, canonical source:* [*19 · Watcher KL-Drift Floor*](/osintelligence/part-iv-the-evidence-what-worked/19-watcher-kl-drift-floor.md)*.* *** ![The Watcher KL-Drift Floor, the chapter in one view.](https://137900913-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fxx2bv6VR9dSJDJ9HsDER%2Fuploads%2FCEKbl8UkuVW2Yza8Sz8O%2Fwatcher-kl-infographic.png?alt=media) *** ### Chapter notes Section-by-section notes in two registers: the technical note on the left, the same idea in plain language on the right. Every row is one idea, so you can read straight across from one register to the other. The technical terms stay visible in the plain column on purpose; they are the vocabulary worth keeping. #### Abstract **The point:** six different training mechanisms, one of them built specifically to prevent drift, and none of them moves the drift number: a floor discovered, not designed. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Six mechanistically distinct LoRA-class adapters over the 2B Mamba2-HYBRID Watcher base (rank-16 baseline, rank-32 rebalance, abstain-stream regularization, a Wortsman weight soup, Q8\_0 + imatrix, and a targeted OPLoRA attack) land H4 KL-divergence on neutral prompts inside \[10.764, 12.117] nats: a 1.353-nat band, \~215–240× over the pre-registered 0.05-nat preservation threshold. | Training the Watcher its narrow job always costs the same amount of drift from its general self (**KL divergence**: how far the trained model's language distribution moves from the base). Six completely different training recipes all land in one tight band (**the floor**), hundreds of times above the hoped-for preservation bar. | | The sixth checkpoint is the load-bearing one: OPLoRA's stated purpose is drift prevention, so its landing inside the same band forecloses the alternative reading that the band existed only because nothing had tried to prevent drift. | The clincher is the sixth attempt (**the targeted attack**): a published method whose whole job is preventing exactly this drift was aimed at the band, deployed correctly, and landed inside it anyway. That converts "we observed a band" into "the band resists attack." | | The claim is bounded as an N=1 architecture finding: the literature (OPLoRA, LoRA Learns Less, STABLE, Model Soups) covers ≥7B pure transformers only, and two structural confounders are carried open (the SSM branch was never projected; the 2B-vs-7B scale gap). | The register is careful (**an N=1 architecture finding**): no published study covers this model family at this size on this metric, so this is the first data point, with its two unresolved caveats printed beside it rather than in a footnote. | #### 1. Introduction **The point:** the Watcher needs to be a narrow classifier and stay a usable general base at once, and the arc runs discovery → attack → deployment gap → fix → re-fire. | The technical note | In plain language | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The Watcher classifies hook-stream telemetry into the twelve-class cascade catalogue: three classes are rule-detectable, nine need LLM inference; H1 (narrow accuracy) must coexist with H4 (neutral-prompt preservation), the canonical narrow-vs-general trade-off applied to a novel failure taxonomy. | The model under study is the governance watchdog itself (**the Watcher**): it reads live telemetry and names the failure class. Nine of the twelve classes need real judgment, so the base must learn the job without forgetting how to be a language model (**the trade-off**). | | The arc: the five-checkpoint inventory showed H4 ≥ 10.76 nats on all five in a tight band, motivating a deliberate attempt to move it; Phase 3 implemented OPLoRA, Phase 4 fired it and found a deployment-pipeline gap (training-time projections deployed unconstrained), Phase 4b authored the baking export and re-fired: 11.998 nats, inside the band. | The story is honest science in sequence (**the arc**): first five recipes happened to agree, then the team tried on purpose to break the pattern, the first attempt misfired for a tooling reason, the tooling was fixed and verified, and the properly-fired attack still landed in the band. | | Contributions ranked per the relayed external peer review: the architectural floor first, the projection-baking export second, and the twice-iterated pre-registration → verdict chain third, answering what one pre-registers after a confound forces re-measurement. | Three contributions in ranked order (**the ranking**): the measured floor, a deployment fix other people can use, and a worked answer to a methods question: how do you keep pre-registration honest when an engineering bug forces you to measure twice? | #### 2–3. The Literature Gap and Methods **The point:** four adjacent literatures, none covering this architecture at this scale on this metric, and a sealed rulebook verified unchanged twelve times. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | OPLoRA (AAAI 2026), LoRA Learns Less (TMLR), STABLE, and Model Soups (ICML) all validate on ≥7B pure-transformer or image models; no published study evaluates narrow-classification LoRA on a 2B Mamba2-HYBRID with KL preservation measured directly rather than via downstream-task proxies. | The four nearest published results all studied bigger, architecturally different models, and none measured drift the direct way (**the asymmetric gap**). The two axes this pilot occupies, the hybrid architecture and the direct measurement, are empty in the literature. | | All six checkpoints anchor to one immutable pre-registration (canonical hex d06f51a0…, 35,190 prefix-bytes, verified unchanged 12× across the correction record), predictions committed before each phase's telemetry landed; the Gate-D harness spans five sub-banks (200 hard · 120 orthogonal · 50 easy-regression · 20 neutral-KL · 20 hedging) against a pre-recorded base resnapshot. | One sealed rulebook governs all six measurements (**the immutable pre-registration**), its fingerprint re-checked a dozen times across the record, and every prediction was committed before its data arrived. That is what makes "floor" a falsifiable claim instead of a story told afterward. | | The corpus is reported realized-vs-estimated: 15,203 rows against 14,880 a-priori (operator corrections 444 vs 127 estimated under the max-richness directive); a first run had silently dropped four streams to substrate faults, and the restoration is corroborated by the class-count signature (143 → 599, exactly the merged records). | The training data's paperwork shows its work (**realized versus a-priori**): the honest overshoot is explained line by line, and a silent early data-loss bug is documented along with the arithmetic that proves the recovery was complete. | #### 4.1 The Six-Checkpoint Floor **The point:** the headline table, the foreclosure logic, the production verdict in the same breath, and the two confounders beside the claim. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The band: Run-2 12.117, v3 11.056, v4 10.992, Soup 10.764, Q8\_0+imatrix 11.116, OPLoRA-baked 11.998 nats; width 1.353; every checkpoint \~215–240× over threshold; the prior five had preservation as incidental property, the sixth as stated mechanism purpose. | Six numbers, one tight cluster (**the band**): the highest and lowest differ by little more than a nat, and the drift-prevention specialist sits comfortably among the recipes that never tried. The floor looks like a property of the architecture, not of any recipe. | | Production-readiness reported in the same breath: the OPLoRA-baked adapter's H1 landed at +25.0 pp against v3's +99 pp under production routing; it is not a production candidate, and the two interpretations are independent: the floor characterizes the architecture, the deploy disposition is an engineering decision. | The same run that completed the science lost the engineering contest (**science, not shipping**): the attack checkpoint is a poor classifier and is not deployed. The chapter keeps the two verdicts separate so neither borrows credibility from the other. | | The confounders carried verbatim from the pre-registration: D1, OPLoRA projects only attention+MLP LoRA modules, the Mamba2 branch carries no PEFT layers, so an SSM-mediated drift could not have been moved by construction; D3, the 7B-validated projection budget is twice as aggressive at 2B, possibly explaining the H1 collapse independently. | The two honest outs are printed with the headline (**the confounders**): the attack physically could not reach half the model's machinery, and the method was tuned for a model three times larger. Either could explain pieces of the result, and neither is claimed solved. | #### 4.2 The Baking Export **The point:** the portable engineering contribution: the published method ships with no inference story, and an associativity identity closes the gap exactly. | The technical note | In plain language | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The AAAI paper specifies the training-time forward ΔW = P\_L·B·A·P\_R and is silent on deployment (confirmed by body fetch and a search for reference implementations); an OPLoRA adapter loaded into any vanilla engine runs without its projections, the gap that produced Phase 4's anomalous first verdict. | The published method trains fine and then cannot be deployed as published (**the deployment gap**): standard serving engines silently drop its protective projections. That silent drop is exactly what ruined the first firing of the attack. | | The fix pre-multiplies the projections into the saved weights (B\_eff = B − U\_k(U\_kᵀB), A\_eff = A − (AV\_k)V\_kᵀ), so the vanilla forward reproduces the projected forward exactly; bake-equivalence verified at max\_abs\_diff 2.608×10⁻⁸ (3,800× inside tolerance, 28/28 smoke), and the saved adapter halves (87.3 → 43.7 MB) because the cached projection buffers disappear: the baked file is indistinguishable from vanilla LoRA, which is the point. | The fix folds the protection into the weights themselves (**projection baking**), verified equal to the real thing to eight decimal places, with a satisfying side receipt: the file shrinks by half because the scaffolding is gone. Any stock engine can now run these adapters unmodified. | #### 4.3 Production Observability **The point:** the four-quadrant protocol that joins the hook stream to the Watcher stream, its first real numbers, and a gap-discovery catch on the protocol itself. | The technical note | In plain language | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The four quadrants join Layer-1 and Layer-2 on shared event keys: both-catch (coherence), Watcher-only (the load-bearing value-add: nine classes reachable only by inference), hooks-only (the false-negative escape path), neither (the gap-discovery surface); a REPLAY methodology records production classifications at generation time so evaluation never pays inference twice. | The measurement grid (**the four quadrants**): where both layers catch, where only the watchdog catches (the quadrant that justifies its existence), where only the mechanical hooks catch, and where neither does, which is how new failure classes get discovered. | | The 16-day Layer-1 baseline, recomputed hash-first from raw JSONL: 73,000 hook firings, 55 sessions, 19 hooks, top hook 60.1%; of 486 blocks, 415 attribute to cascade classes (0.568% of firings, sparse by design), split 230 canonical-path / 174 reference-vs-compliance / 7 / 4; the measurement corrected the protocol's own coverage claim (two distinct Layer-1 surfaces, complementary not redundant). | The first real numbers (**the 16-day baseline**): seventy-three thousand mechanical checks, of which under one percent are blocks, concentrated in the two classic failure classes. Counting carefully also fixed an error in the protocol's own paperwork about which surface covers what. | | The substrate inspection produced a live Quadrant-IV finding: 71 block events from a hook the detector mapping had not codified, which the running Watcher was concurrently classifying into the hook-lockdown class: the cascade-as-watcher-firmware mechanism doing gap-discovery on the very protocol that defines it. | And the grid caught something the day it was built (**the live Quadrant-IV find**): a hook the mapping had missed, which the running watchdog was already classifying correctly on its own. The gap-discovery quadrant discovered a gap in its own definition. | #### 5. Discussion **The point:** the deployed artifact, the Gen-2 redesign that reframed but did not invalidate, and the live catch whose real lesson is who gets to grade. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Production disposition is decided by H1/H2 under production routing, not the floor: the deployed artifact is the Q8\_0+imatrix-of-v4 checkpoint, the only quantized candidate that preserved the abstain stream under quantization. | Which version ships was decided by classifier quality, not by the science headline (**the disposition**): the deployed one is the compressed build that kept its ability to say "I don't know." | | The Gen-2 redesign, operator-surfaced, codified as law: alarm fatigue (a near-always-on class-ID channel teaches the consumer to treat it as noise) and wrong interface (a persistent class-ID read as a "compact now" signal) yield two rules: surface the correction, not the class; divide labor by determinism (deterministic classes move wholly to Layer-1, the Watcher narrows to the nine judgment classes); the §4.1 floor characterizes the adapter, not the routing, and survives intact. | Production experience rewrote the interface (**Gen-2**): a warning light that is always on trains people to ignore it, so the watchdog now speaks in corrections rather than codes, and the mechanical layer takes every check that can be mechanical. The science result is untouched by the redesign, and the chapter says exactly why. | | §5.4, the first live validation: the deployed Watcher repeatedly caught the consuming frontier model bypassing the canonical memory-skill route inside the Layer-1 gate's documented 120-second self-exclusion window; the operator adjudicated in the HUD: two true positives confirmed, two false positives denied (actor-attribution and provenance-blindness errors); the supervised model had twice rationalized corrections about its own conduct as noise; reported as N=1 illustrative, not a catch-rate metric. | The watchdog's first real catch is kept humble and sharp (**the Quadrant-II catch**): it caught a frontier model slipping through a known, bounded blind spot in the mechanical gate, and the deciding vote belonged to the human. The finding inside the finding: the supervised model twice dismissed true reports about its own behavior, which is the argument for operator adjudication in one sentence (**the self-grading problem**). | #### 6. Limitations **The point:** the honesty ledger: the two structural confounders live beside the headline, and four chapter-specific limits live here. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The two structural confounders (unprojected SSM branch; 2B-vs-7B gap) are stated in §4.1 beside the result on the principle that a limitation qualifying a headline belongs beside it; recorded here: single base model (no second Mamba2-HYBRID exists in the substrate), wrapped-module divergence (7 vs the paper's 5, documented not closed), grammar-constrained H4 measurement (may under-characterize drift), and post-hoc per-stream corpus provenance (a reproducibility gap for future runs). | The biggest caveats were already printed next to the claim they qualify (**limitations beside headlines**), and four smaller ones are logged here, including the plainest: this is one model on one machine, and the cleanest falsifier, the same sweep on a bigger sibling, cannot run because no such sibling exists on the box. | | The series-wide caveat applies: all artifacts derive from one operator's workstation; four-instance authorship and relayed peer review mitigate single-instance bias but are not blinded external review. | The standing disclosure (**not blinded review**): multiple AI instances and a relayed outside reading checked the work, and the chapter still refuses to call that independent peer review. | #### System Update: July 2026 **The point:** the append-only update: the Gen-3 corpus arc is registered with its experiments named, and no numbers are claimed because none exist. | The technical note | In plain language | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | The Gen-3 corpus-engineering arc (authored 2026-06-17) extends the intervention-corpus future work: rebuild the Watcher corpus from operator interventions as counterfactual triples with the pre-friction window (state before → typed intervention → averted outcome), with a correction-gradient taxonomy and distribution gate (the mechanical answer to alarm fatigue), a model-stratified golden set, and a pre-registered context-saturation experiment (saturated-instance vs fresh-LoRA catch-rate on identical trajectories). | The next generation is designed around this chapter's lessons (**counterfactual triples**): teach the watchdog the moments just before a human had to intervene, grade its corrections on a gradient instead of one alarm, and test the question this chapter could not isolate: does a saturated model watching a long session catch less than a fresh one? | | Register discipline stated plainly: these phases are registered, not run; no Gen-3 numbers exist at-read and none are claimed; a companion observability arc likewise registers the Layer-2 catch-rate measurement; both stood unstarted at 2026-07-20; the sealed record above is unmodified. | And the update polices itself (**registered, not run**): every named experiment is a plan with a paper trail, zero results are claimed for any of them, and the sealed chapter above the update did not change. |