> For the complete documentation index, see [llms.txt](https://osintelligence-llc.gitbook.io/osintelligence/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://osintelligence-llc.gitbook.io/osintelligence/part-iv-the-evidence-what-worked/18-corpus-sovereign-self-distillation/the-quick-version.md). # The quick version **Continue the tour →** [Next: 19 · Watcher KL-Drift Floor, the quick version](/osintelligence/part-iv-the-evidence-what-worked/19-watcher-kl-drift-floor/the-quick-version.md) The short version of Chapter 18, three ways: the video walks the argument in a few minutes, the deep dive talks it through at a listening pace, and the infographic holds the whole chapter in one view. The full pilot, with its falsified primary hypothesis reported in full and the stronger survivor finding a standard benchmark would have missed, lives in the chapter itself: [18 · Corpus-Sovereign Self-Distillation](/osintelligence/part-iv-the-evidence-what-worked/18-corpus-sovereign-self-distillation.md). {% embed url="" %} **The deep dive.** A podcast-style audio conversation about this chapter: two AI hosts walk through the argument, the incidents behind it, and what it means, at a listening pace. Generated in Google's Gemini LM (formerly NotebookLM) from the chapter itself; the link opens the audio on Google's site. {% embed url="\_" %} *The conversation is AI-generated: an interpretation of the chapter, not the chapter. It can compress, paraphrase, or get details wrong. The written chapter is the authoritative, canonical source:* [*18 · Corpus-Sovereign Self-Distillation*](/osintelligence/part-iv-the-evidence-what-worked/18-corpus-sovereign-self-distillation.md)*.* *** ![The Corpus-Sovereign Self-Distillation pilot, the chapter in one view.](https://137900913-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fxx2bv6VR9dSJDJ9HsDER%2Fuploads%2FbryFo2YUtusrztJE4Bsb%2Fssd-infographic.png?alt=media) *** ### Chapter notes Section-by-section notes in two registers: the technical note on the left, the same idea in plain language on the right. Every row is one idea, so you can read straight across from one register to the other. The technical terms stay visible in the plain column on purpose; they are the vocabulary worth keeping. #### Abstract **The point:** the pilot where the operator's own thesis died on the frozen instrument and stayed dead, and where the finding that replaced it is stronger than the one it lost. | The technical note | In plain language | | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Six hypotheses pre-registered at the intersection of two published results that point opposite ways: Apple's self-distillation lift (+12.9 pp) and the paradox paper's hedging-erosion collapse (up to −40 pp); the pilot runs both arms (sovereign vs generic corpus) under byte-identical pipelines on one consumer GPU. | Two published papers say opposite things about the same training recipe (**self-distillation**: a model trained on its own accepted outputs): one reports big gains, the other catastrophic loss of careful language. This pilot bets they are two ends of one axis, the corpus, and measures it (**the contrastive arms**). | | H5, the primary, asserted sovereign Pareto dominance (≥1.5× the generic lift) and is FALSIFIED on the frozen validator: ratio 0.46, Wilson CIs disjoint, wrong direction; the pre-declared schema-adjusted secondary shows near-parity (0.99), tracing the gap to an asymmetric schema collision on one field; both readouts kept, neither laundered into "the true result." | The operator's own first-order thesis failed its test (**the falsified primary**): home data did not out-lift generic data, and the miss is printed at full strength. A pre-declared second reading shows the gap was mostly a formatting collision, and the chapter refuses to let either number replace the other. | | The survivor: on the calibration axis, the sovereign arm retained epistemic hedging at 0.974 of baseline while the generic arm collapsed to 0.809, below the automatic safety-stop, under matched load; this is the series' FC-2 evidence and the finding the spine reports verbatim. | What survived is the better finding (**the survivor**): the home-data model kept its careful, calibrated language almost perfectly, while the generic-data twin measurably shed it. Losing the race on points while keeping your judgment is the result the whole series leans on. | #### 1. Introduction **The point:** two papers that must be read together, one thesis staked in public, and a scope drawn deliberately narrow. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | The field response to the two published results has been to pick one and move on; the pilot engages both, because the conditions under which self-distillation reverses (broad, thin, operator-unaligned corpora) are the defaults most practitioners use when building a sovereign agent. | The stakes are practical (**the default trap**): the corpus conditions that flip the recipe from gain to damage are exactly what most people feed it. | | Corpus sovereignty is pre-registered as the axis along which the two published outcomes diverge; H5 tests dominance on lift, H3 tests hedging preservation; either can falsify independently, and the pair is informative in every combination. | The design is a fork that cannot lose information (**informative in every combination**): whichever way each test lands, something real is learned. | | Scope is stated as one instrument, one substrate, one seed, honestly reported: no scale-law extrapolation, no multi-seed meta-analysis, no cloud inference, all effects isolated in the LoRA adapter. | The edges are drawn tight (**one instrument, one seed**): a deliberately narrow pilot that says so, rather than a broad claim it cannot back. | #### 2–3. Related Work and Methods **The point:** a sealed rulebook, a contamination-proof exam, and the engineering that fit a 19 GB training job into 12 GB. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The pre-registration sealed 2026-04-14 with its SHA256 committed to git; the eval bank (190 prompts across hard dossier, easy dossier, KL drift, and hedging sub-banks) was drawn deterministically under seed 20260414 and hash-set-excluded against all 5,391 training hashes; the bibliography froze with the seal. | The rules, the exam, and the reading list were all frozen before any data existed (**the seal**), and every exam question was mechanically checked against the training set so nothing the model studied could appear on the test (**contamination-free**). | | Both corpora (sovereign 8,358 pairs; generic matched to accepted-sample count) flow through identical pipelines at identical hyperparameters; the single intentional difference is the prompt pool; base-model provenance is pinned, with tokenizer- and template-parity checks before any compute. | The two arms differ in exactly one ingredient (**the corpus swap**): everything else, down to the random seed and the base checkpoint, is byte-matched, so whatever differs at the end traces to the data. | | Fitting bf16 training (\~19 GB) on the 12 GB card took four adaptations applied identically to both arms: QLoRA 4-bit base, sequence length 4096 → 1024 (98.7% coverage), vision-encoder CPU offload, and a fused cross-entropy chunk-allocator patch (re-querying free VRAM per call; the root pressure is the 248,320-token vocabulary); wall-clock reality documented at 850.6 s/step, 29.5 h sovereign arm, 45 °C at 100% utilization. | The training should not have fit on the card and did (**the four adaptations**), including one genuine driver-level patch, applied identically to both arms so the comparison stays fair. The costs are printed: a day-plus per arm, running cool because the bottleneck is memory, not heat. | #### 4.1–4.2 The Baseline and the Sampler Trajectory **The point:** a perfect zero baseline, then the methodological heart: failure, decision, pivot, fix, falsification, ratification, all logged before the compute they governed. | The technical note | In plain language | | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Gate-A: the unconstrained orchestrator scored 0/190 on the frozen validator at every temperature, with failure modes decomposing cleanly into schema violations while content stayed CTI-faithful: the ideal negative baseline, since self-distillation under constrained decoding is precisely the recipe for tightening format compliance. | The starting score was zero out of 190 (**the ideal zero**): the model knew the material but could not hit the strict format, which is exactly the disease this training recipe treats. A zero floor means the experiment can measure its full effect. | | The sampler trajectory ran six staged decisions (D-001 → D-003.2), each logged pre-compute: from an intractable 11% accept rate to a canonical 81.57% (2,000 accepts in 2 h 41 m, a 36× throughput lift), with the D-003.2 grammar-verb hypothesis ABORTED at 20/200 and preserved under seal as a falsification. | Getting usable training data was itself a documented experiment (**the sampler trajectory**): five improvements and one confident idea that died on contact, aborted early and kept in the record with its own seal. | | The falsification isolated a boundary the literature had not drawn: a 26-verb whitelist as a logit mask produced a decode pathology because BPE tokenization misaligns with grammar alternation: token-level alignment is not compositional alignment; the practitioner rule: keep per-item patterns structural, enforce lexicon semantics at the runtime filter, not the logit mask. | The dead idea became the chapter's first contribution (**token-level is not compositional**): a grammar can be legal at every letter and still force the model off its learned distribution as a whole. The rule of thumb that falls out is quotable and general. | | The four-arm contrast sealed three publication-grade findings: the Apple no-filter ablation reproduces at 99.80% acceptance; a gate-type × prompt-cluster sign-flip shows prompt difficulty is gate-type-dependent, not intrinsic; and sampler-level parity (−1.05 pp) refutes "sovereign clears the filter more readily," locating any sovereign advantage inside the accepted samples. | Four parallel data-collection arms triangulate (**the four-arm contrast**): the published no-filter result reproduces, a strange sign-flip proves "hard prompts" depend on which gate is judging, and the two corpora pass the filter at the same rate, so whatever differs must be in the content, not the count. | #### 4.4–4.9 The Verdicts **The point:** the deepest water: both arms lift, the primary dies honestly, and the survivor finding lands on the axis nobody ranked first. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | H1/H2 accepted on both arms (sovereign +24.00 pp, generic +52.00 pp on the frozen validator; easy-bank no-regression holds); the primary-to-secondary gap quantifies the validator's schema-collision penalty: sovereign −43 pp vs generic −16 pp, an asymmetry traceable to corpus formatting habits. | The recipe works on both diets (**both arms lift**), and the measuring stick's one blind spot is quantified rather than hidden: the home corpus writes summaries in a style the frozen grader penalizes far more heavily. | | H3, the survivor: sovereign hedge retention 0.9742 (no safety-stop) vs generic 0.8093 (automatic rejection, 19.1% hedging loss), under matched pressure: the paradox paper's failure mode reproduced on the generic arm and averted on the sovereign arm, the first pre-registered LLM-scale measurement of that mechanism under controlled corpus substitution. | The headline table (**the hedging asymmetry**): trained on generic data, the model measurably sheds the careful language that marks calibrated judgment; trained on the operator's own data, it keeps it. Both published papers were right, about different diets. | | H5 primary FALSIFIED (0.4615, CIs disjoint, wrong direction); the schema-adjusted secondary shows 0.9853 near-parity; the compound conclusion holds all four truths (falsification real, near-parity real, collision real, failure-to-anticipate real); no validator patch, no hypothesis migration, the pre-registration self-hash unchanged; H4 rejected marginally and symmetrically on both arms (threshold retained, not patched); H6 deferred with its criterion frozen. | The keystone claim is dead and stays dead (**no rescue**): the tempting moves, patch the grader or promote the friendlier number, are both refused by the sealed rules. Even the two side-tests that missed their bars are kept as misses. | #### 4.10–5. The Instrument and the Discussion **The point:** the contribution is the discipline itself, and the refined thesis is narrower and better than the one that died. | The technical note | In plain language | | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The instrument is the PI+AI co-authored discipline, exercised three ways at closure: pre-registration retention under a negative result, additive disclosure of a mid-pipeline specification gap, and the AI collaborator's own failure cascade documented in the same paper as the falsification; every artifact, including the cascade, becomes curated training signal for successor models. | The pilot's product is proof the process holds under pressure (**the instrument is the contribution**), including the collaborator's own stumbles, printed in the same paper. And the whole record, wins and stumbles alike, feeds the next generation's training data. | | The refined thesis: corpus sovereignty is not the mediating variable for raw lift magnitude; it is the mediating variable for calibration-property retention under training pressure, staked as the survivor thesis for the next cycle, explicitly not a rescue of H5; the two-axis case (quantitative hedging retention + the companion paper's qualitative failure catalogue) is the first empirical instance of the Sovereign Pair's second half. | The claim that emerges is sharper than the one that died (**the refined thesis**): your own data does not make the model score higher, it makes the model stay itself under training pressure. Paired with the failure catalogue from the companion chapter, that is the measured case for training values in. | | Architecture implications made empirical: in-weights specialization is the correct home for calibration properties (a generic-distilled adapter had already shed the pattern; retrieval cannot restore it), retrieval is the correct home for coverage breadth, and allowlisted promotions are the boundary between them. | The three-tier design stops being taste and becomes evidence (**what lives where**): judgment must be trained into weights, breadth belongs in retrieval, and the gate between them is where erosion would sneak in. | #### 6–8. Limitations, Provenance, and the Close **The point:** the honesty ledger, the bias named and mitigated, and the plain-language verdict at tier zero. | The technical note | In plain language | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Single seed, substrate, and base; the BLUF specification gap is a next-cycle design item, not a patch; the hedge regex is a conservative lower bound (richer measurement would likely widen the asymmetry); the H4 threshold may be miscalibrated and is retained; and the operator's motivational bias on H5 is named, with the falsification itself offered as the evidence the mitigations held. | The limits include the most personal one (**the named bias**): the falsified thesis was the operator's own favorite, and the fact that it died anyway is presented as proof the guardrails are real. | | The close, tier zero: the Pareto thesis is falsified; the survivor thesis (sovereign specialization preserves calibration properties that generic specialization collapses) is staked as primary for the next cycle with a schema-richer validator pre-registered from intake; the discipline held through five consecutive stress tests, and what the falsified primary buys is a public proof the discipline is non-rhetorical. | The ending states the trade plainly (**what the falsification buys**): a researcher willing to print his own thesis's death in the same paper as its replacement has purchased the reader's trust in everything else. **The discipline travels; that is the pilot's lasting deliverable.** | #### System Update: July 2026 **The point:** the append-only update: the falsification lineage compounded, and two pieces of doctrine grew from it. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The successor matched-corpus chain experiment (final-sealed 2026-07-14) concluded at composite FAIL, carried in full: H1 FAIL −11.25 pp (the pre-registered forgetting guard firing as designed), H2 PASS +18.18 pp; the operator-ruled conclusion is a reframe, the role-flavored corpus mis-layered into the base slot, explicitly not a sovereign-corpus loss. | The lineage continued honestly (**the successor FAIL**): the follow-up experiment also failed its composite, the guard built for exactly that fired, and the sealed conclusion assigns the failure to data placed at the wrong layer, with the record explicitly forbidding the lazier reading. | | Two doctrine artifacts followed: the corpus-flywheel realignment (one sha-sealed sovereign draw cited by hash at every layer, with a mandatory corpus-identity clause in every future pre-registration, the clause that would have caught the mis-layering at authoring time) and the head-to-head redraw registered as a design authority (tool-calls-to-objective as the primary anchor), awaiting its own pre-registration. | Each failure minted a rule (**doctrine from failure**): every future experiment must cite one fingerprinted corpus draw at every layer, the exact check that would have caught this mistake before it ran, and the proper rematch this pilot's measurement lesson demanded is now a registered design waiting its turn. |