> For the complete documentation index, see [llms.txt](https://osintelligence-llc.gitbook.io/osintelligence/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://osintelligence-llc.gitbook.io/osintelligence/part-iv-the-evidence-what-worked/13-sovereign-ioc-classifier/the-quick-version.md). # The quick version The chapter in minutes: the video explainer, the audio deep dive, the one-view infographic, chapter notes, and a self-test quiz on the Sovereign IOC Classifier. **Continue the tour →** [Next: 14 · Sovereign CTI-NER, the quick version](/osintelligence/part-iv-the-evidence-what-worked/14-sovereign-cti-ner/the-quick-version.md) The short version of Chapter 13, three ways: the video walks the argument in a few minutes, the deep dive talks it through at a listening pace, and the infographic holds the whole chapter in one view. The full result, with its sealed pre-registration and the first deployable proof of in-weights specialization at micro-agent scale, lives in the chapter itself: [13 · Sovereign IOC Classifier](/osintelligence/part-iv-the-evidence-what-worked/13-sovereign-ioc-classifier.md). {% embed url="" %} **The deep dive.** A podcast-style audio conversation about this chapter: two AI hosts walk through the argument, the incidents behind it, and what it means, at a listening pace. Generated in Google's Gemini LM (formerly NotebookLM) from the chapter itself; the link opens the audio on Google's site. {% embed url="\_" %} *The conversation is AI-generated: an interpretation of the chapter, not the chapter. It can compress, paraphrase, or get details wrong. The written chapter is the authoritative, canonical source:* [*13 · Sovereign IOC Classifier*](/osintelligence/part-iv-the-evidence-what-worked/13-sovereign-ioc-classifier.md)*.* *** ![The Sovereign IOC Classifier, the chapter in one view.](https://137900913-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fxx2bv6VR9dSJDJ9HsDER%2Fuploads%2F7cgz1kAox6Bm9N3Kc84V%2Fioc-infographic.png?alt=media) *** ### Chapter notes Section-by-section notes in two registers: the technical note on the left, the same idea in plain language on the right. Every row is one idea, so you can read straight across from one register to the other. The technical terms stay visible in the plain column on purpose; they are the vocabulary worth keeping. #### Abstract **The point:** the first specialist the stack ever trained: four minutes on one consumer GPU, a 56-row corpus, and a probe the prompted baseline cannot touch. | The technical note | In plain language | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The first sovereign in-weights micro-agent: an IOC classifier trained as a LoRA adapter on a 0.8B hybrid base, served with adapter hot-load, output-constrained by a 665-byte GBNF grammar, gated behind a feature flag that touches none of the four quarantined pipeline files. | The subject is the stack's first trained specialist (**the first micro-agent**): a tiny add-on layer (**a LoRA adapter**) taught to classify threat indicators, wearing a straitjacket that makes malformed output impossible (**the grammar**), and wired in behind an off-by-default switch. | | Training: 229.8 seconds wall-clock on one RTX 5070, 56 sovereign-curated rows, trainable parameters 6,389,760 of 859,375,680 (0.74%), loss 0.28 to 0.06, adapter 25 MB safetensors merged to a 12.8 MB GGUF. | The whole training run took under four minutes on the desk GPU (**229.8 seconds**), taught less than one percent of the model's weights (**0.74% trainable**), and produced a 12.8 MB file. | | The sealed A/B against a prompted 2B on identical prompts: accuracy 1.000 vs 0.222, F1 macro 1.000 vs 0.152, 35% faster at p95; the baseline's grammar-valid 1.000 is free-form JSON that is the wrong type or value 78% of the time. | Against a model over twice its size that was merely prompted, the specialist scored perfect where the baseline managed 22.2%, and answered a third faster (**the sealed A/B**). The baseline's output looked valid while being wrong most of the time, which is exactly the trap the design closes. | | The register is stated in the same breath: N=9 is a capability probe, not a population claim; the Wilson 95% interval on 9-of-9 is \[0.701, 1.000], and the statistically significant replication is pre-registered as RQ-1. | The honesty rides in the headline itself (**a capability probe, not a population claim**): nine test rows prove the pattern works, not how well it generalizes. The math of small samples (**the Wilson interval**) says at least 70%, and the bigger rerun is already on the books. | #### 1. Introduction **The point:** why a tiny classifier carries a whole chapter: it is the first instance of the pattern every later specialist inherits. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The Sovereign Pair's second half, in-weights specialization, first touched ground here: the claim is that corpus sovereignty plus in-weights specialization at micro-agent scale is a deployable engineering pattern, not an aspiration. | The series' central principle says: gates where checks can be mechanical, training where behavior must hold under load (**the Sovereign Pair**). This chapter is where the training half first became a working thing instead of a diagram. | | The deeper motivation is the precedent class: P4 was the first LoRA trained on this stack, establishing the ten-step template (sovereign dataset → train → merged GGUF → router preset → grammar-constrained output → A/B → quarantine respected → flag off → sealed) that CTI-NER, the schema pipeline, the SSD pilot, and the pruning study all trace lineage to. | The score matters less than the mold (**the precedent class**): this was the first, and it set the ten-step recipe (**the template**) every later specialist copies. Had the first one failed, none of the successors would have been attempted. The commit reads "first sovereign micro-agent forged." | | The spec's stated prior (\~85% at \~5.2 s for the prompted 2B) did not survive contact: D5 found Stage 1 called no LLM classifier at all, and the prompted 2B measured here scores 22.2% under strict grading; the \~85% figure is explicitly not the baseline this paper beat. | One inherited number is publicly corrected (**the prior that did not survive**): the old spec assumed the pipeline already used an 85%-accurate model. It used none at all, and the real prompted model scores 22.2%. The paper refuses the flattering comparison. | | Scope is one probe: one agent, one base, one GPU, one seed, one validator snapshot, with statistics, variance, cross-corpus, base-vs-instruct, and quantization each deferred to a pre-registered RQ, and production integration deferred to operator authorization. | The claim's edges are drawn tight (**one probe**): one model, one seed, one machine. Everything bigger is written down as a numbered future test, not implied. | #### 2. Background and Prior Art **The point:** the recipe sits on published foundations, and the one literature that could kill the design turns out to predict it. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | LoRA and QLoRA supply the parameter-efficient regime (rank-decomposed updates under 1% of parameters, paged 8-bit optimizer); the probe's 229.84 s against the 9B sibling arm's 106,324 s (29.5 h) is an economy comparison, not a controlled one: minutes-scale training is what makes a daily-cadence specialist loop feasible on one GPU. | The training method is standard published science (**LoRA**), and the economics are the point: minutes instead of a day and a quarter for the bigger sibling. That speed is what turns retraining from a project into a daily habit (**the daily cadence**). | | The probe operates below the small-language-model envelope at 0.8B, deliberately: the base has no chat template and no general capability claim; smallness lets the router hot-load and evict the 1.5 GB base plus 12.8 MB adapter in seconds alongside the heavyweight roster. | The model is tiny on purpose (**below the SLM envelope**): no claim it can chat, only that it nails one tight job, and it is small enough to load in seconds next to the big models without hogging the card. | | Constrained decoding rests on logit-mask-over-FSM foundations; the format-tax literature is engaged as the disconfirming lane and survives it favorably: base models often benefit from structural constraints where instruction-tuned models degrade, classification stays stable where generation falls away, and the dominant structured-output cost enters at the prompt, not the decoder; the grammar enforces shape only, the adapter carries content. | The one research lane that could sink the design (**the format tax**: forcing structured output often makes models worse) is quoted, not dodged, and read closely it favors this exact setup: a base model, a classification task, and the task knowledge trained into weights rather than begged in the prompt. The division of labor is the design (**shape versus content**): the grammar owns the shape, the adapter owns the meaning. | #### 3. Methods **The point:** the full recipe with its fingerprints, and six departures from the spec sealed with reasons instead of smoothed over. | The technical note | In plain language | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Substrate: the operator's primary workstation (i7-14700F, RTX 5070 12 GB, 128 GB DDR5), ≤45°C peak against an 85°C ceiling, zero thermal events; the full SHA chain from upstream checkpoint through corpus, adapter, merged GGUF, grammar, and manifest is sealed; the hybrid-architecture caveat is carried: LoRA attaches only to attention and MLP projections, Mamba blocks and vision tower untouched. | The lab is the same desk the whole stack runs on (**the substrate**), every artifact in the chain is fingerprinted, and one architectural honesty note rides along: the adapter only touches part of this unusual hybrid model, which sufficed here and might not for other task types. | | Corpus: 56 training rows + 7 validation + 9 held-out A/B, operator-authored across the canonical IOC classes; the spec called for ≥500 rows, and the de-scope is sealed as deviation D3 rather than smoothed; hyperparameters sealed in Table 1 (rank 16, alpha 32, dropout 0.05, 60 steps, seed 42). | The training set is 56 hand-curated examples (**the sovereign corpus**), far below the spec's 500-row ambition, and that shortfall is stamped into the record as a deviation instead of being quietly reworded. | | The six sealed deviations: D1 Base-not-Instruct (operator-ruled; structurally the stronger claim), D2 hyperparameter rescale (the spec's numbers never complete warmup on 56 rows), D3 the N=9 test set, D4 f16-not-Q8\_0 (the quantizer overflowed on zero-range hybrid tensors), D5 the gate in a new module (Stage 1 turned out to call no LLM at all; the quarantine rule forced a cleaner architecture), D6 an isolated training environment. | Six places where reality diverged from the plan, each sealed with its reason (**the six deviations**): the untuned base model chosen on purpose, the training schedule rescaled to actually fit the tiny corpus, the small test set owned, a file-format fallback forced by a tool bug, the integration point moved because the old pipeline never had a classifier to replace, and a walled-off training environment. Deviations documented beat deviations patched. | | Grammar and serving: the 665-byte GBNF generated from a Pydantic schema (with the sealed lesson: resolve $ref before conversion), safetensors merged to GGUF f16 atomically, a router preset with a deliberately small 2,048-token context, LRU eviction reloading in \~2 s; the A/B harness runs both arms on identical prompts at temperature 0.01 and preserves raw per-row results. | The output straitjacket is 665 bytes of grammar compiled from the schema (**GBNF**), the model reloads in about two seconds when called, and the comparison harness ran both contenders on identical inputs, keeping every raw answer on file. | #### 4. Results **The point:** the sealed numbers: a clean run, a perfect probe, a perfect out-of-sample live-fire, and validity decoupled from learning. | The technical note | In plain language | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Training: 229.84 s, 60 steps, loss 0.28 to 0.06, zero OOM/NaN/thermal events; the wall-clock economy against the 9B sibling converts the recursive moat from a batch process into a daily cadence. | The run was boring in the best way (**structurally clean**): under four minutes, converged smoothly, nothing overheated, nothing crashed. | | Live-fire: five out-of-sample strings spanning every canonical type (ip, domain, cve, email, hash), all correct at confidence 1.0; confidence calibration is explicitly uncharacterizable at this N and deferred to RQ-1's calibration curve. | Five fresh indicators the model had never seen, five correct answers (**the live-fire**). Whether its confidence numbers can be trusted as probabilities is a different question, honestly deferred. | | The sealed A/B (Table 2): accuracy 1.000 vs 0.222, F1 1.000 vs 0.152, latency p95 6,024 vs 9,288 ms (35% faster); the baseline's shape-valid output is wrong in type or value 78% of the time; the candidate's p95 misses the spec's 1-second target, owned by RQ-2. | The scoreboard (**the sealed A/B**): perfect against 22.2%, a third faster, and one miss stated in the same table: the specialist is still slower than the spec's one-second dream, and that investigation has its own numbered test. | | Grammar validity 21 of 21 across every candidate emission: structural validity is guaranteed by the grammar regardless of learning; task semantics are learned by the adapter regardless of the grammar; each covers the failure the other cannot express. | Twenty-one outputs, twenty-one perfectly formed (**21 of 21**), and the deeper point: the straitjacket guarantees the shape even if learning fails, and the learning supplies the meaning the straitjacket cannot. Two independent safety nets, by construction. | #### 5. Discussion **The point:** the verdict in one line: the capability is proven, the dataset is small, the discipline scales. | The technical note | In plain language | | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | At the B2 register this is the first peer-review-grade demonstration on this stack that in-weights sovereign specialization works at micro-agent scale: +77.8 pp accuracy, +84.8 pp F1, 35% faster, structural validity guaranteed at the logit level, with the N=9 caveat traveling with every restatement. | The finding, with its leash attached everywhere it appears (**the verdict**): training your own tiny specialist on your own data is a working engineering pattern, proven here for the first time on this stack. | | The two Cycle-13 anchors are joint-necessity-orthogonal: the SSD pilot carries calibration-retention at 9B (primary hypothesis falsified, survivor finding standing), this paper carries first-instance feasibility at 0.8B; neither alone establishes what both together do; the 56-row corpus is C₀ of the corpus-is-the-product loop. | Two sealed experiments hold up different corners of the same claim (**joint necessity**), and this one is also the training-data flywheel's first turn (**C₀**): the first corpus the workflow produced as a byproduct of doing the work. | | Deviation D5 reframes constraint as substrate: forcing the gate into a standalone module kept the broadcast path untouched, decoupled the gate from the call-graph, and made production integration an explicit operator decision; six axes on which the result does not generalize are listed against their RQs. | The rule that looked like drag produced the better design (**constraint as substrate**): the quarantine forced a cleaner integration than the spec's own plan. And the paper lists six specific ways its result might not travel, each mapped to the test that would find out. | #### 6. Limitations **The point:** the honesty ledger: a wide interval, one seed, one machine, a narrow task, and a baseline the paper itself argues was handicapped. | The technical note | In plain language | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | N=9 yields Wilson \[0.701, 1.000]: the lower bound clears the 22.2% baseline comfortably, but the width forbids a tight claim; single seed, single substrate, single base, all unmeasured variance; the tiny parameter count argues for seed stability but does not demonstrate it. | The small-sample math is printed rather than hidden (**the Wilson interval**): the result is at least well above the baseline, but nine rows cannot say how far. One seed, one machine, one model: every untested axis is named. | | The asymmetric-baseline admission: a fine-tuned 0.8B beat a prompted 2B running free-form JSON, precisely the regime the format-tax literature identifies as costly; some share of the 22.2% is plausibly that tax rather than a capability gap; the fine-tuned-2B comparison belongs to RQ-4, and the Admiralty B2 grade stands until RQ-1. | The paper undercuts its own headline where honesty requires (**the asymmetric baseline**): the loser was prompted, not trained, and run in the very mode research says taxes models hardest. Part of the gap may be tax, not talent. The fair fight is a numbered future test. | #### 7. Falsification Design **The point:** five pre-registered replication questions, each independently falsifiable, gating the grade promotion. | The technical note | In plain language | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | RQ-1 statistical retrain (≥0.95 accuracy on N≥50, Wilson lower bound ≥0.90, McNemar vs baseline; gates B2 to A1), RQ-2 the latency floor (p95 ≤1.0 s), RQ-3 cross-corpus (≥0.90 on a different class set), RQ-4 base-vs-instruct paired arms, RQ-5 Q8\_0-vs-f16; all pre-registered in the same commit as the paper, none gated on its conclusions. | Five written-down future tests (**the replication questions**): the big rerun that would upgrade the evidence grade, the speed target, the different-data test, the fair-baseline fight, and the file-format head-to-head. Each can fail, and each failure mode is named in advance. | | The paper states plainly that a separate hash-sealed pre-registration document was deliberately not authored: the capability-probe scope is too narrow, and the table itself is the pre-registration record. | One meta-honesty note: for a probe this small, the table in the paper is the pre-registration, and the paper says so instead of performing extra ceremony (**the table is the record**). | #### 8. Conclusion **The point:** the close in the chapter's own cadence: the capability is proven, the dataset is small, the discipline scales. | The technical note | In plain language | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The first sovereign micro-agent works: 0.74% trainable parameters, 56 rows, 229.8 seconds, 9/9 with F1 1.000 and grammar 21/21, 5-of-5 live-fire, against a prompted baseline at 22.2%; what the seal buys beyond the numbers is architectural confidence to scale: the template, the recipe class, the deviation-honesty discipline, and the corpus substrate all travel. | The first specialist works, and what that bought was bigger than the score (**architectural confidence**): a proven mold, a proven honesty discipline, and a corpus loop that every later specialist inherits. | | The reader who walks the trail from spec to recipe to sealed results to gate code to adapter binary inherits a complete record: hypothesis, probe, six documented deviations, quarantine compliance, harness numbers, falsification design, and the moat's first cycle. | Everything is walkable end to end (**the complete record**): from the plan, through the training, to the binary file itself, with the wrong turns documented. **The capability is proven. The dataset is small. The discipline scales.** | #### System Update: July 2026 **The point:** the append-only update: fifteen weeks of production residency, the template claim exercised by a dated decision, and a gate that honestly stayed shut. | The technical note | In plain language | | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The adapter has been resident in the production roster for 105 days: the router preset has stood since the seal, found loaded by a 2026-05-21 infrastructure probe, still present in a preset file actively maintained as recently as 2026-07-12; an availability record, with no performance claim made from it. | The four-minute experiment became a standing production asset (**105 days of residency**), and the update is careful about what that proves: the adapter is available and loaded, which is a fact about presence, not performance (**residency, not performance**). | | The precedent-class claim was exercised on 2026-05-22: at architectural review the operator selected this probe's trainer as the canonical template for the Watcher cascade classifier, rejecting a competing embedding-based trainer; the Watcher classifier scales the same geometry from 0.8B to 2B and six enum values to thirteen; CTI-NER shares the same base and preset pattern. | The inheritance stopped being a prediction (**the template propagated**): a later, bigger specialist was built by explicit dated decision on this exact recipe, with a rival approach rejected in its favor. The claim is now a record, not an assertion. | | The Stage-1 gate remains closed: the flag defaults off, no configuration enables it, automatic triage insertion has never been switched on; no invocation-volume statistic is offered because the gate that would generate one has not opened; RQ-1 through RQ-5 remain open, and B2-to-A1 promotion still waits on RQ-1. | And the third update is a non-event kept honest (**the gate stayed shut**): the production switch the paper deferred to the operator is still off, so the paper refuses to invent usage numbers, and the evidence grade stays exactly where the seal left it. | *** ### Test yourself A short quiz on this chapter, generated in Google's Gemini LM (formerly NotebookLM) from the chapter itself. Work through the notes above first, then check what stuck; the link opens the quiz on Google's site. {% embed url="\_" %} *The quiz is AI-generated: its questions and answer keys are an interpretation of the chapter, not the chapter, and can misstate a detail. Where a question and the text disagree, the written chapter is the authoritative, canonical source:* [*13 · Sovereign IOC Classifier*](/osintelligence/part-iv-the-evidence-what-worked/13-sovereign-ioc-classifier.md)*.*