> For the complete documentation index, see [llms.txt](https://osintelligence-llc.gitbook.io/osintelligence/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://osintelligence-llc.gitbook.io/osintelligence/part-iv-the-evidence-what-worked/14-sovereign-cti-ner/the-quick-version.md). # The quick version **Continue the tour →** [Next: 15 · Sovereign Imatrix Calibration, the quick version](/osintelligence/part-iv-the-evidence-what-worked/15-sovereign-imatrix-calibration/the-quick-version.md) The short version of Chapter 14, three ways: the video walks the argument in a few minutes, the deep dive talks it through at a listening pace, and the infographic holds the whole chapter in one view. The full result, with the 0.8B adapter that Pareto-dominates the larger incumbent and the falsified primary hypothesis kept in full, lives in the chapter itself: [14 · Sovereign CTI-NER](/osintelligence/part-iv-the-evidence-what-worked/14-sovereign-cti-ner.md). {% embed url="" %} **The deep dive.** A podcast-style audio conversation about this chapter: two AI hosts walk through the argument, the incidents behind it, and what it means, at a listening pace. Generated in Google's Gemini LM (formerly NotebookLM) from the chapter itself; the link opens the audio on Google's site. {% embed url="\_" %} *The conversation is AI-generated: an interpretation of the chapter, not the chapter. It can compress, paraphrase, or get details wrong. The written chapter is the authoritative, canonical source:* [*14 · Sovereign CTI-NER*](/osintelligence/part-iv-the-evidence-what-worked/14-sovereign-cti-ner.md)*.* *** ![The Sovereign CTI-NER, the chapter in one view.](https://137900913-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fxx2bv6VR9dSJDJ9HsDER%2Fuploads%2FEHUPfkWgyNwzQnvmqHoW%2Fctiner-infographic.png?alt=media) *** ### Chapter notes Section-by-section notes in two registers: the technical note on the left, the same idea in plain language on the right. Every row is one idea, so you can read straight across from one register to the other. The technical terms stay visible in the plain column on purpose; they are the vocabulary worth keeping. #### Abstract **The point:** the second specialist beats the model that taught it: a 0.8B adapter, trained on labels the operator's own 9B produced, dominates that 9B on every deployment axis. | The technical note | In plain language | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | A 0.8B decoder-class LoRA, trained on a fully sovereign corpus (operator-curated MITRE ATT\&CK material, permissively-licensed public CTI datasets projected onto a STIX 2.1 subset, and silver labels from the operator's own deployed 9B), reaches relaxed entity-F1 0.720 on a 1,002-row held-out split versus the incumbent 9B's 0.258: 2.79×, at 11× fewer parameters. | The task is pulling structured threat-intel entities out of text (**NER**: names of malware, actors, tools, targets). The specialist was trained with no outside teacher at any point (**sovereign silver labels**: the operator's own bigger model wrote the practice answers), and it beats that bigger model by nearly three times. | | The dominance is across the board: schema-valid 0.999 vs 0.997, p50 latency 4,068 vs 4,539 ms, and 8.5 GB of VRAM returned to the card. | The win is not one number (**Pareto-dominates**): better answers, more reliable formatting, faster typical response, and it hands back most of the graphics memory the big model was occupying. | | The load-bearing novelty is the supply-chain argument: silver-label poisoning is named as the fine-tuning analogue of the calibration-corpus attack, and sovereign silver-labeling is the defense that is also quality-improving rather than a quality tax. | The deeper point is security, not scores (**silver-label poisoning**): training data labeled by someone else's model is a tampering surface. Labeling with your own model closes it, and here that defense came with a quality gain instead of a cost. | | §4.0 deliberately reports a falsified infrastructure hypothesis alongside the source-verified root cause: the negative result is part of the record, not an embarrassment to be edited out. | And the paper keeps its own wrong guess in print (**the falsified hypothesis**): the debugging dead-end is reported with the same rigor as the result, on principle. | #### 1. Introduction **The point:** the encoder-model status quo fails two sovereign constraints, so the paper inverts both: native structured output, and no external teacher anywhere. | The technical note | In plain language | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Published CTI NER converged on encoder-class models (BERT-class) supervised on aggregated public corpora; two properties disqualify that for a sovereign agentic deployment: token-label output needs a fragile post-hoc wrapper to become JSON, and every published encoder depends on a pretrained domain checkpoint the operator cannot re-derive. | The field's standard tools fail this deployment twice (**the encoder paradigm**): their output is not the structured JSON the pipeline's agents eat, and they are all built on someone else's opaque starting model, which fails the audit bar no matter the license. | | The sovereign constraint set binds four requirements: training-time supply-chain audit, serve-time grammar-constrained structured output, consumer-hardware training budget, and in-distribution corpus discipline as a defensive surface. | Four house rules shape the design (**the constraint set**): every training ingredient auditable, output shape enforced by the server, training done in hours on the desk GPU, and the corpus treated as a security boundary, not just data. | | The paper explicitly does not claim to beat SecureBERT 2.0 on the public benchmark (the comparison is not apples-to-apples); it claims Pareto-dominance over the operator's own incumbent 9B under the agentic deployment profile. | The claim is drawn carefully (**the honest target**): not "beats the published state of the art," which the paper says it cannot fairly measure, but "beats the model actually doing this job in production, on every axis that matters there." | | This is the supervised-fine-tuning extension of the imatrix companion's thesis: the same in-distribution discipline that improved quantization calibration 16× is applied to training labels, and the same supply-chain attack generalizes. | This chapter and the next are one argument in two halves (**the in-distribution thesis**): tune the compression on your own data, and now train the specialist on your own labels, for the same twin reasons of quality and tamper-proofing. | #### 2. Background **The point:** the dataset landscape audited corpus by corpus, and the serving mechanism that makes a decoder the right shape for agents. | The technical note | In plain language | | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Five corpora anchor CTI NER (CyNER, DNRTI, APTNER, AttackER, CyberNER's harmonization); each was audited for redistributable licensing, and the schema here is an 11-class operational subset of CyberNER's 21-class STIX-aligned taxonomy, used as a design reference rather than a data source. | The public datasets got individually vetted (**the corpus audit**), and the label scheme is a deliberate 11-class subset of the field's newest unified taxonomy, borrowing its design while refusing its unlicensed data. | | llama.cpp's constrained decoding masks the logit distribution at each step so schema-invalid characters are unemittable; the known failure mode is documented honestly: a model that exhausts its token budget emits a valid-prefix-but-incomplete payload, observed in 4 of 2,004 rows and corroborated cross-vendor. | The output straitjacket works at the letter-by-letter level (**constrained decoding**): invalid JSON physically cannot be typed. Its one documented weakness, running out of budget mid-object, is measured here (4 rows in 2,004) and traced across other vendors' tools to show it is the mechanism, not this machine. | | Encoder NER emits per-token label distributions requiring downstream span assembly, overlap resolution, and schema projection, each a failure surface; decoder + GBNF collapses the pipeline into one server-side primitive. | The old way needs a chain of translation steps between model and pipeline, each a place to break. The new way emits the finished object directly (**decoder plus grammar**): one moving part instead of five. | #### 3. Method **The point:** an 11-class schema with its decisions shown, a three-tier corpus with an audit trail, and a labeling protocol where the operator sees every label. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Schema: 11 STIX 2.1-aligned classes, each with a concrete downstream consumer; the threat\_actor/intrusion\_set split preserved per STIX semantics (all 268 v4 threat\_actor rows promoted to intrusion\_set under a deterministic recast); one post-seal correction is stated in place (vulnerability retained, course\_of\_action not carried, the sealed artifacts ruling). | Eleven entity types, each kept only because a downstream agent actually consumes it (**the schema**). One naming subtlety is handled with care (named APT groups versus unnamed attackers), and one place where the plan and the sealed data disagreed is corrected in the open rather than papered over. | | Corpus: three tiers totaling 8,088 rows (7,086 train / 1,002 test, seed-pinned split): Tier 1 the operator corpus (762 rows recast deterministically, zero needs-review), Tier 2 the two license-cleared public corpora (7,288 rows projected onto the 11 classes), Tier 3 sovereign silver labels (38 passages). | Three ingredients (**the tiers**): the operator's own curated rows, the two public datasets that passed the license bar, and a small fresh layer labeled from the operator's own broadcast archive. Every row carries its source in a manifest. | | Silver labeling: the deployed 9B labels in-distribution passages under strict JSON Schema, confidence ≥ 0.80, and a verbatim-substring check; hallucinated entities are dropped (7 of 254); model-emitted character offsets are discarded (\~100% offset-mismatch, consistent with decoder models having no character-counting mechanism) and recovered deterministically by local string search. | The labeling line has three quality gates (**the acceptance gates**): the format must validate, the model must be confident, and every claimed entity must literally appear in the text. One known decoder weakness, counting character positions, is routed around by recomputing positions locally (**offsets recovered locally**), keeping them auditable. | | Training: the IOC classifier's exact recipe class (rank 16, alpha 32, seven projection targets), 4,096-token sequences, effective batch 16, 3 epochs over 7,086 rows = 1,329 steps, under the SHUSH broadcast gate and the 85°C thermal gate, re-checked every 10 steps; one documented substitution: the spec named 2B-Instruct, the run used the on-disk 0.8B-Base, with the expected long-tail cost stated in advance. | The training reuses the proven recipe from the first specialist, runs quietly around the broadcast schedule and the thermal ceiling, and one departure is documented rather than hidden (**the base substitution**): a smaller starting model than planned, with the predicted cost named before the results. | | Serving: the adapter merges and registers as a router preset sharing the same f16 base as the IOC classifier, so the marginal cost is a ≤15 MB adapter; clients hit the endpoint with the training scaffold and a Pydantic-derived GBNF grammar guaranteeing schema-valid output by construction. | In production the specialist costs almost nothing (**the shared base**): it is a 15 MB add-on to a model already on the roster, answering in guaranteed-valid JSON. | #### 4. Results **The point:** the deepest water: a crash, a wrong hypothesis falsified on purpose, the real bug found in upstream source, and then the clean sweep. | The technical note | In plain language | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | §4.0, the experience report: the first A/B collapsed at row 39 with an access violation in llama.dll; the initial hypothesis (a CUDA 13.1 kernel fault on Blackwell, supported by three public sources) was tested by rebuilding on CUDA 12.8, which reproduced the identical failure signature: hypothesis falsified. | The honest centerpiece (**the falsified hypothesis**): the eval server kept crashing, the obvious well-sourced explanation was tested properly, and the test came back negative. The wrong guess is printed with its evidence, not deleted. | | The verified root cause: llama.cpp issue #20093, a heap buffer over-read in prompt-cache restoration for M-RoPE models (the Qwen 3.5 family), matching every observed attribute; a three-patch bundle (+18/−2 lines) against tag b7992 closed it; attribution goes to the upstream reporter and PR author, with this paper contributing the falsification, the correlation, and the reproducibility bundle. | The real bug was found in the open-source server's own issue tracker (**the M-RoPE heap over-read**): a memory bug specific to this model family, fixed with a tiny community patch. Credit is given where the fix came from; this paper's contribution is proving which explanation was right. | | The three-run ledger (§4.5) reports 13.1 (39/1,002 then wedged), unpatched 12.8 (identical signature, the falsification evidence), and patched 12.8 (2,000/2,004 = 99.80%, zero events): the middle column is the scientific crux, and omitting it would present a false toolchain-pivot resolution. | All three runs are published side by side (**the three-run ledger**), and the paper names why: without the middle column it would look like "we changed compilers and it worked," which is exactly the false story the data disproves. Method: hypothesize, falsify cheaply, verify against source, confirm categorically. | | The headline A/B on the patched binary: relaxed F1 0.720 vs 0.258 (2.79×), strict 0.152 vs 0.052, schema-valid 0.9990 vs 0.9970, p50 4,068 vs 4,539 ms, p95 honestly worse (11,349 vs 9,509 ms), 11× fewer parameters, 8.5 GB VRAM returned; the strict/relaxed gap is shared by both arms, indicating a property of the projected labels, not the model. | The scoreboard (**the Pareto table**): the specialist wins on answers, formatting, typical speed, size, and memory, and the one axis it loses (worst-case latency) is printed in the same table. The paper also explains why the "strict" score looks low for both contenders: the measuring stick, not the models. | | Per-class results carry the diagnosis: intrusion\_set F1 0.914 where the baseline scores 0.000 (the schema recast validated: a learned discrimination, not a prior), and the weakest classes (campaign 0.182, infrastructure 0.189) are the scarcest in training data, with the baseline weaker still on the same classes. | The class-by-class table tells two stories (**the schema recast validated**): the specialist mastered a distinction the big model has never heard of, and the weak spots track thin training data, not a flawed approach, since the incumbent is even weaker there. | #### 5. Discussion **The point:** what the result proves, what it deliberately does not, and the supply-chain threat model that is the chapter's real payload. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Both arms were served by the same silver-labeling teacher (the 9B itself), so the candidate's advantage is attributable to learned output-shape discipline plus task specialization, not label privilege; the strong-form question (matching published encoders on the full 21-class benchmark) is explicitly deferred, with the 0.720 figure "comparable in the approximate sense, explicitly not in the benchmark-matched sense" to the encoder baseline's 0.71. | The comparison is fair by construction (**no label privilege**): the loser wrote the winner's training labels. And the tempting headline, "matches the published state of the art," is explicitly declined: close in the loose sense, unproven in the strict sense. | | §5.3, the threat model: an adversary contributing a subtly-drifted labeling model to the open ecosystem poisons every downstream LoRA, visible only as diffuse quality regression indistinguishable from labeling noise; sovereign silver-labeling defeats it because the operator runs the teacher, sees every label, and would catch a labeler regression in production first. | The payload argument (**silver-label poisoning**): poison one popular teacher model and you quietly poison everything trained on its labels, with the damage disguised as ordinary noise. Running your own labeler closes the channel, and any drift in it would show up in daily use before it reached training. | | §5.4 lists the limits plainly: single architecture, single domain, single operator, the benchmark-collapsing decision (in-distribution and public test are the same split), the base-model substitution, and the strict-F1 reading; §5.5 gives the full reproduction budget: \~3.5 h training + \~3 h evaluation on one RTX 5070, ≈0.91 kWh. | The honesty ledger names its six soft spots, including the one that most tempts overselling (**benchmark collapsing**: the two test framings are the same split, so external validity is bounded), and the whole thing reproduces overnight on one consumer GPU for under a kilowatt-hour. | #### 6. Ethics, Licensing, and Data Governance **The point:** the license audit as a security control: unlicensed corpora are excluded twice over, legally and as unverifiable supply-chain artifacts. | The technical note | In plain language | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The audit ruling: CyNER (MIT) and AttackER (CC BY 4.0) clear the permissive bar; DNRTI, APTNER, and the CyberNER harmonization artifact default to all-rights-reserved under Berne and are excluded; citation-request notes in READMEs do not constitute license grants; per-row provenance is carried in the manifest. | Two datasets in, three out (**the license audit**), on a strict reading: no license means no permission, and a polite "please cite us" note is not a license. Every training row remembers where it came from. | | §6.4 upgrades the legal rule to a security rule: unlicensed artifacts cannot be verified as untampered over time because no framework obligates the publisher to preserve the audited state; sovereign pipelines narrow the surface to artifacts whose provenance is contractually maintainable. | The closing turn (**licensing as supply chain**): an unlicensed dataset is not just a legal risk, it is a file nobody promises to keep unaltered. The license bar and the tamper bar turn out to be the same bar. | #### 7. Conclusion **The point:** the thesis carries from calibration to fine-tuning intact: the operator's own data is the under-used lever, and the pattern generalizes. | The technical note | In plain language | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Sovereign post-training adaptation is named a first-order lever the open-source community systematically under-uses: 2.79× the incumbent's F1 at 11× fewer parameters, 8.5 GB returned, higher schema-valid rate, reproducible end-to-end on one consumer GPU with zero external API calls. | The close makes the general claim (**the under-used lever**): the field keeps assuming more parameters or a fancier teacher is the path to quality. This chapter's counterexample beat both, using nothing but the operator's own data and hardware. | | The pattern is offered with its generalization conditions: operator-authoritative data, a checkable output shape, and a general-purpose incumbent under constrained decoding; the OS-INTelligence deployment is the reference implementation. | The recipe travels under three conditions (**the pattern**): you own the domain data, the output has a checkable shape, and the job is currently done by a prompted generalist. Where those hold, a weekend specialist likely wins. | #### System Update: July 2026 **The point:** the append-only update: the one-off became a fleet pattern, and this method section became the template. | The technical note | In plain language | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | By May 2026 the adapter was serving in the production roster as one of several specialized LoRAs sharing one base (Threat-Watch, operator-voice, housekeeping, strategist), corroborating the §5.2 deployment-shape claim under sustained load rather than one-shot A/B. | The specialist did not stay an experiment (**production residency**): it serves daily alongside a family of siblings that share its base model, which is the deployment argument proven by use. | | The corpus and training pipeline generalized into a declarative multi-persona framework compiling per-persona corpora across dozens of operator personas, with the 11-class taxonomy pinned as the canonical CTI-NER schema; the sovereign-versus-generic question has been carried to larger scale as a pre-registered head-to-head, and the same discipline underwrites the compression flagship. | The bigger echo (**fleet-wide propagation**): the machinery built for this one specialist now stamps out training corpora for dozens of roles, this paper's method section retroactively reads as the first worked example, and the bigger version of its central question is already pre-registered downstream. |