> For the complete documentation index, see [llms.txt](https://osintelligence-llc.gitbook.io/osintelligence/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://osintelligence-llc.gitbook.io/osintelligence/evidence-and-seals/ioc-classifier-micro-agent-p4.md).

# IOC Classifier Micro-Agent (P4)

*Evidence for* [***13 · Sovereign IOC Classifier***](/osintelligence/part-iv-the-evidence-what-worked/13-sovereign-ioc-classifier.md)*. This is the record behind the chapter's claim to the first sovereign in-weights micro-agent: a rank-16 LoRA on Qwen3.5-0.8B-Base, GBNF-constrained, that classified indicators at 100% versus a prompted 2B janitor baseline's 22% on the held-out set and 5/5 on out-of-sample live-fire, the pipeline proven end to end on one consumer GPU.*

*How this evidence is classed, and how to read a seal generally, is on* [***Verifying a Seal***](/osintelligence/evidence-and-seals/verifying-a-seal.md)*.*

**Phase run** 2026-04-11 · **verdict GREEN (capability probe, Admiralty B2)** · pipeline green end to end; broad-N evaluation deferred to a full-corpus retrain.

## Evidence class (read this first)

**Report-class, capability probe, not a pre-registration seal.** P4 ran 2026-04-11, before the 2026-04-18 hash-provenance correction that created this program's canonical-prefix sealing discipline (documented on the [Self-Distillation Pilot (P7)](/osintelligence/evidence-and-seals/self-distillation-pilot-p7.md) page). It has no canonical self-hash to quote, and this page manufactures none. It also states its own statistical honesty in the report: the held-out set was 9 rows (P3 de-scoped the data volume), so the result is a **capability probe that proves the pipeline, not a stat-significant production claim**, and the report says exactly that. What it carries instead: a seed-pinned training manifest, a GBNF grammar hashed `56b1c8bf…68c390`, six documented spec deviations, and a live-fire out-of-sample check. Reproduced verbatim and labelled for what it is.

## Result (as reported, honest scope)

| Metric                                          | Baseline (2B janitor, prompted)      | Candidate (0.8B + LoRA + GBNF)  |
| ----------------------------------------------- | ------------------------------------ | ------------------------------- |
| accuracy (9-row held-out)                       | 0.222                                | **1.000**                       |
| F1 macro                                        | 0.152                                | **1.000**                       |
| grammar-valid rate                              | 1.000 (valid JSON, wrong type/value) | **1.000**                       |
| latency p50                                     | 5,457 ms                             | **3,618 ms**                    |
| live-fire (5 distinct IOC types, out-of-sample) | —                                    | **5/5 correct, confidence 1.0** |

Training: seed 42, LoRA r16/α32/dropout 0.05, 6.39M trainable params (0.74%), 56 train rows, 229.8 s on an RTX 5070, final loss 0.06. The report is explicit that the 100% on 9 rows plus 5 live-fire strings proves the forge pipeline (train → GGUF → Router-Mode preset → feature-flag gate) end to end, and that a full-corpus retrain with statistical significance is the deferred follow-up. The four pipeline-quarantine "sacred files" were untouched; the feature flag lives in a new standalone module rather than editing a quarantined file, the discipline the record keeps around its own production pipeline.

## The final report, verbatim

Reproduced exactly as written at phase completion (`P4_ioc_classifier_report.md`).

````md
# P4 — IOC Classifier Micro-Agent — Results Report

**Status:** GREEN (capability probe — see Data Reality caveat)
**Date:** 2026-04-11
**Spec:** [SPEC_P4_ioc_classifier.md](../SPEC_P4_ioc_classifier.md)
**Roadmap:** velvet-swimming-parrot — first sovereign micro-agent forge

## TL;DR

- LoRA trained, exported to GGUF, registered as Router Mode preset, hot-loaded
- A/B harness: **candidate beats baseline 100% / 22% accuracy on 9-row test set**
- Live-fire end-to-end: 5/5 IOC types classified correctly via feature-flag gate
- All 4 sacred files in pipeline-quarantine zone untouched

## Artifacts

| File | Size | Purpose |
|------|------|---------|
| `INFRA/MODELS/MICRO_AGENTS/ioc_classifier/v1/Qwen3.5-0.8B-Base-f16.gguf` | 1.51 GB | Base model (own conversion from `unsloth/Qwen3.5-0.8B-Base`) |
| `INFRA/MODELS/MICRO_AGENTS/ioc_classifier/v1/ioc_classifier-v1.gguf` | 12.8 MB | LoRA adapter (f16) |
| `INFRA/MODELS/MICRO_AGENTS/ioc_classifier/v1/schema.gbnf` | 665 B | GBNF grammar (sha256 `56b1c8bf…68c390`) |
| `INFRA/MODELS/MICRO_AGENTS/ioc_classifier/v1/adapter/` | 25 MB | PEFT safetensors source |
| `INFRA/MODELS/MICRO_AGENTS/ioc_classifier/v1/training_manifest.json` | — | seed, hyperparams, deviations |
| `INFRA/llama.cpp/models.ini` | — | New `[ioc-classifier]` preset |
| `REFINERY_MEMORY_SERVICE/SRC/training/train_ioc_classifier.py` | — | Trainer (seed-pinned, SHUSH/thermal gated) |
| `REFINERY_MEMORY_SERVICE/SRC/utils/delegator.py` | — | New `ioc-classifier` worker + `classify_ioc()` raw-mode method |
| `REFINERY_MEMORY_SERVICE/SRC/utils/ioc_classifier_gate.py` | — | `HIVE_IOC_CLASSIFIER_V1` feature flag gate |
| `PLAYBOOK_VAULT/PROMPTS/ROLE_IOC_CLASSIFIER.txt` | — | Identity prompt |
| `PLAYBOOK_VAULT/TEMPLATES/ioc_classifier.json` | — | Playbook template |
| `TEST_SUITE/spec_p4_ioc_ab.py` | — | A/B harness (raw `/completion` candidate) |
| `R:\LOGS\spec_p4_ab_report.json` | — | Full A/B per-row results |

## Training Run

```
seed                : 42
base                : unsloth/Qwen3.5-0.8B-Base (HF → own GGUF f16)
LoRA r/α/dropout    : 16 / 32 / 0.05
target_modules      : q_proj k_proj v_proj o_proj gate_proj up_proj down_proj
epochs / steps      : 15 / 60          (deviation — see below)
batch / grad_accum  : 4 / 4 (eff 16)
lr / scheduler      : 2e-4 / cosine, warmup 5
optim               : paged AdamW 8-bit
trainable params    : 6,389,760 / 859,375,680  (0.74%)
train rows          : 56  (P3 de-scoped)
val rows            : 7
runtime             : 229.8 s on RTX 5070 (sm_120, FA off, Xformers fallback)
final train_loss    : 0.06 (started 0.28)
adapter size        : 25 MB safetensors → 12.8 MB GGUF f16
```

## A/B Result (9-row held-out test set)

| Metric | Baseline (Janitor 2B prompted, port 8081) | Candidate (LoRA + GBNF, port 8080) |
|---|---|---|
| accuracy | **0.222** | **1.000** |
| F1 macro | 0.152 | 1.000 |
| grammar_valid_rate | 1.000* | 1.000 |
| latency p50 | 5,457 ms | 3,618 ms |
| latency p95 | 9,288 ms | 6,024 ms |

\* Janitor `response_format=json_object` produces *valid JSON* but the wrong type/value.

**Live-fire (out-of-sample, 5 indicators, all distinct types):**
| input | type | value | confidence |
|---|---|---|---|
| `8.8.8.8` | ip | 8.8.8.8 | 1.0 |
| `evil.example.com` | domain | evil.example.com | 1.0 |
| `CVE-2024-3094` | cve | CVE-2024-3094 | 1.0 |
| `admin@corp.org` | email | admin@corp.org | 1.0 |
| `d41d8cd98f00b204e9800998ecf8427e` | hash | d41d8cd…8427e | 1.0 |

## Documented Deviations from SPEC_P4

1. **Base = Qwen3.5-0.8B-Base, not -Instruct.** Operator directive. Base has no
   chat template, so the trainer uses a plain `### System: / ### Indicator: /
   ### Classification:` scaffold and the candidate is served via the `/completion`
   raw endpoint instead of `/v1/chat/completions`.
2. **Hyperparameter rescale.** Spec said 3 epochs + 50 warmup steps; that yields
   only ~10 total optimizer steps for a 56-row corpus, with warmup never
   completing. Bumped to 15 epochs + 5 warmup steps for 60 total steps. Loss
   converged 0.28 → 0.06. Recorded in `training_manifest.json`.
3. **Test set size.** Spec assumed ≥50 held-out rows; P3 de-scoped to 9 (P3
   green-flagged the infrastructure but not the data volume). A/B numbers are
   reported as raw counts and treated as a **capability probe**, not a
   stat-significant production claim. The 100% accuracy on 9 rows + 5 live-fire
   strings is sufficient to prove the pipeline; broader evaluation deferred until
   the dataset reaches the spec target.
4. **Quantization.** Spec said Q8_0; the f16 quant pass produced overflow
   warnings on Mamba/vision tensors that have zero-range weights. Switched to
   f16 (1.5 GB — still well under VRAM budget). Q8_0 path can be revisited once
   llama.cpp's quantizer handles the hybrid SSM tensors cleanly.
5. **Feature-flag wiring.** Spec called for the gate to live in
   `enrichment_engine.py` or "wherever Stage 1 calls the classifier". Stage 1
   does not currently call any LLM classifier — `extract_iocs()` is pure regex.
   The pipeline-quarantine rule forbids editing `main.py`, `enrichment_engine.py`,
   `sequencer_telemetry.py`, or `shakedown_runner.py`. The feature flag therefore
   lives in a NEW standalone module
   `REFINERY_MEMORY_SERVICE/SRC/utils/ioc_classifier_gate.py` exposed via env
   var `HIVE_IOC_CLASSIFIER_V1`. Sequencer-side adoption requires either an
   operator-approved amendment to the quarantine rule or a new orchestrator hook
   that imports the gate. **No quarantined file was modified in this phase.**
6. **Training environment.** Built an isolated `venv-training/` with
   torch 2.10.0+cu128 + Unsloth 2026.4.4 + transformers 5.5.0 to avoid disturbing
   the main `venv` (still pinned to transformers 4.46.3 for LLMLingua-2). The
   main venv is untouched.

## Hybrid-Architecture Caveat

`unsloth/Qwen3.5-0.8B-Base` is a hybrid Mamba-SSM + Linear-Attention +
Full-Attention + Vision model (`Qwen3_5ForConditionalGeneration`). LoRA adapters
attach only to the standard projection modules in the full-attention layers and
the MLPs — Mamba blocks and the vision tower remain untouched. For a structural
classifier this is fine and the perfect-score result confirms it; if a future
agent needs deeper adaptation, switching to a text-only fork
(`ConicCat/Qwen3.5-0.8B-Text-Only`, `principled-intelligence/Qwen3.5-0.8B-text-only`)
would also drop ~400 MB of dead vision-tower weight.

## Key Lessons (for P5 onward)

1. **Router Mode + `/completion` requires `model` field.** Without it the
   server returns HTTP 400. Bit me on the first A/B run; fixed in both the
   harness and the delegator.
2. **`--lora` in models.ini works.** llama.cpp parses the `lora =` key and
   passes `--lora` automatically. Verified via `/lora-adapters?model=<preset>`.
3. **Base model GGUFs need to be regenerated from the same HF checkpoint
   the LoRA was trained on.** The pre-existing `Qwen3.5-0.8B-Q8_0.gguf` in
   `MODELS/Qwen3.5-0.8B-Utility/` may have come from the Instruct fork; safer
   to do our own conversion under the same `unsloth/*-Base` directory.
4. **`SchemaConverter.visit()` does not auto-resolve `$ref`.** Call
   `resolve_refs(schema, '')` first. (Pattern lifted from
   `TEST_SUITE/spec_p3_curator_harness.py`.)
5. **`TrainerCallback` subclassing is mandatory** for HF Trainer hooks
   (transformers v5 added `on_pre_optimizer_step` etc. — duck-typed callbacks
   crash mid-train).

## Verification Commands

```bash
# Confirm preset is loaded
curl -s http://localhost:8080/v1/models | grep ioc-classifier

# Confirm LoRA is active
curl -s "http://localhost:8080/lora-adapters?model=ioc-classifier"

# Live test via gate
HIVE_IOC_CLASSIFIER_V1=1 venv/Scripts/python.exe -c \
  "import sys; sys.path.insert(0,'REFINERY_MEMORY_SERVICE/SRC'); \
   from utils.ioc_classifier_gate import classify_indicators; \
   print(classify_indicators(['8.8.8.8']))"

# Re-run A/B
venv/Scripts/python.exe TEST_SUITE/spec_p4_ioc_ab.py
```

## Next (P5)

CTI NER micro-agent. Same pattern: GBNF + plain scaffold + Router Mode preset
+ feature-flag gate. Once dataset volume permits, retrain IOC classifier on the
full 500-row corpus and re-run A/B with stat significance.

**Admiralty Grade:** B2 (capability probe — small N, but pipeline GREEN end-to-end).
````

***

*Evidence & seals · IOC Classifier Micro-Agent (P4) · report-class capability probe reproduced verbatim; pre-dates the canonical-prefix sealing discipline (P7, 2026-04-18), classed honestly as a validated report rather than a pre-registration seal · CC BY 4.0 · © Jamey Kistner, OSINTelligence LLC*