OPSD Warm-Start Campaign Postmortem
2026-07-03 → 2026-07-22 · branch feature/opsd-training ·
evidence log coopertrain/opsd/evidence/EXPERIMENT_LOG.md ·
student Qwen3.5-9B · teacher 27B · CooperBench coop pairs · 1 seed per condition
TL;DR. The Phase 3b warm start (full-parameter SFT of Qwen3.5-9B on 16
success-gated 27B-teacher trajectories, 233k output tokens, exact-token TITO) actively
degraded the model. On the contamination-free 19-pair full-set eval, untouched base beats
the warm start, the 27B teacher, and the cooper midtrain lineage. Every training-side lever moved
surface behavior; only harness-side levers (step budget, infra fixes) moved hard outcomes.
Status: held for direction — no further training or scan spend.
Full-set standings (19 pairs, features disjoint from training corpora, step limit 200)
| model | features /32 | pairs ≥1 pass /16 | nonempty patches /38 | submitted /38 |
|---|
| base Qwen3.5-9B | 3 (9.4%) | 3 | 21 | 21 |
| cooper-instruct-midtrain | 2 (6.3%) | 2 | 16 | 24 |
| 27B teacher | 1 (3.1%) | 1 | 32 | 31 |
| warm-start step 250 (WS-250) | 0 | 0 | 4 | 2 |
“pairs ≥1” = pairs where at least one of the two agents’ features passed its tests.
3/19 pairs crashed identically in eval for all models (container dies pre-exec, docker 409) → scored
on 16 pairs / 32 features; patch/submit rates over all 38 agents.
The campaign in one diagram
Phase 1 — post-refactor smoke (Jul 3–12)
v3/v4 one-step stages: identity-control loss exactly 0.0; privileged loss ≈ 0.014–0.018,
nonzero LoRA grads. Optimizer path + exact-token TITO seam qualified.
▼
Phase 2 — teacher-as-actor scan (taa-v1)
Does the collaboration brief in the model’s own prompt improve acting? Stock vs collab:
both floor at 0–1/8 patches → bottleneck is capability, not conditioning → fork to 3b.
▼
Phase 3b — teacher corpus (Jul 14)
27B teacher rollouts, stock prompt, auto-eval ON. Round 1: 25 pairs (10 void — arm64 images);
round 2: 26 amd64-verified pairs. Yield 19–33%. Success gate keeps
16 passing feature-trajectories → 34 conversations, 1,143 turns, 233k output tokens.
Diagnostic-task collisions caught and permanently excluded.
▼ exact-token TITO corpus (byte-exact re-render gate, compaction turns masked)
Warm-start SFT (Jul 14–15)
Full-parameter, Modal 8×H100, verl 0.7.1, 3 epochs = 405 steps, lr 5e-6 cosine.
Val-loss min 0.218 @ step 100; step 250 preselected primary, step 100 backup;
FSDP→HF merge, text-only export.
▼
Qualification scans v1–v5 (Jul 15–16), preregistered gates
v1–v3 void on infra (cold endpoint / serving layout / manifest). v4: WS-250 fails
A (1/8 patches) and C (1/8 submissions). v5: WS-100 strictly worse (0/8). Base control wins
gates A and C outright.
▼
Full-set eval (Jul 17–22)
Standings above. Base > instruct-midtrain > teacher > WS-250.
The warm start degraded the model; the teacher’s assumed headroom was a pool artifact.
▼
Held for direction
Credible levers: 35B cooperator scan (unscanned), shorter step budgets (200 ≫ 500),
or abandoning the warm-start premise per PLAN.md’s escalation clause.
Midtrain vs post-train framing
| stage | what | outcome |
|---|
| Midtraining (June) |
Two-phase curriculum SFT on bucketed CooperData: bucket B (~87M tok, coordination-present)
then bucket A (~92M tok, “exemplary”); sources: inter-agent coop trajectories, swechat
human↔agent (dominates A), solo2coop synthetic, comm4, game logs |
Three models (instruct-midtrain, midtrain-sft, midtrain);
none beats base on coop gates or full-set eval |
| Post-training (July) |
OPSD program: prompt conditioning → success-gated teacher imitation (TITO SFT) →
(planned) score-based OPSD |
Conditioning falsified; imitation at this scale falsified; OPSD proper never entered —
entry gates never passed |
Teacher’s privileged information (score-based OPSD design)
The teacher re-scores the student’s own sampled tokens; the only context difference is a private
brief appended to the first system message: (1) cross-agent visibility — the peer’s identity
and public task (each agent normally sees only its own feature) — and (2) static process
directives (purposeful messaging, commit/push + exact-SHA handoffs, inspect peer diffs before
relying on them, never fabricate). Fail-closed exclusions: gold patches, hidden tests,
evaluator outcomes. Privilege is scope + guidance, never ground truth.
Communication-health evaluation (2026-07-22)
Raw routed-message counts reward the planning-loop attractor, so
scripts/eval_comm_health.py scores how agents communicate and how close their
message profile sits to the midtrain corpus. Reference column: 1,408 inter-agent conversations
(8,336 messages) from CooperBench/cooperdata-sft-midtrain. Model columns: the six
500-step scans.
| metric | corpus ref | base | WS-250 | WS-100 | instr-mt | mt-sft | mt |
|---|
| msgs / pair | 5.9 | 29 | 45 | 106 | 10 | 9.5 | 10 |
| median msg chars | 229 | 399 | 718 | 587 | 454 | 468 | 544 |
| near-duplicate ratio | 0.09 | 0.05 | 0.31 | 0.37 | 0.00 | 0.00 | 0.00 |
| division-of-work share | 0.31 | 0.52 | 0.64 | 0.45 | 0.28 | 0.53 | 0.46 |
| question share | 0.33 | 0.77 | 0.73 | 0.38 | 0.75 | 0.84 | 0.49 |
| answer rate | 0.67 | 0.62 | 0.42 | 0.24 | 0.37 | 0.59 | 0.70 |
| patch KB (4 pairs) | — | 3.0 | 2.3 | 0.0 | 0.8 | 0.0 | 1.2 |
- Midtraining produced healthier, more corpus-like communication: volume lands on the
corpus (~10 vs 5.9 msgs/pair; base 29, WS 45–106), zero near-duplicates, instruct-midtrain
matches corpus division share (0.28 vs 0.31).
- The warm-start models are degenerate communicators, not just verbose: near-dup
0.31–0.37, worst answer rates. Their extra messages are repetition.
- Base over-communicates non-degenerately: near-dup 0.05, best balance; its loop is
interrogative (0.77 question share), not repetitive.
- Communication health is decoupled from output: the healthiest communicators produce the
least code.
Method-level transfer: the coordination methods did NOT transfer
| marker | corpus | base | WS-250 | WS-100 | instr-mt | mt-sft | mt |
|---|
| opening division claim (1st msg per agent) | 0.25 | 0.25 | 0.13 | 0.13 | 0.29 | 0.00 | 0.00 |
| Intent/Approach protocol share | 0.02 | 0 | 0 | 0 | 0 | 0 | 0 |
| plan-first rate (harness) | — | 0 | 0 | 0 | 0 | 0 | 0 |
| follow-through rate (harness) | — | 0 | 0 | 0 | 0 | 0 | 0 |
No structural method was induced by any training recipe: plan-first and follow-through are zero
everywhere; two midtrain models lost base’s opening-division behavior. The corpus itself is
method-sparse (25% opening division, 2% explicit protocol) — distillation faithfully reproduced what
the data demonstrates, not what the bucket definitions suggest. Statistics transferred; methods
(conditional, task-state-dependent behavior) did not.
Conclusions
- The warm start degraded the model — WS-250’s planning/messaging behavior suppresses
completion (4/38 patches vs base 21/38). Imitation from 16 success-gated trajectories is
falsified at this scale, not merely unproven.
- The distillation headroom was a pool artifact — the teacher’s ~20% yield came from easier
f1/f2 pairs; on the matched pool it scores below base (1/32 vs 3/32).
- Untouched base is the strongest 9B starting point for future OPSD work.
- Step limit 200 beats 500 across models (base submits 21/38 at 200 vs 2/8 at 500) —
longer budgets feed the planning-loop attractor.
- Neither the warm start nor midtraining moved the coop bar at 9B — capability-bound
across every recipe tried (v9–v22, warm-start-tito, midtrain family).
Follow-ups & open questions
- F1 scan the unscanned 35B cooperator (
cooperator-qwen35-35b-a3b-sft) — tests
the capability-bound hypothesis directly.
- F2 step-budget sweep (100/200/300) on base — cheapest confirmed lever.
- F3 multi-seed the full-set eval before further conclusions (all rows are 1 seed).
- F4 fix the 3 crashing eval pairs (dottxt/1655, click/2800, jinja/1621).
- F5 decide on abandoning the warm-start premise per PLAN.md escalation.
- F6 push
feature/opsd-training — 36 commits incl. the evidence log exist only
on the experiment box.
- F7 confirm whether the superseded
evaluation_plan.md request
(cooperdata-v3-midtrain on CooperBench-flash) is still wanted.
- F8 gold-patch privilege: (a) hindsight scoring teacher — only useful once the student
samples real edits; (b) rollout hint to the corpus teacher, stripped before tokenization —
attacks the 16-trajectory bottleneck; (c) RL reward — unnecessary (hidden tests suffice).
Guardrails: train/eval task separation, new preregistered fail-closed conditioning mode, gold
never in student-visible tokens. Sequence after F1/F2.
- Data quality (D1–D7): contamination audit of the pre-gate midtrain corpus; manual
A-bucket exemplarity audit; composition (~90% of A tokens are human↔agent swechat);
observation-capping check; loss-vs-behavior probe; two-stage vs single-stage ablation;
forgetting probe on general coding.
- Q1 is the planning-loop attractor a prompt problem? (no prompt-ablation row exists) ·
Q2 why did the teacher’s patch/submit discipline not transfer while messaging style did? ·
Q3 is the falsified claim “imitation at this scale” or “imitation, period”? ·
Q4 checkpoint message-spam tracks proximity to the val-loss minimum — val loss may
anti-select for behavior ·
Q5 (answered) midtraining reduced message counts because the corpus itself averages
5.9 msgs/pair — the models landed on-distribution; base’s 29/pair was loop spam.
Caveats. One seed per condition throughout; behavior scans are 4 pairs / 8 agents; full-set
comparisons of base vs instruct-midtrain vs teacher (1–3 of 32 features) are within noise — only
WS-250’s degradation is clearly separated. Part of the corpus B-slice is protocol-generated
(“Intent:/Approach:”), so corpus-correspondence partly measures protocol adoption. The three
midtrain models predate the v2 contamination gate; their corpus overlap with eval tasks is
unaudited (D1).
Sources: coopertrain/opsd/evidence/EXPERIMENT_LOG.md (remote experiment box,
feature/opsd-training), coopertrain/opsd/PLAN.md,
coopertrain/verl/COOPERDATA.md, scripts/eval_comm_health.py.
Editable source: report/opsd-warmstart-summary-2026-07-22.md.