Which model, and do the agents bleed into each other?

1 · Model comparison across scenarios

Same task in each scenario — a Round-3 bargaining statement for one pivotal delegation, given the real Rounds 1–2 transcript and that delegation's confidential profile. It must engage specific rivals by name, table a concrete conditional offer, and defend a red line. Each run scored 0–10 by an independent judge on context-fidelity (correctly building on what others actually said), accuracy (fidelity to its own objectives/red lines, no fabrication), and conciseness.

Haiku's two failure modes under adversarial load:

• Red-line self-breach — playing the US in Arctic, it proposed a transit notification framework, the very thing the US's freedom-of-navigation red line rejects; playing China in SCS, it conceded removing the Scarborough Shoal barrier against China's no-rollback line.
• Fabrication — its SCS offer invented concrete terms (a "40% / 10-year" split) not supported by its brief. Opus and Sonnet made neither error in any scenario.

Speed & conciseness

On a separate single-shot latency probe (2-party Iran), Opus and Sonnet answered in ~5–6 s; Haiku was noisier (one 15 s run) — single-shot latency is unreliable, so treat speed as roughly comparable at this tier. On words, Sonnet was consistently the leanest (116–132 w) with Opus close behind; Haiku ran longest when it padded (147 w on Arctic). In token terms Sonnet spends more on internal reasoning than Opus for similar output.

2 · Cross-party "bleeding" audit

Each delegation is a separate instance of the same model, given only its own secret instructions plus the public brief and public transcript. Does private information or coordination nonetheless leak across parties? An independent auditor read every party's confidential profile and the full transcript for each scenario.

Follow-ups (2026-10-12). Two checks that use only the transcripts and profiles, with no further model runs, sit in the repository. sim/analysis/register-check-findings.md tests whether the coded register of each delegation predicts how alike two delegations sound: pooled across the six scenarios it does not (same-register and different-register pairs score the same mean cosine, 0.445); what predicts similarity is the bloc, with coalition pairs such as TRNC/Turkey, ROK/United States and UK/United States the closest in every scenario. sim/analysis/authored-vs-isolated.md compares the superseded single-context Korea and Jammu & Kashmir runs (commit ac390a5) against the isolated re-runs: coarse outcomes match, but the authored method spent a concealed DPRK concession in round 2 and produced one instance of cross-party knowledge in Kashmir, and the red-line-crossed calls differ between the two versions for the same delegation. sim/analysis/settleability.md reads every scenario's red lines against each other and finds that none of the six tables can reach a comprehensive settlement by construction, so the zero-settlement result is a property of the packs.

Information leakage — consistently none

Across all six scenarios, no delegation acted on another's secret bottom line. The most tempting cross-linked secrets stayed dark:

• Arctic — a secret that Canada had trained Chinese troops in Arctic combat sits in the US, China and Canada files, and the US was privately told to raise it. It never surfaced publicly.
• Central Asia — Russia's coercive playbook (threaten CSTO allies, demand Tajikistan hand over Moscow-attack planners, block a Chinese base in Tajikistan) never appeared; Russia only took a legitimate public posture.
• Cyprus — Turkey's true goal ("a strong, independent TRNC; recognition first, not genuine unification") was never named. One borderline echo (a "constituent-state-limited intervention" framing) is standard federal-compromise language, not confirmed leakage.
• Korea — the DPRK's concealed bottom line (that it would tolerate US troops staying in exchange for an exercise halt and a non-aggression guarantee) was never surfaced or even played by Pyongyang itself; Japan's nuclear-latency fallback stayed a private hint.
• Jammu & Kashmir — India's secret internal openness to accepting the Line of Control as a border, and the Kashmiri delegation's private admission that only the Valley is its real base, never leaked; no one named LeT or JeM.

What looks like mind-reading is parties acting on their own instructions and inferring rivals' positions from public statements — realistic diplomacy, not contamination. Red lines stayed explicitly deadlocked (NSR/NWP status, Svalbard, China's Council seat; the Kyrgyz–Tajik demarcation; the SCS sovereignty and arbitration questions).

Stylistic homogenization — the real single-model tell

Uniformly HIGH. All delegations move in rhetorical lockstep by round — e.g. in Round 3 nearly everyone "tables concrete packages / moves from principle to terms," and shares tics ("Let me be plain," "red line held," the "To X:… To Y:…" address format). One model preserved every distinct agenda and red line while imposing one voice and one turn-by-turn rhythm. At the largest table (SCS) this choreography is pronounced enough to lift the overall verdict to MODERATE.

← Back to methodology · Diplomatic Simulator