A research team tested six frontier language models — Claude Opus 5, GLM-5.3, GPT-5.6 Sol, Grok 4.6, Gemini 3.8 Flash, and Kimi K3 — in a two-agent game where a questioner must identify, among N Wikipedia lead paragraphs, a target known only to an answerer, using exactly log2(N) yes/no questions. Because both roles are played by the same model, the setup measures how well a system communicates with itself under information asymmetry. The experiment covered 408 games across document sets ranging from 4 to 1024 paragraphs, at a total API cost of $363.
The results show a clear gap for Claude Opus 5, which wins only 28 of its 68 games, compared with 45 to 56 wins for the other five models, whose performance remains closely bunched. Pooling these five models, win rate declines with set size in a near-perfect correlation (r = -0.973), fitting a simple model where success rate equals p raised to the power of log2(N), with p estimated at 0.928 per round.
Failures split roughly evenly between answer errors and discrimination failures at the final guess. A detailed review of the weakest model's unanimous mistakes found that 32 of 34 involved incorrect "No" answers about properties stated in the document's opening sentence, despite explicit instructions warning against defaulting to "No." By contrast, models almost never name a document that their own prior answers had already ruled out.
Information efficiency, estimated from answer balance, correlates strongly with win rate (r = +0.88). Only two models manage to extract a full bit of information per question, by adopting a partitioning strategy based on document titles — an approach absent below 32 candidates but used in roughly a quarter of questions above that threshold. Reasoning-token usage varies by a factor of 4.5 across models, with little relationship to success: reasoning traces grow longer as the candidate pool shrinks, without a matching improvement in reliability. The findings suggest that extended reasoning alone does not translate into efficient communication under information asymmetry, a relevant concern for the design of multi-agent AI systems.