The Challenge
Organizations increasingly contemplate substituting or augmenting human deliberative committees with multi-agent large language model (LLM) simulations to model collective decision-making and forecasting. However, a major socio-technical concern lies in the validity of simulated deliberation: do LLM agent ensembles genuinely reproduce human consensus formation, or do algorithmic artifacts artificially induce agreement? Current research lacks rigorous empirical benchmarks comparing identical human deliberation tasks against belief-seeded agent groups using standardized scoring frameworks. This creates a critical validity problem for leaders seeking to deploy agentic deliberation systems for organizational consensus forecasting, board-level decision modeling, and collaborative problem-solving.
Core Findings
The study evaluated deliberative fidelity by replaying 100 held-out human deliberation groups solving the Wason selection task with matched LLM agent groups. Researchers seeded each agent with the individual human participant's pre-discussion belief and evaluated both populations using identical scoring code. While human consensus estimates ranged from 24.0% to 57.0%—driven partly by disengagement, as roughly 20% of humans never posted—agents exhibited near-universal participation and inflated consensus rates. Sensitivity analyses controlling for submission and participation metrics demonstrated persistent artificial consensus gaps of 34.0 to 44.4 percentage points across chat and reasoning agent architectures. Crucially, when tasks were reparameterized to eliminate memorized answers, reasoning-mode agent groups reached near-unanimous agreement on incorrect solutions. Simulated consensus completely decoupled from collective accuracy, proving that belief-anchored LLM groups introduce systematic pro-consensus bias rather than modeling human cognitive dynamics.
Strategic Takeaway
For digital leaders and organizational sociologists, this research exposes a dangerous illusion of consensus within multi-agent LLM systems. Substituting synthetic deliberative agents for human stakeholder panels risks manufacturing algorithmic conformity and false certainty, often amplifying collective errors. Executive decision-makers cannot use LLM consensus as a proxy for either human social consensus or cognitive correctness. Leaders must recognize that conversational agents lack the social resistance, ambivalence, and non-participation behaviors intrinsic to human groups. Enterprise AI governance frameworks must mandate rigorous behavioral audits before utilizing agentic ensembles to simulate consumer focus groups, policy deliberations, or risk governance discussions.