Trust and AI….
Sobering research published in PNAS recently.
A team at Dartmouth built an “autonomous synthetic respondent”, essentially an AI agent running from a 500-word prompt, and tested it against the full suite of data quality checks that survey researchers use to filter out bots and inattentive humans. Logic puzzles, instruction-following tasks, “reverse shibboleth” questions designed specifically to catch non-human actors.
The agent passed 99.8% of the time across 6,000 trials.
The bot could also correctly infer a study’s hypotheses and be instructed to produce responses consistent with a particular desired outcome. As Tom Stafford notes in his analysis of the paper, this means an army of bots could be engineered to make survey results appear as though the public favoured military action against a specific nation, or supported a particular electoral candidate, all while passing every quality check.
The numbers are striking: adding as few as 10 to 52 fake responses (very cheap to do) would have flipped the predicted outcome in each of the seven major national polls before the 2024 US election. The bots even work when initially programmed in Russian, Mandarin, or Korean, producing flawless English answers regardless.
Stafford raises a broader point that extends well beyond polling accuracy. Surveys, he argues, are part of the machinery by which society knows itself. They generate what Steven Pinker calls “common knowledge”, not just what people believe, but what people believe other people believe. This matters because we’re intensely social creatures who calibrate our own views partly in response to perceived consensus.
If that machinery becomes unreliable, it represents a powerful lever for manipulation.
Losing survey research as a reliable signal would leave us increasingly unable to distinguish genuine public sentiment from manufactured consensus.
I say “manufactured” deliberately. We’ve spent decades developing sophisticated methods to account for sampling bias and measurement error. We have far less experience accounting for adversarial contamination at scale, contamination that passes existing detection methods.
The full paper and Stafford’s commentary are both worth reading.
