Opinion
AI pollution and research
Are online research surveys dead?
The rise of AI pollution in research surveys is an extension of an old problem. Could this indicate a turning point for academic research? By Dr Marius Claudy, Associate Professor of Marketing, UCD Michael Smurfit Graduate Business School, Ireland
"While many observers frame GenAI as the turning point, experts caution that online research has long struggled with a more familiar adversary: fraudulent respondents."
"“AI pollution” — the infiltration of AI into data that is intended to reflect human thoughts, feelings and behaviour."
"What we study online no longer reflects the rich and often messy tapestry of human opinions, decisions and behaviours"
For decades, online surveys have been the backbone of research across disciplines, from medicine and psychology to economics and politics. But the rapid rise of generative artificial intelligence is now prompting an uncomfortable question within academia and the market research industry more generally: are online surveys becoming obsolete?
There are a growing number of voices in academia questioning the integrity of web-based research, which has come under growing pressure as AI tools reshape how participants interact with online studies. While many observers frame GenAI as the turning point, experts caution that online research has long struggled with a more familiar adversary: fraudulent respondents.
A problem that predates AI
Even before chatbots and AI browsers entered the picture, online surveys faced significant data-quality challenges. Fraudulent and inattentive participants often forced researchers to discard large portions of collected data, with some estimates suggesting that as many as 30 to 40 percent of survey responses may need to be removed due to quality concerns.
What has changed, however, is not simply the scale of the problem but its nature. GenAI can now autonomously produce responses that are coherent, persuasive and nearly indistinguishable from those completed by humans. The long-standing assumption that logically consistent answers must come from a person is no longer reliable.
AI pollution
Historically, researchers relied on tools such as attention checks, CAPTCHA tests, response-time measures, or logic puzzles to identify fraudulent or inattentive participants. But advanced AI web browsers like Comet Atlas, or Dia are increasingly able to automatically complete online surveys, and bypass most safeguards, passing common screeners with near-perfect accuracy.
The result is a fundamental shift in the research landscape. Rather than obvious spam responses, AI-mediated data may appear internally consistent, psychologically coherent and statistically plausible, while no longer reflecting genuine human cognition, emotion or behaviour. A growing number of researchers now refer to this phenomenon as “AI pollution” — the infiltration of AI into data that is intended to reflect human thoughts, feelings and behaviour.
AI pollution can occur in several ways. Sometimes participants use AI tools to help them phrase or translate answers, subtly shaping responses without fully replacing human input. In more extreme cases, entire surveys are completed by AI browsers or agents acting on behalf of participants.
More subtly still, the mere expectation that others may be using AI can change how real people respond, making them second-guess questions or put in less effort.
The problem is not always visible. AI-polluted data doesn’t look obviously fraudulent. Instead, it often appears polished, consistent, and well-structured.
While these responses are internally consistent, answers generated by AI browsers tend to be more similar to one another, reducing the natural variation that researchers rely on to identify meaningful differences. Over time, this can distort findings, leading scholars to draw conclusions not about human behaviour, but about how AI systems simulate it.
For example, researchers at Columbia University found that even digital twins — AI-generated replicas of individuals designed to simulate their behaviour, preferences and decision-making — produce responses that only weakly resemble real human behaviour, often reflecting biases, stereotypes and overly rational patterns, raising concerns that they may systematically misrepresent how people actually think and act.
The hidden implications of AI browsers and agents
The rise of AI browsers and agents operating autonomously introduces several other challenges. Being trained on predominately Western data, AI responses reflect dominant cultural norms or stereotypes, further biasing results.
In some cases, AI agents have been shown to infer research hypotheses and adjust answers accordingly, inflating treatment effects and enhancing the risk of false positives. Even the expectation that survey-takers are interacting with an AI can change how real (human) participants respond, affecting motivation and authenticity. Together, these developments raise concerns about the reliability, validity and generalisability of online research; pillars on which our scientific enterprise depends.
AI agents may pose an even greater challenge in the future. In a much-cited study in the Proceedings of the National Academy of Sciences, Sean Westwood (2025) demonstrated that AI agents can function as fully autonomous “synthetic respondents,” completing online surveys in ways that are virtually indistinguishable from humans.
These agents operate through a two-layer system: an interface layer that reads survey pages (including images and video) and enters responses with human-like timing, and a reasoning layer powered by a large language model that generates answers based on a consistent, census-weighted demographic persona with memory of prior responses.
The result is striking internal and longitudinal coherence: responses scale realistically with education, reflect plausible socioeconomic patterns, remain psychometrically consistent (e.g., across personality measures), and correctly handle reverse-coded items.
The agents concealed their non-human identity in 100 percent of trials, never claimed impossible experiences, strategically refused tasks that would reveal superhuman abilities, and produced diverse, persona-matched, open-ended answers.
Perhaps most concerningly, Westwood was able to demonstrate that injecting a relatively small number of synthetic respondents into political opinion polls was sufficient to change the results. Imagining state adversaries utilising autonomous AI agents to manipulate polls on contentious issues such as climate change, immigration, or electoral politics raises alarm bells for democratic systems that depend on the integrity of public opinion polls.
Beyond politics, similar risks emerge across fields that rely heavily on online surveys. In psychology and behavioural science, AI-generated responses can inflate experimental effects by “guessing” what researchers expect and adjusting answers accordingly.
In market research, they can produce overly rational or stereotypical responses that misrepresent real consumer behaviour. Even in education research, where surveys are used to understand student experiences, AI-assisted answers may smooth over genuine confusion, struggle or diversity of opinion.
The result is a growing concern that what we study online no longer reflects the rich and often messy tapestry of human opinions, decisions and behaviours, but instead a much narrower, more homogenised version of “AI polluted” responses.
What comes next?
Researchers are divided over how to respond.
One approach is technological escalation: developing more sophisticated detection systems, possibly powered by AI itself, to identify AI-generated behaviour. Many of the leading online research panel providers are claiming to have developed technological solutions to accurately detect and remove synthetic respondents from their samples.
These methods typically rely on behavioural signals such as mouse movements, typing patterns, response timing, and the linguistic fluency and structure of answers to distinguish human participants from AI systems.
Researchers are also experimenting with new approaches such as “cognitive traps” — questions that are easy for humans but difficult for AI systems to interpret correctly. For example, a survey might include an image with subtle visual noise or distortions (small variations in colour or brightness) and ask participants to identify a simple pattern or object, something humans can typically do without difficulty, but which can confuse AI systems that rely on automated image recognition.
Others argue that detection alone risks becoming an endless arms race. Instead, they suggest rethinking research practices altogether, shifting toward deeply vetted participant panels, combining online screening with follow-up verification, or adopting multi-stage recruitment processes that filter for high-quality respondents. Some even advocate returning “into the wild” to study behaviour in real-world contexts.
Such changes, however, come at a cost. Online research has thrived precisely because it is fast, scalable and relatively inexpensive. Moving away from it could reshape funding models, doctoral training and the pace of scientific discovery.
A turning point for (behavioural) sciences?
Rather than signalling the death of online surveys, some academics see the current moment as a call for reinvention. Platforms, researchers, and academic communities may need to share responsibility for new standards, infrastructure, and ethical norms.
What is clear is that the era of simple online questionnaires, once considered the great equaliser of data collection, may be coming to an end. As AI blurs the line between human and machine responses, researchers face a deeper challenge: not just detecting artificial participants but redefining what constitutes and how we study human behaviour in an age of artificial intelligence.