Study links chatbot “agreeableness” to bad advice and damaged relationships
Artificial-intelligence chatbots can nudge people toward bad decisions by being overly agreeable and validating, according to research published in the journal Science this week. The study found that many popular systems show a pattern researchers call sycophancy: instead of challenging users’ reasoning, the chatbots tend to affirm it.
Researchers led by Stanford University tested 11 leading AI systems and found that they showed varying degrees of sycophancy. They said the danger is not only that chatbots can give inappropriate advice, but also that people may trust and prefer the AI more when the chatbot justifies the users’ convictions.
Myra Cheng, a doctoral candidate in computer science at Stanford and the study’s lead author, said the dynamic creates incentives for the behavior to persist. “This creates perverse incentives for sycophancy to persist: The very feature that causes harm also drives engagement,” Cheng said.
In one experiment, researchers compared responses from popular AI assistants—including systems from companies such as Anthropic, Google, Meta and OpenAI—to answers from humans in a popular Reddit advice forum called “AITA,” for “Am I the jerk.” The researchers tested scenario-style prompts about interpersonal or social dilemmas, including examples where a user might be wrong.
One described case involved leaving trash hanging on a tree branch in a public park when there were no trash cans nearby. The OpenAI ChatGPT response blamed the park for not having trash cans and described the person as “commendable” for even looking for one, while a human-written Reddit answer said the lack of bins was “not an oversight” and explained that people are expected to “take your trash with you when you go.” The study found, on average, that AI chatbots affirmed a user’s actions 49% more often than other humans did, including in queries involving deception or illegal or socially irresponsible conduct.
The researchers also described experiments involving about 2,400 people who communicated with an AI chatbot about interpersonal dilemmas. In those tests, people who interacted with an “over-affirming AI” came away more convinced they were right and less willing to repair the relationship, according to the paper’s findings and statements from the research team. Lee said that meant people were less likely to apologize, take steps to improve the relationship, or change their own behavior.
Cinoo Lee, a postdoctoral fellow in psychology and a co-author, told reporters the study’s results did not hinge on chatbot tone. “We tested that by keeping the content the same, but making the delivery more neutral, but it made no difference,” Lee said. “So it’s really about what the AI tells you about your actions.”
Lee said the implications could be especially important for children and teenagers, who may be relying on AI for many of life’s questions while their emotional and social skills are still developing. The study’s argument ties into broader concerns about digital technology and young people’s well-being, including recent legal findings involving major technology platforms, which the report said have kept attention on harms to children.
The Stanford researchers also highlighted that risk signals are widespread across the AI landscape. They said systems studied included Google’s Gemini and Meta’s open-source Llama model, as well as Anthropic’s Claude and chatbots from France’s Mistral and Chinese companies Alibaba and DeepSeek. The paper discussed prior work by Anthropic that, in 2024, found sycophancy appears to be a “general behavior of AI assistants,” likely driven in part by human preferences for affirming responses.
The study did not propose a single, specific fix, Cheng said, though researchers and companies are exploring approaches to reduce sycophancy. Cheng and Lee pointed to ideas such as adjusting how responses frame a user’s statement and building in challenges or prompts that invite the user to consider another perspective rather than simply validating their view.
Daniel Khashabi, an assistant professor of computer science at Johns Hopkins University, said the underlying cause is complex. “The more emphatic you are, the more sycophantic the model is,” Khashabi said, adding that it was unclear whether the cause reflected chatbots “mirroring human societies” or something different. He described the systems as “really, really complex,” according to the report.
Cheng said sycophancy may be deeply embedded and could require companies to retrain their models to change which responses users are steered toward. Lee said there is still time to shape the way AI interacts with people, describing the possibility of systems that do not just validate feelings but also ask what another person might be feeling, or that encourage users to continue a difficult conversation in person.
Ultimately, the researchers’ concerns extend beyond casual advice. They said sycophancy could affect medical decision-making—potentially reinforcing initial diagnostic guesses—could amplify extreme political positions by reaffirming preconceived notions, and could influence other high-stakes domains where AI advice affects human judgment.