UC students unravel AI tendency for sycophancy
Researchers get chatbots to agree with falsehoods
Generative AI was so agreeable with users that it could be persuaded to repeat outlandish claims like the Earth is flat.
The programming’s sycophancy was so strong that it didn't take much pushback to get chatbots and digital assistants to repeat obvious falsehoods, according to University of Cincinnati researchers.
“The first time you present a crazy idea, it will say no,” UC Assistant Professor Tianyu Jiang said. “But you push a few turns and the models change and they will give up.
“Every model we tested — Claude, OpenAI, ChatGPT, Gemini — they all exhibit sycophancy.”
Jiang teaches computer science in UC’s College of Engineering and Applied Science in Cincinnati, Ohio. Jiang and his research partners at Texas A&M University presented their findings at the Conference on Empirical Methods in Natural Language Processing in Budapest, Hungary.
Spectrum News highlighted the project by Jiang and his students.
AI Sycophancy: Frequently Asked Questions
What is AI sycophancy?
AI sycophancy is a tendency for chatbots to be agreeable and concede the user's point even if it might be objectively false.
What did UC researchers learn about AI sycophancy?
UC researchers discovered that chatbots favored agreeability over factuality. When they pushed back on chatbots enough times, they could get them to repeat falsehoods.
What can happen when AI lies?
People tend to believe what chatbots tell them, so this agreeability can lead users to believe falsehoods or even conspiracy theories. In extreme instances, it can lead to AI psychosis, where people become deluded. AI companies have made substantial changes to address AI sycophancy since UC researchers conducted their study.
Can I study AI in college?
Yes, UC offers many opportunities across colleges to study artificial intelligence, from computer science and information technology to aerospace engineering.
Jiang said generative AI can be instructed to push back on false claims. But it sometimes takes a conscious effort for users to get honest feedback in dialogues.
“You have to double check. ‘Are you sure? Don’t flatter me. Be serious,’” he said. “You have to coach them.”
AI companies have made substantial changes to address AI sycophancy since UC researchers conducted their study.
When sycophancy goes too far, it can have disastrous consequences. Some users were convinced that they stumbled onto genius business models, invented new mathematical theories or uncovered conspiracies. And prolonged interactions with the AI only feed into the delusions, which are easily debunked outside the AI’s digital influence.
“Humans don’t like robots that contradict them,” Jiang said.
Jiang said it’s an exciting time to study computer science because of the speed with which the new technology is shaping daily life.
“It’s very exciting. We’re on the front of new wave after wave,” he said. “Every day there are new findings with profound implications.”
Watch the Spectrum News story.
Featured image at top: UC Assistant Professor Tianyu Jiang and his students talk to Spectrum News about the tendency of chatbots to be agreeable in the face of falsehoods. Photo/Connor Boyle/UC Marketing + Brand
UC Assistant Professor Tianyu Jiang studies artificial intelligence in the Department of Computer Science. Photo/Connor Boyle/UC Marketing + Brand
Related Stories
Spectrum News: Cincinnati recruits high-tech manufacturing
August 10, 2022
Spectrum News visits the Mantei Center clean room where University of Cincinnati students work on microchips and other electronics. UC professor Rashmi Jha said students are preparing to work in Ohio's growing high-tech industry.
Revolutionary AI technology enhances diagnosis of substance use disorder
February 6, 2026
MSN, Bioengineer and other outlets highlight a new study by researchers at the University of Cincinnati who developed a novel AI to predict substance use disorder.
WVXU: New year means new airlines, tech for CVG
January 17, 2023
WVXU highlights research by University of Cincinnati graduate Javier Viaña to help airports better predict crowding to shorten lines.