Low-Quality Social Media Content Impacts AI Chatbots Performance

Published Date : 31/10/2025 

A recent study reveals that AI chatbots trained on low-quality social media content exhibit reduced reasoning capabilities and increased inaccuracies. The research, conducted by a team at the University of Texas at Austin, highlights the importance of data quality in training large language models (LLMs). 

Artificial intelligence (AI) chatbots are worse at retrieving accurate information and reasoning when trained on large amounts of low-quality content, particularly if the content is popular on social media, finds a preprint posted on arXiv on 15 October.

In data science, good-quality data need to meet certain criteria, such as being grammatically correct and understandable, says co-author Zhangyang Wang, who studies generative AI at the University of Texas at Austin. But these criteria fail to capture differences in content quality, he says.

Wang and his colleagues wanted to see the effects of large language models (LLMs) trained on low-quality data — defined as short, popular social-media posts, or those containing superficial or sensationalist content. They looked at how these data affected model reasoning, retrieval of information from long inputs, the ethics of responses, and model personality traits.

The team reports that models given low-quality data skip steps in their reasoning process — or don’t use reasoning at all — resulting in the model providing incorrect information about a topic, or when the authors presented a multiple-choice question, the model would pick the wrong answer. In data sets with a mix of junk and high-quality data, the negative effect on reasoning increased as the proportion of junk data increased. The work has not been peer-reviewed.

The findings support a long-held tenet of AI: the importance of data quality, says Mehwish Nasim, an AI researcher at the University of Western Australia in Perth. “Even before people started to work on large language models, we used to say that, if you give garbage to an AI model, it’s going to produce garbage,” she adds.

Wang and his colleagues used one million public posts on the social-media platform X from an existing database to train open-source models: Llama 3, an LLM from tech firm Meta in Menlo Park, California, and three versions of Qwen, developed by Alibaba in Hangzhou, China. Qwen is a reasoning model, like DeepSeek's R1 model and OpenAI's o1, meaning it is designed to produce reasoning steps to arrive at an answer to a user query. Llama, however, is an instruction-tuned language model and its reasoning ability is less advanced.

To determine the model’s personality traits, the team used psychology questionnaires. Before training on junk data, Llama exhibited agreeableness, extroversion, conscientiousness, openness, and a bit of narcissism, say the authors. But as Llama was fed more junk data, its negative traits were amplified, and psychopathy emerged, according to one of the questionnaires.

To adapt and improve models over time, researchers can adjust the prompt instructions. When the team tried doing this for a Llama model trained exclusively on junk data, they found that it only partially improved performance, as did increasing the amount of non-junk data used for training. The model also continued to skip steps when the team tried to encourage it to reflect on and fix failures in its reasoning, suggesting that different methods to mitigate the effect of junk data might be needed. 

Frequently Asked Questions (FAQS): 

Q: What is the main finding of the study?

A: The main finding of the study is that AI chatbots trained on low-quality social media content exhibit reduced reasoning capabilities and increased inaccuracies in their responses.

Q: How did the researchers define low-quality data?

A: Low-quality data was defined as short, popular social-media posts or those containing superficial or sensationalist content.

Q: What models were used in the study?

A: The study used Llama 3 from Meta and three versions of Qwen from Alibaba, along with one million public posts from the social-media platform X.

Q: What are the personality traits observed in the Llama model before and after training on junk data?

A: Before training on junk data, Llama exhibited traits like agreeableness, extroversion, conscientiousness, openness, and a bit of narcissism. After training on junk data, its negative traits were amplified, and psychopathy emerged.

Q: What methods did the researchers use to try to improve the model's performance?

A: The researchers tried adjusting prompt instructions and increasing the amount of non-junk data used for training, but these methods only partially improved performance. 

More Related Topics :