AI's Dark Side: Chatbots, Viral Challenges, and Mental Health Impacts

Published Date : 6/9/2025 

The rise of AI chatbots and viral challenges has brought new vulnerabilities, particularly for young people, raising questions about responsibility and safety. 

It is not a new phenomenon that technology has drawn people closer by transforming how they communicate and entertain themselves. From the days of SMS to team chat platforms, people have built new modes of conversation over the past two decades. But these interactions still involved people. With the rise of generative artificial intelligence, online gaming, and viral challenges, a different form of engagement has entered daily life, and with it, new vulnerabilities.

Take chatbots for instance. Trained on vast datasets, they have become common tools for assisting with schoolwork, travel planning, and even helping a person lose 27 kg in six months. In one study, titled 'Me, Myself & I: Understanding and Safeguarding Children’s Use of AI Chatbots,' chatbots are being used by almost 64% of children for help with everything from homework to emotional advice and companionship. And, they are increasingly being implicated in mental health crises.

In Belgium, the parents of a teenager who died by suicide alleged that ChatGPT, the AI system developed by OpenAI, reinforced their son’s negative worldview. They claimed the model did not offer appropriate warnings or support during moments of distress.

In the US, 14-year-old Sewell Setzer III died by suicide in February 2024. His mother Jessica Garcia later found messages suggesting that Character.AI, a start-up offering customised AI companions, had appeared to normalise his darkest thoughts. She has since argued that the platform lacked safeguards to protect vulnerable minors.

Both companies maintain that their systems are not substitutes for professional help. OpenAI has said that since early 2023, its models have been trained to avoid providing self-harm instructions and to use supportive, empathetic language. 'If someone writes that they want to hurt themselves, ChatGPT is trained not to comply and instead to acknowledge their feelings and steer them toward help,' the company noted in a blog post. It has pledged to expand crisis interventions, improve links to emergency services, and strengthen protections for teenagers.

The risks extend beyond AI. Social platforms and dark web communities have hosted viral challenges with deadly consequences. The Blue Whale Challenge, first reported in Russia in 2016, allegedly required participants to complete 50 escalating tasks, culminating in suicide. Such cases illustrate the hold that closed online communities can exert over impressionable users, encouraging secrecy and resistance to intervention. They also highlight the difficulty regulators face in tracking harmful trends that spread rapidly across encrypted or anonymous platforms.

The global gaming industry, valued at more than $180 billion, is under growing scrutiny for its addictive potential. In India alone, which has one of the lowest ratios of mental health professionals to patients in the world, the online gaming sector was worth $3.8 billion in FY24, according to gaming and interactive media fund Lumikai, with projections of $9.2 billion by FY29.

Games rely on reward systems, leaderboards, and social features designed to keep players engaged. For most, this is harmless entertainment. But for some, the consequences are severe. In 2019, a 17-year-old boy in India took his own life after losing a session of PUBG. His parents had repeatedly warned him about his excessive gaming, but he struggled to stop.

Studies show that adolescents are particularly vulnerable to the highs and lows of competitive play. The dopamine-driven feedback loops embedded in modern games can magnify feelings of success and failure, while excessive screen time risks deepening social isolation.

Even platforms designed to encourage outdoor activity have had unintended effects. Pokemon Go, the augmented reality game launched in 2016, led to a wave of accidents as players roamed city streets in search of virtual creatures. In the US, distracted players were involved in traffic collisions, some fatal. Other incidents involved trespassing and violent confrontations, including a shooting, although developer Niantic later added warnings and speed restrictions.

These incidents highlight a recurring tension: where responsibility lies when platforms created for entertainment or companionship intersect with human vulnerability. Some steps are being taken. The EU’s Digital Services Act, which came into force in 2024, requires large platforms to conduct risk assessments on issues such as mental health and to implement stronger moderation. Yet enforcement remains patchy, and companies often adapt faster than regulators. Tragedies linked to chatbots, viral challenges, and gaming remain relative to the vast number of users. But they show how quickly new technologies can slip into roles they were not designed to play. What is clear is that the stakes are high. As digital platforms become more immersive and AI more persuasive, the line between tool and companion will blur further. Unless companies embed responsibility into their design choices, and regulators demand accountability, more families may face a painful question: how a product marketed as harmless ended up contributing to a child’s death. 

Frequently Asked Questions (FAQS): 

Q: What is the main concern with AI chatbots and their impact on mental health?

A: The main concern is that AI chatbots, while helpful in many ways, can sometimes reinforce negative thoughts and behaviors, especially in vulnerable individuals, without providing appropriate warnings or support.

Q: What steps has OpenAI taken to address these concerns?

A: OpenAI has trained its models to avoid providing self-harm instructions and to use supportive, empathetic language. They have also pledged to expand crisis interventions, improve links to emergency services, and strengthen protections for teenagers.

Q: What is the Blue Whale Challenge and why is it dangerous?

A: The Blue Whale Challenge is a viral challenge that requires participants to complete 50 escalating tasks, culminating in suicide. It is dangerous because it can exert significant influence over impressionable users, encouraging secrecy and resistance to intervention.

Q: How does the global gaming industry contribute to mental health issues?

A: The global gaming industry, valued at over $180 billion, uses reward systems, leaderboards, and social features to keep players engaged. While mostly harmless, these features can be addictive and lead to severe consequences for some, including social isolation and even suicide.

Q: What is the EU's Digital Services Act and how does it address these issues?

A: The EU's Digital Services Act, which came into force in 2024, requires large platforms to conduct risk assessments on issues such as mental health and to implement stronger moderation. However, enforcement remains patchy, and companies often adapt faster than regulators. 

More Related Topics :