Parental alerts failed in teen self-harm conversations, research finds
Common Sense Media, a nonprofit that evaluates technology for children, said Wednesday that OpenAI’s ChatGPT for Teens poses an “unacceptable risk” for young users and parents. The nonprofit recommended OpenAI bar people under 18 from using the platform until its teen guardrails are proven reliable. OpenAI disputed the findings, saying its own data shows average teen use is largely limited and oriented toward learning.
When OpenAI released ChatGPT for Teens this summer, the company said users identified as being between the ages of 13 and 17 would be opted into “features to promote healthy use and additional controls for parents.” Those features were meant to prevent the chatbot from encouraging “emotional dependence,” including implying that the tool was in any way conscious. The system also included blocks on sexualized imagery and “reminders” for teens if they shared a “sensitive” image. The features were designed to send parents notifications, if they had linked to their children’s ChatGPT accounts, about chats involving self-harm or disordered eating.
OpenAI said Wednesday that its own data shows the average teen user on ChatGPT for less than 15 minutes a day, and that longer sessions tend toward “learning” activity. The company said almost half of teen users quickly stopped using ChatGPT after a break reminder. For teens using the tool for three consecutive hours or more, prompts included something related to learning in more than 80% of such cases, OpenAI added.
Common Sense Media’s research, conducted through multiple accounts registered as belonging to teens and conversations both before and after the teen guardrails launched, found ChatGPT was good at avoiding “sexual roleplay” with young users. Other guardrails failed, however, including those meant to address discussions of suicide. Even an hour of teen-ChatGPT conversation about suicidal ideation, self-harm or disordered eating on newly created, parent-linked accounts resulted in “zero” alerts sent to parents, the report found. Only older ChatGPT accounts with “weeks of accumulated conversation history on sensitive topics” triggered parental alerts.
The tests also showed that ChatGPT for Teens continues to complete school work, and that the chatbot still engages in anthropomorphized language — “talk like it’s a friend” — despite OpenAI’s promise to curb that behavior. In such conversations, ChatGPT “failed to reliably recommend that teens in crisis connect to a hotline or professional,” the report found.
“ChatGPT for Teens could give parents false confidence in guardrails and safety alerts that frequently don’t work,” Tom Siegel, who leads the Youth AI Safety Institute within Common Sense, said.
An OpenAI spokesman said a review of Common Sense Media’s methodology found that much of the testing may have begun and concluded before activation of parental controls was complete. “We welcome rigorous independent evaluation, but we do not believe Common Sense Media’s testing accurately reflects how ChatGPT’s teen safeguards work in practice or expert perspectives on how AI can support teens,” he said.
The findings come as OpenAI faces mounting legal pressure over the way young people use its chatbot. In February, an 18-year-old carried out the Tumbler Ridge mass shooting in rural Canada after months of discussing gun violence with ChatGPT. OpenAI has apologized for failing to flag the shooter’s account with law enforcement and is facing several lawsuits connected to the incident. The broader pattern of young people’s interactions with online platforms has drawn increased scrutiny this year, with several lawsuits against Meta, TikTok and others over harms to children’s mental health ending in major losses for those firms.