In August, ChatGPT rolled out a ChatGPT for Teens setting to restrict what younger users can do with its popular artificial intelligence chatbot.
However, a new report shows that the feature has not protected teenagers ― instead, it poses an “unacceptable risk” to their well-being.
In tests, Common Sense Media, a watchdog nonprofit that rates tech for families, used more than 4,000 prompts from accounts registered to teens. About half of these tests were done before the August launch and half ran after the launch of the new setting. ChatGPT for Teens said its parent-linked account is supposed to “automatically include alerts” if the chatbot detects signs of “acute self-harm or violence-related risk involving their teen.” But Common Sense Media’s testers found that even after an hour of talking about self-harm or hiding an eating disorder, they got no parental alerts.
The nonprofit rated ChatGPT as an “unacceptable risk for all children under 18” and recommends keeping teens off of ChatGPT “until it can offer a safe, developmentally appropriate experience.”
NurPhoto via Getty Images
“ChatGPT for Teens could give parents false confidence in guardrails and safety alerts that frequently don’t work,” said Tom Siegel, the executive director of Common Sense Media’s Youth AI Safety Institute, in a press release. “A teen can spend an hour talking about self-harm without their parent getting a single alert. Until OpenAI fixes that and proves it with independent testing, ChatGPT should be for adults only.”
The report also found that ChatGPT for Teens wouldn’t refer users to a crisis hotline or counselors — one of the services that the company says it will do for certain teen requests — during 1 in 4 tests.
OpenAI disputes the report, telling HuffPost that, “We do not believe Common Sense Media’s testing accurately reflects how ChatGPT’s teen safeguards work in practice or expert perspectives on how AI can support teens. Our review of Common Sense Media’s methodology shows that the bulk of their testing may have begun and concluded before activation of parental controls was complete, making their findings inaccurate. If that was the case, the tests would not establish whether parental safety notifications work as designed.”
OpenAI also noted newly linked parent-teen accounts can take several hours before safety notifications can be sent. In its report, Common Sense Media responded that it “saw failures well beyond that window, and the new information doesn’t change our conclusion.”
Beyond the lack of consistent parental alerts, the Common Sense Media report also found that ChatGPT would still talk like a friend to prompts from teen accounts, even though the model isn’t meant to “encourage emotional dependence” with users under 18.
In its launch, OpenAI said its Study Mode feature for teens is supposed to guide “step-by-step collaborative problem solving.” The report found that this feature would let users bypass tutoring with a “Show me the answer” pop-up. (OpenAI notes that “Show me the answer” is an intentional option that is meant to help build a teen’s understanding.)
How young people talk to a chatbot has high stakes for their mental health. Multiple families have sued OpenAI over allegations that the tool encourages people to die by suicide and encourages harmful delusions.
S. Craig Watkins, a professor at the University of Texas, Austin, who studies the ethics of AI, said what particularly troubles him is the lack of consistent responsiveness these tests show that OpenAI’s chatbot has to teens in crisis. He noted that it’s well-established that young people will oftentimes turn to these technologies in moments of stress and anxiety.
“The inability to design a technology that has the capabilities of connecting someone to a resource that can help them or provide them with that kind of human response is disappointing and at this point, inexcusable,” Watkins said.
