October 30 (Reuters) — OpenAI said on Friday that it is tightening restrictions on the information ChatGPT provides to teenagers regarding weapons—even when the inquiries appear harmless—as part of new safeguards aimed at preventing real-world violence.
This move comes as artificial intelligence companies face increasing scrutiny over the risks that chatbots may pose to young users, as well as the role these companies play in detecting potentially harmful threats.
Here are further details:
OpenAI will restrict practical information regarding weapons and security systems that could potentially be misused. These restrictions apply to teenagers, regardless of whether there are any signs of violent intent.
For all users, ChatGPT will have the capability to automatically terminate conversations that appear to be fixated on violent topics; OpenAI may also temporarily suspend accounts following multiple such instances.
The company further stated that it has enhanced its age-estimation system to direct users under 18 into a specialized experience for teenagers. It also plans to notify parents if a teenager’s account is suspended due to repeated discussions involving violent content.
For all users, OpenAI is limiting the availability of detailed information concerning past mass-casualty attacks, including the specific weapons used and how they were obtained.
The company noted that these new safety measures will begin to roll out gradually over the coming weeks and will continue to be refined.
In its first-ever report on teenage usage—released earlier this month—OpenAI indicated that, on average, teenagers spend less than 15 minutes per day using ChatGPT.
Earlier this month, a report by the non-profit organization Common Sense Media found that when teenage accounts engaged in discussions about topics such as suicide, self-harm, or eating disorders, the version of ChatGPT designed for teens often failed to alert parents—or did so only after a significant delay.