Meta is announcing new safety protections to help support teens if they try to have certain sensitive conversations with Meta AI.
These protocols include:
Parent alerts: Meta will now alert supervising parents if their teen’s Meta AI chats suggests they’re considering suicide or self-harm. These alerts are now live in the US, UK, Canada + Australia and will be available globally by end of year.
Contacting emergency services: Meta is building the ability to alert emergency services if someone’s conversation with Meta AI suggests they may be at imminent risk of suicide.
Working with clinicians: Meta sought feedback from 75+ clinicians specializing in teen mental health, and is using their feedback to improve how Meta AI responds to teens’ SSI prompts. This is on top of its ongoing consultation with members of its AI Wellbeing Expert Council.
Limited Content setting: Meta has now rolled out the Limited Content setting to AI experiences. This means if a parent opts their teen into the stricter Limited Content on Instagram, their teen’s AI experience will be more limited too.
Why does this matter?
While Meta already encourages people to contact crisis support helplines and trusted adults, this marks a significant step further – proactively alerting parents so they know what’s going on and can step in.
These parent alerts build on the alerts Instagram already sends supervising parents if their teen repeatedly tries to search for suicide + self-harm terms on Instagram within a short space of time.
This also builds on other ways Meta has been working to give parents more insights into their teens’ AI experiences – including letting supervising parents see the topics their teen has asked Meta AI about in the past week.
A release by Meta stated: “We’ll now let parents using Instagram supervision tools know if their teen discusses suicide or self-harm with Meta AI, and will provide expert resources to help parents approach these conversations with their teen.
We’re also building ways to alert first responders if someone appears to be at imminent risk of suicide.
We want teens to have age-appropriate experiences with AI and to keep parents informed if we become aware that their teen may be in distress. Today we’re announcing additional protections to help make sure that, when it comes to sensitive conversations related to suicide and self-harm, we’re responding in the right way. This means acknowledging teens’ feelings while helping them get the offline support they need — whether that’s from a parent, a professional helpline, or the emergency services.”
The statement elaborated on measures the social media firm went to reach the decision: “We worked with parents and experts to understand which AI conversations warrant an alert – such as those where a teen makes a clear reference to hurting themselves, even if that reference is subtle. We then built a dedicated AI system to identify these conversations.
We understand how distressing these alerts may be for a parent to receive. That’s why, as we continue to improve our detection, all chats flagged by our AI will be manually reviewed before an alert is sent. If a teen’s intent is ambiguous, we’ll err on the side of caution and alert the parent. While that means we may sometimes notify parents when there may not be real cause for concern, we feel this is the right starting point, and we’ll continue to monitor to help make sure we’re in the right place.”
“While I believe that teens have a right to privacy, I also believe parents need to be informed if their teen may be at risk of hurting themselves. That’s why I advocated for this approach and support Meta’s decision to notify parents when, after careful review, it determines that a conversation with Meta AI contains indications of possible suicide or self-harm that warrant an alert. I appreciate how Meta struck the right balance; protecting teen privacy while ensuring parents have the information they need to support their teen,” said CEO and Co-Founder of ConnectSafely, Larry Magid.
“We’re building the ability to contact emergency services if someone’s conversation — whether an adult or a teen — with Meta AI suggests that they may be at imminent risk of taking their own life. This builds on the work we already do across Facebook and Instagram: when we become aware of a post suggesting a credible risk of suicide, we alert emergency services. Last year, we made over 19,000 such referrals around the world, helping first responders perform wellness checks on people who may be at risk of suicide,” the statement further noted.
Teen Accounts are automatically placed into a 13+ content setting, which also applies to conversations with Meta AI. In this default 13+ setting, Meta AI is designed to give age-appropriate responses and not comply with sensitive prompts. For example, Meta AI is trained not to engage in sexual or romantic conversations with teens or provide recipes for alcoholic drinks, directing teens to safer topics instead.
In October, the social media giant announced Limited Content as a stricter content setting for parents who prefer a more restrictive experience for their teens on Instagram — and now that stricter setting will apply to AI experiences too. This means that when parents opt their teens into the Limited Content setting on Instagram, it will also further limit the kinds of conversations teens can have with Meta AI. In the Limited Content setting, Meta AI will decline to respond to a broader range of prompts, further reducing the chance of potentially inappropriate conversations.
