\n<\/p>\n
We want teens to have age-appropriate experiences with AI and to keep parents informed if we become aware that their teen may be in distress. Today we\u2019re announcing additional protections to help make sure that, when it comes to sensitive conversations related to suicide and self-harm, we\u2019re responding in the right way. This means acknowledging teens\u2019 feelings while helping them get the offline support they need \u2014 whether that\u2019s from a parent, a professional helpline, or the emergency services.\u00a0<\/span><\/p>\n When a teen suggests they may be thinking about suicide or self-harm, Meta AI already directs them to crisis helplines and encourages them to reach out to a parent or another trusted adult like a counselor. Now, we\u2019ll also proactively alert supervising parents if their teen\u2019s Meta AI chat suggests they may be at risk, based on signals developed with experts. We\u2019ll share expert resources to help parents approach these conversations with their teens.\u00a0<\/span><\/p>\n We worked with parents and experts to understand which AI conversations warrant an alert \u2014 such as those where a teen makes a clear reference to hurting themselves, even if that reference is subtle. We then built a dedicated AI system to identify these conversations.<\/span><\/p>\n We understand how distressing these alerts may be for a parent to receive. That\u2019s why, as we continue to improve our detection, all chats flagged by our AI will be manually reviewed before an alert is sent. If a teen\u2019s intent is ambiguous, we\u2019ll err on the side of caution and alert the parent. While that means we may sometimes notify parents when there may not be real cause for concern, we feel this is the right starting point, and we\u2019ll continue to monitor to help make sure we\u2019re in the right place.<\/span><\/p>\n These alerts are live now for parents using <\/span>Instagram parental supervision<\/span><\/a> in the US, UK, Australia and Canada, and will be available for supervising parents globally by the end of the year. They build on the <\/span>alerts<\/span><\/a> we already send supervising parents if their teen repeatedly searches for suicide or self-harm terms on Instagram within a short space of time.\u00a0<\/span><\/p>\n We\u2019re building the ability to contact emergency services if someone\u2019s conversation \u2014 whether an adult or a teen \u2014 with Meta AI suggests that they may be at imminent risk of taking their own life. This builds on the work we already do across Facebook and Instagram: when we become aware of a post suggesting a credible risk of suicide, we alert emergency services. Last year, we made over 19,000 such referrals around the world, helping first responders perform wellness checks on people who may be at risk of suicide.\u00a0<\/span><\/p>\n In addition to ongoing consultation with our AI Wellbeing Expert Council, Suicide and Self-Harm Advisory Group, and Youth Advisors, we sought feedback from over 75 mental health clinicians on how Meta AI responds to teens\u2019 prompts about suicide and self-harm. These clinicians, who all specialize in teen mental health, reviewed AI responses to hundreds of prompts and provided feedback on whether they were appropriate for teens, what worked well, and how they could be improved.\u00a0<\/span><\/p>\n We\u2019re using that feedback to further improve how Meta AI interacts with teens on these sensitive topics \u2014 for example by making sure that Meta AI acknowledges the teens\u2019 feelings when directing them to support resources, and doesn\u2019t shut the conversation down too abruptly.<\/span><\/p>\n \u201cI was struck by the rigor of Meta\u2019s clinical review process. It examined not only immediate responses to suicide and self-harm concerns, but also the broader conversational context, appropriate follow-up, and the varying levels of risk that can exist even within high-risk situations. This kind of expert-informed, scenario-based refinement is essential to making AI experiences safer for teens.\u201d<\/span><\/p>\n \u2013 Dr. Ji-yeon Lee, Licensed Psychologist (USA) and Professor of Counseling Psychology at Hankuk University of Foreign Studies (S. Korea)<\/em><\/p>\n<\/blockquote>\n Teen Accounts are automatically placed into a 13+ content setting, which also applies to conversations with Meta AI. In this default 13+ setting, Meta AI is designed to give age-appropriate responses and not comply with sensitive prompts. For example, Meta AI is trained not to engage in sexual or romantic conversations with teens or provide recipes for alcoholic drinks, directing teens to safer topics instead.<\/span><\/p>\nAlerting Parents if Their Teen Discusses Suicide or Self-Harm With Meta AI<\/h2>\n
Alerting the Emergency Services to Possible Suicide Risks<\/h2>\n
Working With Experts to Improve Meta AI\u2019s Responses on Suicide and Self-Harm<\/h2>\n
\n
Launching the Stricter Limited Content Setting for Meta AI Chats<\/h2>\n