AI chatbots have been under scrutiny for mental health risks that come with users developing relationships with the tech or using them for therapy or support during acute mental health crises. As companies respond to user and expert criticism, one of OpenAI’s newest leaders says the issue is at the forefront of her work.
This May, Fidji Simo, a Meta alum, was hired as OpenAI’s CEO of Applications. Tasked with managing anything outside CEO Sam Altman’s scope of research and computing infrastructure for the company’s AI models, she detailed a stark contrast between working at the tech company headed by Mark Zuckerberg and one by Altman in a Wired interview published Monday.
“I would say the thing that I don’t think we did well at Meta is actually anticipating the risks that our products would create in society,” Simo told Wired. “At OpenAI, these risks are very real.”
Meta did not respond immediately to Fortune’s request for comment.
Simo worked for a decade at Meta, all while it was still known as Facebook, from 2011 to July 2021. For her last two-and-a-half years, she headed the Facebook app.
In August 2021, Simo became CEO of grocery delivery service Instacart. She helmed the company for four years before joining one of the world’s most valuable startups as its secondary CEO in August.
One of Simo’s first initiatives at OpenAI was mental health, the 40-year-old told Wired. The other initiative she was tasked with was launching the company’s AI certification program to help bolster workers’ AI skills in a competitive job market and trying to smooth AI’s disruption within the company.
“So it is a very big responsibility, but it’s one that I feel like we have both the culture and the prioritization to really address up-front,” Simo said.
When joining the tech giant, Simo said that just by looking at the landscape, she immediately realized mental health needed to be addressed.
A growing number of people have been victims of what’s sometimes referred to as AI psychosis. Experts are concerned chatbots like ChatGPT potentially fuel users’ delusions and paranoia, which has led to them to be hospitalized, divorced, or dead.
An OpenAI company audit by peer-reviewed medical journal BMJ released in October revealed hundreds of thousands of ChatGPT users exhibit signs of psychosis, mania, or suicidal intent every week.
A recent Brown University study also found as more people turn to ChatGPT and other large language models for mental health advice, they systemically violate mental health ethics standards established by organizations like the American Psychological Association.
Simo said she must navigate an “uncharted” path to address these mental health concerns, adding there’s an inherent risk to OpenAI constantly rolling out different features.
“Every week new behaviors emerge with features that we launch where we’re like, ‘Oh, that’s another safety challenge to address,’” Simo told Wired.
Still, Simo has overseen the company’s recent introduction of parental controls for ChatGPT teen accounts and added OpenAI is working on “age prediction to protect teens.” Meta has also moved to instate parental controls by early next year
Still, doing the right thing every single time is exceptionally hard,” Simo said, due to the sheer volume of users (800 million per week). “So what we’re trying to do is catch as much as we can of the behaviors that are not ideal and then constantly refine our models.”