OpenAI’s new social app is filled with terrifying Sam Altman deepfakes

In a video on OpenAI’s new TikTok-like social media app Sora, a never-ending factory farm of pink pigs are grunting and snorting in their pens — each is equipped with a feeding trough and a smartphone screen, which plays a feed of vertical videos. A terrifyingly realistic Sam Altman stares directly at the camera, as though he’s making direct eye contact with the viewer. The AI-generated Altman asks, “Are my piggies enjoying their slop?”

This is what it’s like using the Sora app, less than 24 hours after it was launched to the public in an invite-only early access period.

In the next video on Sora’s For You feed, Altman appears again. This time, he’s standing in a field of Pokémon, where creatures like Pikachu, Bulbasaur, and a sort of half-baked Growlithe are frolicking through the grass. The OpenAI CEO looks at the camera and says, “I hope Nintendo doesn’t sue us.” Then, there are many more fantastical yet realistic scenes, which often feature Altman himself.

He serves Pikachu and Eric Cartman drinks at Starbucks. He screams at a customer from behind the counter at a McDonald’s. He steals NVIDIA GPUs from a Target and runs away, only to get caught and beg the police not to take his precious technology.

People on Sora who generate videos of Altman are especially getting a kick out of how blatantly OpenAI appears to be violating copyright laws. (Sora will reportedly require copyright holders to opt out of their content’s use — reversing the typical approach where creators must explicitly agree to such use — the legality of which is debatable.)

“This content may violate our guardrails concerning third-party likeness,” AI Altman says in one video, echoing the notice that appears after submitting some prompts to generate real celebrities or characters. Then, he bursts into hysterical laughter as though he knows what he’s saying is nonsense — the app is filled with videos of Pikachu doing ASMR, Naruto ordering Krabby Patties, and Mario smoking weed.

This wouldn’t be a problem if Sora 2 weren’t so impressive, especially when compared with the even more mind-numbing slop on the Meta AI app and its new social feed (yes, Meta is also trying to make AI TikTok, and no, nobody wants this).

OpenAI fine-tuned its video generator to adequately portray the laws of physics, which make for more realistic outputs. But the more realistic these videos get, the easier it will be for this synthetically created content to proliferate across the web, where it can become a vector for disinformation, bullying, and other nefarious uses.

Aside from its algorithmic feed and profiles, Sora’s defining feature is that it is basically a deepfake generator — that’s how we got so many videos of Altman. In the app, you can create what OpenAI calls a “cameo” of yourself by uploading biometric data. When you first join the app, you’re immediately prompted to create your optional cameo through a quick process where you record yourself reading off some numbers, then turning your head from side to side.

Each Sora user can control who is allowed to generate videos using their cameo. You can adjust this setting between four options: “only me,” “people I approve,” “mutuals,” and “everyone.”

Altman has made his cameo available to everyone, which is why the Sora feed has become flooded with videos of Pikachu and SpongeBob begging Altman to stop training AI on them.

This has to be a deliberate move on Altman’s part, perhaps as a way of showing that he doesn’t think his product is dangerous. But users are already taking advantage of Altman’s cameo to question the ethics of the app itself.

After watching enough videos of Sam Altman ladling GPUs into people’s bowls at soup kitchens, I decided to test the cameo feature on myself. It’s generally a bad idea to upload your biometric data to a social app, or any app for that matter. But I defied my best instincts for journalism — and, if I’m being honest, a bit of morbid curiosity. Do not follow my lead.

My first attempt at making a cameo was unsuccessful, and a pop-up told me that my upload violated app guidelines. I thought that I followed the instructions pretty closely, so I tried again, only to find the same pop-up. Then, I realized the problem — I was wearing a tank top, and my shoulders were perhaps a bit too risqué for the app’s liking. It’s actually a reasonable safety feature, designed to prevent inappropriate content, though I was, in fact, fully clothed. So, I changed into a t-shirt, tried again, and against my better judgement, I created my cameo.

For my first deepfake of myself, I decided to create a video of something that I would never do in real life. I asked Sora to create a video in which I profess my undying love for the New York Mets.

That prompt got rejected, probably because I named a specific franchise, so I instead asked Sora to make a video of me talking about baseball.

“I grew up in Philadelphia, so the Phillies are basically the soundtrack of my summers,” my AI deepfake said, speaking in a voice very unlike mine, but in a bedroom that looks exactly like mine.

I did not tell Sora that I am a Phillies fan. But the Sora app is able to use your IP address and your ChatGPT history to tailor its responses, so it made an educated guess, since I recorded the video in Philadelphia. At least OpenAI doesn’t know that I’m not actually from the Philadelphia area.

When I shared and explained the video on TikTok, one commenter wrote, “Every day I wake up to new horrors beyond my comprehension.”