By its very nature, TikTok is harder to moderate than many other social media platforms, according to Cameron Hickey, project director at the Algorithmic Transparency Institute. The brevity of the videos, and the fact that many can include audio, visual, and text elements makes human discernment even more necessary when deciding whether something violates platform rules. Even advanced artificial intelligence tools, like using speech-to-text to quickly identify problematic words, is more difficult “when the audio that you’re dealing with also has music behind it,” says Hickey. “The default mode for people creating content on TikTok is to also embed music.”
That becomes even more difficult in languages other than English.
“What we know generally is that platforms do best at the work of addressing problematic content in the places where they are based or within the languages in which the people who created them speak,” says Hickey. “And there are more people making bad stuff than there are people at these companies trying to get rid of the bad stuff.”
Many pieces of disinformation Madung found were “synthetic content,” videos created to look like they might be from an old news broadcast, or they use screenshots that appear to be from legitimate news outlets.
“Since 2017, we’ve noticed that there was a burgeoning trend at the time to appropriate the identities of mainstream media brands,” says Madung. “We are seeing rampant usage of this tactic on the platform, and it seems to do exceptionally well.”
Madung also spoke with former TikTok content moderator Gadear Ayed to get a better understanding of the company’s moderation efforts more broadly. Although Ayed did not moderate TikToks from Kenya, she told Madung that she was often asked to moderate content in languages or contexts she was not familiar with, and would not have had the context to tell whether a piece of media had been manipulated.
“It’s common to find moderators being asked to moderate videos that were in languages and contexts that were different from what they understood,” Ayed told Madung. “For example, I at one time had to moderate videos that were in Hebrew despite me not knowing the language or the context. All I could rely on was the visual image of what I could see but anything written I couldn’t moderate.”
A TikTok spokesperson told WIRED that the company prohibits election misinformation and the promotion of violence and is “committed to protecting the integrity of [its] platform and have a dedicated team working to safeguard TikTok during the Kenyan elections.” The spokesperson also said that it works with fact-checking organizations, including Agence France-Presse in Kenya, and plans to roll out features to connect its “community with authoritative information about the Kenyan elections in our app.”
But even if TikTok removes the offending content, Hickey says that may not be enough. “One person can remix, duet, reshare someone else’s content,” says Hickey. That means that even if the original video is removed, other versions can live on, undetected. TikTok videos can also be downloaded and shared on other platforms, like Facebook and Twitter, which is how Madung first encountered some of them.
Several of the videos flagged in the Mozilla Foundation report have since been removed, but TikTok did not respond to questions about whether it has removed other videos or whether the videos themselves were part of a coordinated effort.
But Madung suspects that they might be. “Some of the most egregious hashtags were things I would find researching coordinated campaigns on Twitter, and then I would think, what if I searched for this on TikTok?”