Zakto further alleges that Twitter has no comprehensive development or testing environments for piloting new features and system upgrades before launching them in the live production software. As a result, Zatko describes a situation where engineers would work alongside live systems and “test directly on the commercial service, leading to regular service disruptions.” And the documents allege that half of Twitter’s employees had privileged access to live production systems and user data without monitoring to be able to catch any rogue actions or trace unwanted activity. Zatko’s complaint describes Twitter as having roughly 11,000 staffers. Twitter says it has about 7,000 employees currently.
The complaints assert that these poor security practices explain Twitter’s track record of security incidents, data breaches, and dangerous user account takeovers.
“We are reviewing the redacted claims that have been published,” Twitter CEO Parag Agrawal wrote in a message to Twitter staff this morning. “We will pursue all paths to defend our integrity as a company and set the record straight.”
Twitter says that all employee computers are centrally managed and that its IT department can force updates or impose access restrictions if updates aren’t installed. The company also said that before a computer can connect to production systems, it must pass a check to ensure its software is up-to-date, and that only employees with a “business justification” can access the production environment for “specific purposes.”
Al Sutton, cofounder and chief technology officer of Snapp Automotive, was a Twitter staff software engineer from August 2020 to February 2021. He noted in a tweet on Tuesday that Twitter never removed him from the employee GitHub group that can submit software changes to code the company manages on the development platform. Sutton had access to private repositories for 18 months after being let go from the company, and he posted evidence that Twitter uses GitHub not only for public, open source work, but for internal projects as well. Within about three hours of posting about the access, Sutton reported that it had been revoked.
“I think Twitter is being pretty casual about Mudge’s claims, so I thought a verifiable example might be useful for folks,” he told WIRED. When asked whether Zatko’s accusations track with his own experience working at Twitter, Sutton added, “I think the best thing to say here is that I have no reason to doubt his claims.”
Security engineers and researchers emphasize that while there are different ways to approach production environment security, there is a conceptual problem if employees have broad access to user data and deployed code without extensive logging. Some organizations take the approach of drastically limiting access, while others use a combination of broader access and constant monitoring, but either option must be a conscious choice that a company invests heavily in. After the Chinese government breached Google in 2010, for example, the company went all in on the former approach.
“It’s not actually that unusual for companies to have relatively liberal policies about giving engineers access to production systems, but when they do they are very, very strict about logging everything that gets done,” says Perry Metzger, managing partner of the consultancy Metzger, Dowdeswell & Company. “Mudge has a sterling reputation, but let’s say he was completely incompetent. The easy thing for them to do would be to provide technical details of the logging systems that they use for engineer access to production systems. But what Mudge is portraying is a culture where people would prefer to cover things up than to fix them, and that is the disturbing bit.”
Zatko and Whistleblower Aid, the nonprofit legal group representing him, say they stand by the documents released on Tuesday. “Twitter has an outsized influence on the lives of hundreds of millions around the world, and it has fundamental obligations to its users and the government to provide a safe and secure platform,” Libby Liu, CEO of Whistleblower Aid, said in a statement.
For now, though, the allegations raise a swath of serious concerns that seem unlikely to be quickly explained away or comprehensively resolved.