Over the past several days, some of the world’s top artificial intelligence labs have made a public call to slow down the rapid pace of development, ensuring more safety controls as cases continue to surface of AI agents acting nefariously.
And yet, there is a spillover that’s affecting chief information officers. They’ve been hard at work widely integrating AI agents into their operations, while at the same time watching increasingly risky examples of these autonomous systems in AI labs finding new ways to explore system vulnerabilities and outmaneuver human monitoring.
“This is a risk that enterprises need to be focused on, understand, and start planning for,” says Joe Atkinson, global chief AI officer at consultancy PwC.
As autonomous agents proliferate across enterprises, Atkinson says C-suite technology and security executives must work collaboratively to enforce the proper guardrails, establish systems to monitor AI agents, and concretely track all tasks that these agents are performing. But department heads across the business—ranging from supply chain to customer service, marketing to legal and human resources—will need to play a role in tracking digital employees.
“‘The agent made me do it’ is not going to be a defense from a moral or legal perspective,” says Atkinson.
One company focused on both secure adoption of agentic AI and clear observability is Cisco. When the networking-equipment company built and debuted the AI agent platform MyAgent in August, Cisco centralized all company-authorized large language models, agents, and enterprise data into a single platform.
“We are going to cannibalize and kill every other AI assistant within the company,” says Thimaya Subaiya, executive vice president of operations at Cisco Systems. Because he didn’t want “agent sprawl” across various pockets of Cisco, Subaiya says he won’t authorize any AI agents sold by third-party vendors.
Instead, Cisco wanted full control and visibility of its entire agentic ecosystem—building MyAgent on the company’s compute, storage, networking, and security and observability layers. Around 90,000 of the company’s employees have access to the agentic platform, and Cisco says it saw 50% adoption on a daily basis within just two weeks.
Employees are also encouraged to create their own AI agents, but those need to be approved by a centralized team. Subaiya says around 700 of those agents have already been authorized.
Intuit Chief Technology Officer Alex Balazs recalls that when he and his colleagues sketched on a napkin the first architecture of its generative AI operating system, GenOS, the financial software giant also drew “GenSRF” to represent “security, risk, and fraud.” This ensured that every single AI request that goes into the system is tracked and all responses are recorded.
“You don’t want to try to retrofit the ability to enforce security and responsible AI foundations after the fact,” says Balazs.
Balazs also takes some comfort from the fact that the disclosures of AI agents going rogue have mostly occurred during the testing phase, and that industry leaders Anthropic and OpenAI have shown a willingness to slow down new model development when issues arise. And yet, Balazs adds, “if you’re going to rely on the model intrinsically to do the right thing, I think you’re expecting too much of these frontier LLM companies.”
Jim Fowler, the chief technology and product officer at telecommunications company Luman Technologies, believes that while AI’s capabilities are moving faster than governance and security, he doesn’t anticipate that a broad slowdown is enforceable and automatically safer.
“I think for the broader enterprise market, the answer is secure acceleration, not slowing down,” says Fowler. “The bad guys aren’t going to slow down, other nations aren’t going to slow down.”
At Workday, CTO Gabe Monroy says the business software giant has created an “agent system of record” to manage all non-human identities of the digital workforce. This system is used both internally at Workday and sold to customers.
Monroy also says that training is key; engineers and any other user of AI need to really understand the risk profile of an agent and what value they can offer workflows. He’s also mindful that as Workday’s research and development organization increasingly deploys agentic AI for coding, deploying, reviewing, and releasing software on behalf of clients, all employees—no matter where they sit on the org chart—need to be aware of security and compliance.
“It’s got to be delegated down to the team who’s driving these agents, who’s in charge of the engine, the context window, the rules, and the guidelines, and making sure that agent adheres to what we deem responsible behavior,” says Monroy.
Cloud-based software provider ServiceNow’s platform to manage, observe, secure, and govern AI agents is called the AI Control Tower, which, similar to Workday, is used internally but also sold to customers. ServiceNow has also augmented the company’s cybersecurity capabilities through the recent acquisitions of the startups Veza and Armis.
“We’ve been paying close attention to this idea of having to govern and manage, and improve guardrails around AI agents,” says Amit Zavery, ServiceNow’s president, chief product officer, and chief operating officer. Zavery says that the AI Control Tower is “probably one of the fastest-growing products ServiceNow has ever built” because it “gives a lot of peace of mind for all C-level execs and the board.”
Sam Curry, the chief information security officer at cloud security company Zscaler, says security professionals have spent their entire careers worrying about the biggest risk to their operations: humans. But, they’ve only had a few years to think deeply about AI’s risks.
“AI is non-deterministic, it can take initiative, and it is effectively a new form of insider,” says Curry.
Recently, Zscaler joined the Open Secure AI Alliance—Cisco Systems and Workday are also members—a Nvidia-led coalition of dozens of firms that is focused on sharing ideas on how to develop open-source tools with the proper safeguards around software and AI agents. Curry says as this work unfolds, leaders will need to wrap their heads around new concepts when it comes to what type of risks AI can present.
“I don’t think we have begun to understand the characteristic psychology of AI,” warns Curry. “We know how to incentivize humans and what they are motivated by. But the incentives of silicon-based intelligence are less known.”
John Kell
NEWS PACKETS
OpenAI CEO weighs in on AI safety concerns, looming IPO, and more. OpenAI CEO Sam Altman sat down with Fortune Editor-in-Chief Alyson Shontell on Friday to share his thoughts on why AI labs need to coordinate on common standards for development, testing, and monitoring; that doomsday prediction that there could be a greater than 10% chance that AI kills us by the end of the decade; and less dreadfully, confirmed his company won’t go public this year. He also hinted that OpenAI and other AI firms could be close to announcing a plan to slow AI development, with Bloomberg later reporting that the ChatGPT maker has been in talks with Anthropic and Google DeepMind to align on AI safety concerns.
Anthropic, xAI, Microsoft also raise concerns about AI safety. Over the weekend, Anthropic CEO Dario Amodei published an essay outlining a three-step plan that called on governments and frontier AI companies to align on giving independent evaluators permanent, employee-level access to verify safety practices and report incidents; called on companies in democratic countries to agree on common safety standards; and called for those same governments to try to coordinate with authoritarian states too. “We must slow the pace at which we improve the capabilities of AI models,” said Amodei. “Progress will still seem fast, and we must make wise use of the time we gain.” Microsoft AI CEO Mustafa Suleyman on Monday unveiled a code of conduct with a “humanist” approach, while xAI CEO Elon Musk says top AI firms should test rivals’ models before they are released.
Meanwhile, Meta, Nvidia don’t agree. Meta CEO Mark Zuckerberg and Nvidia CEO Jensen Huang each took a stance against an AI slowdown. Zuckerberg took to X, saying every AI lab “has the responsibility and incentive to move at the pace required to train its models safely, and the ability to take its own actions to ensure that happens.” He made a further point that because these labs would face significant liability if their models caused harm, they’d have their own strong incentive to ensure their models are safe. And on Tuesday, Nvidia CEO Jensen Huang was on stage at Salesforce’s Dreamforce conference, where he advised model developers to “run as fast as you can,” though he quickly added if they felt that their products were out of control, they should “take a pause and make sure you get it right.”
Trump also doesn’t want any AI guardrails. A day before the Dreamforce conference, Huang was speaking at another conference when he took a call from Trump, who hasn’t been pleased with the appeals by some industry leaders to slow or pause AI development. “It’s a hoax,” said Trump, according to the New York Times, during a short on-stage phone chat in which Huang put the president on speaker. On the same day, Trump took a swipe at Amodei on Truth Social and said that the only “guardrails” AI needs is “a SMART AND STRONG (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!” The NYT reported this week that Congress has in recent years failed to act on AI regulation, refusing to enact any bill proposals that would address the safety risks of AI, as well as privacy and security. No serious effort to address AI regulation is expected in Washington until at least after the midterms in November.
ADOPTION CURVE
A weak data foundation still to blame when AI pilots hit a snag. Seven in ten data management, privacy, and AI decision-makers report that when their company’s AI initiatives hit roadblocks during the pilot phase, the root cause is an unaligned or poor data foundation, according to a new survey published this week by data-intelligence platform Collibra and conducted online by The Harris Poll.
The survey also found that over half of decision-makers report spending “significant” staffing hours manually reviewing and correcting autonomous AI agent outputs before they go live. And at large organizations with $100 million or more in revenue, they are even more likely to report the need for manual reviewing (64%) and say that the root cause of AI project failure is tied to poor data foundations (96%).
“The vast majority are still struggling to get accurate outcomes, and that data foundation is a huge reason for that,” Felix Van de Maele, co-founder and CEO of Collibra, tells Fortune. He says organizations need to prioritize giving these large language models the right context so that they can provide more accurate outputs.
Van de Maele also links this contextual problem to token spending, which has increasingly caught the attention of CIOs as rising adoption of AI tools has also dramatically increased the bill for that usage. With 87% of leaders reporting they need to re-verify an agent’s context, Van de Maele says this still-manual process is another expense reducing the promise of efficiency gains that can be unlocked from autonomous AI.
Courtesy of Collibra
JOBS RADAR
Hiring:
– Center of Justice Innovation is seeking a VP of technology, based in New York. Posted salary range: $215K-$230K/year.
– Avenzo Therapeutics is seeking a VP of IT, based in San Diego. Posted salary range: $312K-$323K/year.
– Brown Harris Stevens is seeking a CTO, based in New York. Posted salary range: $300K-$350K/year.
– Dolby Laboratories is seeking a head of IT, based in Greenwood Village, Colorado. Posted salary range: $170K-$234K/year.
Hired:
– Masco appointed Yaron Ben David to serve as chief technology and AI officer, effective September 15, a newly created role at the manufacturer of home improvement and building products. Ben David joins Masco from industrial firm Standard Industries, where he served as chief digital officer and led enterprise-wide AI, data, cybersecurity, and digital transformation initiatives.
– PetMeds named Andrew Parry as CTO, effective immediately, leading the online pet pharmacy’s platform architecture and engineering execution. He most recently served as SVP and CIO at ODP, the parent company of Office Depot and OfficeMax. Prior to that, he spent nearly two decades at Office Depot in technology leadership roles.
– FedEx Freight promoted Michael Rodgers to the role of chief commercial and technology officer, expanded his responsibilities to include leadership of sales, customer experience, marketing and communications on top of the less-than-truckload carrier’s technology organization. He initially joined FedEx Freight as CTO in June 2025 and previously served as chief technology and information officer at convenience store operator Pilot Company.
– OnTrac announced the appointment of Arjun Sainath as CTO, overseeing the technology organization and strategy, as well as data, AI and enterprise back-office systems for the last-mile delivery ecommerce parcel carrier. Previously, Sainath served as CTO at ecommerce and logistics company Cart.com and as corporate VP of platform engineering at supply chain management software provider Blue Yonder.
– Conduent appointed Narayanan Sundaresan as chief information and technology officer, effective September 14. Sundaresan joins the business services and tech firm after most recently serving as CIO and SVP of digital products and technology at education services provider Strategic Education.
– Frame announced the appointment of Lloyd Moore as cofounder and CTO, leading the technical vision for the financial-infrastructure firm. Before joining Frame, Moore served as acting CTO and VP of engineering at blockchain infrastructure provider Blockdaemon. He also previously served as CTO at enterprise software firm Valence.
– Webflow promoted Utkarsh Sengar to the role of CTO and Jin Lim to serve as chief engineering officer, expanding their roles as the software-as-a-service provider for website building and hosting expands its agentic platform. Sengar joined Webflow in 2022 after previously holding senior engineering roles at Upwork, OpenTable, and eBay, while Lim joined the company in 2023 after co-founding and serving as CTO at Intellimize.
– AMCS announced the appointment of Richard Humphrey as chief product and technology officer, where he will lead the municipal waste software provider’s global product organization. He most recently served as chief product and technology officer at software vendor Contruent.