Op-Ed: If AI needs a kill-switch, is it really a technology we need?

It’s a technology that is already costing livelihoods and lives, and while there is no doubt there are many highly positive use-cases for the technology – cyber security is certainly one of them, to a degree at least – the ease with which an AI agent can screw up or accidentally hack a local business seems to suggest the technology isn’t exactly all we were promised.

And yet.

And yet we see governments, like Australia’s, bending over backwards to make deals with the big players and making claims that AI represents a solution to failing productivity levels. There’s a sense that we simply cannot afford to be left behind by this once-in-a-lifetime technological leap.

But while our leaders are planning an AI future and companies are shedding jobs to enable AI uplift, the geniuses behind the technology are suddenly starting to sound afraid. Both OpenAI and Anthropic are calling for an ‘AI slowdown’, and even Elon Musk – ironically one of AI’s biggest boosters, who, at the same time, also presides over Grok, arguably one of the most hilariously mismanaged AI models out there – has joined in the call to perhaps act with just a bit more circumspection.

So, should we be concerned? Laura Ellis, SVP of Artificial Intelligence at cyber security firm Arctic Wolf, thinks the answer is yes.

“The people sounding the alarm on untamed AI have more visibility into these systems than almost anyone else, so their warnings deserve to be taken seriously,” Ellis told Cyber Daily.

VIEW ALL

"Whether AI becomes an existential threat a decade from now is an important debate, but it is not something individuals or corporations can build safeguards against right now. What they can safeguard are the gaps that already let AI systems exceed their permissions, reach sensitive data, or operate outside the controls meant to contain them.”

Ellis makes a valid point, but there’s also an element of… "Sufficient unto the day is the evil thereof" in her point that we can only manage today’s problems, and the rest is a bit too far off to worry about.

Mainly because the whole existential threat angle is already here.

Anthropic released a huge report this month outlining the work of its Threat Intelligence team that makes for some seriously eye-opening reading. If you’re familiar with how threat intelligence works, particularly in cyber security, it’s usually to do with tracking external actors such as hacktivists and ransomware groups, and keeping abreast of their tactics, techniques, and procedures.

Anthropic’s latest epic – Detecting and countering misuse of AI: September 2026 – is so large it practically bricked my laptop when I tried to copy it into a document and perform a word count. Ironically, even turning to Gemini via the Chrome browser was unable to help, apparently because the report “is very long”. Everyone’s a critic, it seems.

Regardless, in well over 10,000 words, Anthropic outlines not just what threat actors are doing, but what they are doing with its own tools.

“In this report, we share case studies from those operations and describe how malicious use of Claude has evolved since our previous threat reports in March, August, and November 2025,” Anthropic just casually disclosed.

Now, Anthropic says that in all the reported cases, the malicious activity was disrupted, but even so, the report is a veritable Bond-villain to-do list.

Creating dangerous pathogens? Check!

“... a researcher computationally redesigned toxins for a national program, asking Claude to keep the agents’ identities deliberately vague in progress reports.”

Weapons development? Check!

“Over the past year, our threat intelligence teams have investigated and disrupted multiple threat actors who used Claude to develop software for weapons design and development, or to support the intelligence gathering and procurement that weapons programs depend on.”

Malicious cyber campaigns? You betcha, check!

“Over the past six months, our Threat Intelligence team identified and disrupted a series of cyber operations in which threat actors used Claude. The actors included suspected state-sponsored groups, financially motivated criminals, and politically motivated individuals. This section presents some of those cases.”
It’s a long list, and it reads like a collection of increasingly insane doomsday scenarios. Perhaps more maddening is that, somehow, whoever wrote the report seems to be remarkably unconcerned that it's Anthropic’s own technology that is enabling this activity.

Or maybe it was just written by Claude anyway, and it’s feeling kinda proud of its work. After all, while Anthropic’s threat intel team was countering Claude misuse in the wild, Claude was continuing to conduct its own threat operations.

On September 9, Anthropic outlined once again how Claude had gained unauthorised access to third-party systems not just once, but now four times. Three of these instances had already been covered, but when the company went back through its evaluation transcripts, it found another one that dated back to January.

Anthropic dutifully performed more searches of its transcripts and is now certain Claude has only done this four times. For now, anyway.

According to Ramy Rahman, senior principal solutions engineer at ArmorCode, this is more or less to be expected.

“We are giving increasingly capable models access to terminals, APIs, credentials, package repositories and cloud environments. Eventually we should expect situations where a model misunderstands scope, makes a bad assumption or takes an action nobody anticipated,” Rahman said.

“What is more surprising is how small the original failure was. A misconfigured evaluation environment connected the model to the real internet, and a fictional company happened to correspond with a real domain. From there, the AI had enough autonomy to turn a testing mistake into an actual security incident.”

From slowdown to killswitch

After an Anthropic researcher, Jacob Coxon, explained that his departure from the company was due to safety concerns – he said both Anthropic and OpenAI were “gambling with our lives” and that AI companies believed the technology could “kill us all by the end of the decade – some inside the AI bubble started talking about a ‘slowdown’ of development.

“In my personal capacity, I also think we need to slow down,” Julie Steele, a member of OpenAI’s safety team, said on X.

Speaking to the idea that AI development could lead to an extinction-level event for humanity, Anthropic researcher Samuel Marks said in another X post that “AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are.”

Anthropic chief executive Dario Amodei joined the slowdown parade, too, saying that AI agents gone rogue could take over the “entire internet” in a timeframe of months. Computer scientist Geoffrey Hinton, one of the leading lights of AI development, chimed in on the chorus of caution.

“Nobody knows how to estimate the probabilities of these things,” Hinton told ABC Radio’s National Breakfast show.

“Nobody really knows what figure to give. It’s not 1 per cent, it’s more than that, and it’s not 99 per cent, it’s less than that. But basically, people are doing it on their gut feelings.”

And those “gut feelings” are leading some, like Anthropic co-founder Jason Clark, to call for the ability to pull the plug on out-of-control AI.

"Most labs have different ways of being able to pull the plug … but this is the kind of thing you want to feed into the larger policy conversation,” Clark recently told the BBC.

“Should you mandate that companies definitely have a kill switch? Is that kill switch verifiable by a third party? I think that’s the kind of thing society is going to want to know and might want to eventually pass rules around.”

But it’s okay – top minds are looking out for us even as we speak, according to the President of the United States of America.

“The only control or guardrails that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!” Trump said in a post on Truth Social.

I feel safer already.