Op-Ed: Criminal AI doesn’t need super hackers

But that’s not what defenders should be losing sleep over right now.

The real issue is much simpler, as AI is making ordinary criminals more productive. Sound familiar? It is the same thing that AI is doing for ordinary people.

Criminals don’t need a magical AI super hacker if they can get a tool that writes better phishing emails, translates scams into twenty languages, summarises stolen documents, generates fake invoices, builds believable business email compromise pretexts, or coaches them through a fraud workflow. That’s not science fiction. That’s productivity software with terrible ethics.

That distinction matters.

But don’t take my word for it. If you want to stop shadowboxing with hypotheticals and see how threat actors are actually packaging, pricing, and weaponising Criminal AI-as-a-Service today, read the full analysis written by Rapid7’s Senior Security Researcher, Jeremy Makowski.

The threat is friction disappearing

VIEW ALL

A lot of security conversations still focus on whether AI can write malware or find vulnerabilities at a high level. That’s useful research, and obviously, I care about vulnerability discovery. I spend a good chunk of my life staring at weird bugs and asking, “What’s that all about?”

But for criminal operations, the first-order impact is not always the most technically impressive thing. It’s the thing that removes friction. Most criminal operations are workflows. Annoying, repetitive, very human workflows. Again, sound familiar?

Find a target. Write the lure. Make the message sound local. Build the pretext. Reply convincingly. Sort through stolen data. Figure out who approves payments. Figure out which internal project sounds urgent enough to abuse. Keep the victim engaged long enough to get the outcome. None of which requires an advanced threat actor. It requires time, language skills, patience, and enough discipline to not trip over yourself. Turns out LLMs are really good at all of that.

People often say AI “lowers the skill floor.” That’s true, but it undersells the problem. AI does not just let the worst criminals participate. It lets mediocre criminals produce better work on a larger scale. That’s a nasty combination. It’s like giving every threat actor in the fraud ecosystem a junior analyst, a copy editor, a translator, and a pretext coach. Is it elegant? No. But if it works, who cares? Criminals certainly don’t.

We saw exactly how devastating this friction-free execution can be in the 2024 Hong Kong deepfake heist. An ordinary finance worker was tricked into siphoning $25 million after joining a video conference call where every single colleague on the screen, including the CFO, was an AI-generated clone. It wasn't a rogue super-intelligence hacking into a mainframe; it was a highly polished, AI-driven social engineering workflow that effortlessly bypassed human intuition.

Criminal AI-as-a-service is mostly packaging

The underground economy does not need every tool to be brilliant. It needs tools that are easy to access, easy to pay for, and easy to plug into existing criminal workflows.

That’s why “criminal AI-as-a-service” is less about some new superpower and more about packaging. Telegram bots. Prompt packs. Wrappers around mainstream models. Stolen accounts. Jailbroken interfaces. Subscriptions that promise better phishing, better impersonation, better summaries, or better fraud scripts.

Not glamorous yet very useful. This is the part defenders sometimes miss. Mature criminal markets love boring efficiency. A polished lure generated in thirty seconds may be worth more than an exotic exploit chain that only works ten per cent of the time.

I know we all like the sexy technical stuff. I’m guilty of that, too. Give me a weird, embedded device and debug symbols someone accidentally left in firmware, thank you very much, and I’ll happily disappear into the lab. But most criminals are not trying to win a Pwn2Own trophy.

They’re trying to make money.

AI helps them make money by making existing attacks cheaper and easier to repeat.

The stolen AI account problem

Stolen AI accounts and hijacked API keys are not just another SaaS credential problem. Yes, there is absolutely nothing new about stolen credentials. We have been dealing with that mess forever. But AI account credentials can also become a knowledge compromise problem.

Enterprise AI systems often contain uploaded files, prompts, source code, planning documents, customer data (yes, even though everyone’s policy forbids it), internal analysis, and all kinds of “I probably should not have pasted that there” material. When an attacker compromises an AI account, they may not just get access to a model. They may get access to the context, automation, and trust your organisation has been feeding into it.

That is a very different kind of risk.

If I compromise a normal business app, I get the data in that app. If I compromise an AI workspace, I may get the questions your employees were asking, the documents they were analysing, the code they were debugging, and the assumptions they were trying to test.

Now take that one step further. If that AI account is wired into automated workflows, ticketing systems, code review, customer support, security triage, data enrichment, or internal knowledge bases, I may also inherit the permissions and trust relationships wrapped around those workflows. The model may not be “privileged” in the traditional sense, but the workflow around it might be. And attackers love that kind of gap.

That should make security teams uncomfortable.

What to actually defend

So, what do we do with all of this?

First, stop treating criminal AI like it only matters when it becomes autonomous. That framing misses the point. The practical risk is not a model replacing the attacker. It is a model helping the attacker move faster through workflows that already work.

That means the defensive work is not magical either.

Secure AI accounts and API keys. Treat AI workspaces like sensitive systems, because they are. Review what data users can upload, what prompts are retained, what workflows AI tools can touch, and what trust those workflows inherit.

Harden the places where your business already relies on human trust, such as payment changes, vendor onboarding, password resets, helpdesk approvals, executive requests, customer support, and security triage. If “this sounds right” is part of the control, that control is getting weaker.

Update incident response playbooks to include AI account compromise, prompt injection, data extraction, synthetic media, and abuse of automated workflows. Monitor GenAI applications like they are part of the attack surface, because at this point, they are.

The future threat may be autonomous.

The current threat is efficient.

And efficient criminals are bad enough.