Cyber security firm Barracuda has demonstrated how attackers could use AI assistants to speed up business email compromise attacks, warning that compromised AI-enabled accounts could be a serious enabler of financial fraud.
In a controlled proof-of-concept exercise, the company’s Red Team showed how a single compromised employee account, combined with an AI assistant such as Microsoft Copilot, could be leveraged to compromise a chief executive’s account and facilitate a fraudulent wire transfer worth almost US$250,000.
Barracuda said the technique is not unique to Copilot and could apply to other enterprise-grade AI assistants.
The attack begins with an attacker gaining access to an employee’s account before using the AI assistant to create inbox rules that hide security alerts and other notifications, helping maintain persistence without alerting the victim.
The AI assistant is then used to analyse months of emails, attachments, and calendar activity to map the organisation, pinpoint senior decision-makers, and uncover valuable contextual data about how the business is run.
Armed with that information, the attacker instructs the AI to draft a convincing phishing email in the employee’s writing style and send it to the CEO from the legitimate internal account.
After compromising the executive account through session-token theft, the attackers repeat the process, using the AI assistant to search for financial information, pending invoices, and payment approvals. In the demonstration, the AI located a pending payment of US$247,500, allowing attackers to draft a realistic request directing finance staff to change the destination bank account before the transfer was processed.
Because the email originates from the CEO’s legitimate mailbox, references a genuine transaction, and mirrors the executive’s communication style, traditional email security tools are unlikely to identify it as malicious.
Attackers also used forwarding rules and AI-assisted searches to conceal evidence of the fraud.
Daniel Avulov, senior cyber security researcher on Barracuda’s Red Team, said AI assistants should increasingly be viewed as high-value assets because of the sensitive business information they can access.
“A user’s email history is full of sensitive information and context that can be leveraged by attackers,” Avulov said in a statement.
“The controlled attack shows how AI assistants can become unwitting malicious insiders and improve both the quality and speed of an attack.”
Want to see more stories from trusted news sources?Make Cyber Daily a preferred news source on Google.
