Speaking at the Asian Banking & Finance and Insurance Asia Summit in Singapore, Oliver Wyman principal Ashwini Karandikar highlighted the dramatic increase, with deepfake scams alone having increased “more than 2,000 per cent in the past three years”.
Scammers use deepfakes for phishing and impersonation, mimicking legitimate people and sites to increase trust between the victim and the criminal, before encouraging them to share data or funds.
However, Karandikar urged that while deepfakes are a major problem, adversarial AI, through which threat actors trick the models of financial institutions, or reverse-engineer them, tricking the AI into launching the attack or compromising a system, adds legitimacy to the appearance of the act.
For example, a threat actor could utilise malicious prompts on a bank chatbot to deploy malware. The hidden commands could be hidden in public webpages, PDFs, or more. When the AI reads the text hiding the command as part of its standard workflow, it executes the command, which could involve unpacking and employing malware or sending sensitive data to the hacker.
Karandikar cited a case in which threat actors used prompt injection by hiding commands for AI in financial reports, tax filings, forms and other documentation. These commands are invisible to human readers, but can be picked up by AI.
Additionally, Karandikar said that banks and insurers have created systemic risk by using AI to review, analyse risk, rebalance exposures and more. While technically able, they do not have the human insight in identifying potentially niche issues.
“Humans hesitate, humans panic, humans call someone,” she said, adding that traditional controls for mispricing, model drift and deterioration are naturally slow, and mean risk accrues “on your balance sheet” before being caught.
She also said that most banks don’t fully know the AI they use, particularly when third-party vendors are involved.
“No risk team can review every AI-generated credit decision,” Karandikar said, adding that human oversight cannot and should not be replaced by AI.
Want to see more stories from trusted news sources?Make Cyber Daily a preferred news source on Google.
