Speaking at the Asian Banking & Finance andInsurance Asia Summit in Singapore, Principal at Oliver Wyman Ashwini Karandikar highlighted the dramatic increase, with deepfake scams alone having increased “more than 2,000 per cent in the past 3 years.”
Scammers use deepfakes for phishing and impersonation, mimicking legitimate people and sites to increase trust between the victim and the criminal, before encouraging them to share data or funds.
However, Karandikar urged that while Deepfakes are a major problem, adversarial AI, through which threat actors trick the models of financial institutions, or reverse engineer them, tricking the AI into launching the attack or compromising a system, adding legitimacy to the appearance of the act.
For example, a threat actor could utilise malicious prompts on a bank chatbot to deploy malware. The hidden commands could be hidden in public webpages, PDFs or more. When the AI reads the text hiding the command as part of its standard workflow, it executes the command, which could involve unpacking and employing malware or sending sensitive data to the hacker.
Karandikar cited a case where threat actors used prompt injection by hiding commands for AI in financial reports, tax filings, forms and other documentation. These commands are invisible to the human readers, but can be picked up by AI.
Additionally, Karandikar said that banks and insurers have created systemic risk by using AI to review, analyse risk, rebalance exposures and more. While technically able, they do not have the human insight in identifying potentially niche issues.
"Humans hesitate, humans panic, humans call someone," she said, adding that traditional controls for mispricing, model drift and deterioration are naturally slow, mean risk accrues “on your balance sheet” before being caught.
She also said that most banks don’t fully know the AI they are using, particularly when third-party vendors are involved.
No risk team can review every AI-generated credit decision," Karandikar said, adding that human oversight cannot and should not be replaced by human oversight.
Want to see more stories from trusted news sources?Make Cyber Daily a preferred news source on Google.
