iscover
Share this article on:
Powered by
MOMENTUM
MEDIA
The discussion around the dangers of AI and the cyber risks they create continues, with British officials issuing a warning that AI tools can be fooled into being harmful.
The British National Cyber Security Centre (NCSC) has said the full risks that AI tools present and their ability to replicate human behaviour had not been fully realised by experts, and that large language models could be used in malicious ways, particularly when implemented with customer service, sales, and other business tasks.
Chatbots, which are the usual medium in which AI is utilised at this stage, are also highly risky, with researchers finding ways to get around the rules put in place by developers that limit them from performing harmful tasks, such as writing malicious code or relaying dangerous information.
The NCSC warning was issued via two blog posts on its website.
In the first blog post, “Exercise caution when building off LLMs”, the NCSC has said that the jump from LLMs from machine learning (ML) means that there is a massive lack of understanding of the risks.
“The challenge with LLMs is that, although fundamentally still ML, LLMs (having being trained on increasingly vast amounts of data) now show some signs of more general AI capabilities,” says the NCSC’s David C.
“Creators of LLMs and academia are still trying to understand exactly how this happens and it has been commented that it’s more accurate to say that we ‘grew’ LLMs rather than ‘created’ them.
“It may be indeed more useful to think of LLMs as a third entity that we don’t yet fully understand, rather than trying to apply our understanding of ML or AGI.”
One of the issues that researchers have already detected is that LLMs struggle to make a distinction between an instruction and the data inputted to assist in completing the instruction.
“Consider a bank that deploys an ‘LLM assistant’ for account holders to ask questions, or give instructions about their finances,” the post continues.
“An attacker might be able send a user a transaction request, with the transaction reference hiding a prompt injection attack on the LLM. When the user asks the chatbot, ‘Am I spending more this month?’, the LLM analyses transactions, encounters the malicious transaction, and has the attack reprogram it into sending user’s money to the attacker’s account.”
The other major concern with the development of AI chatbots is the risk that they could be manipulated and corrupted with the input of loaded training data.
The NCSC has said that to defeat data poisoning attacks and prompt injection, systems need to be designed from the ground up with security in mind.
“What we can do is design the whole system with security in mind,” said Martin R.
“That is, by being aware of the risks associated with the ML component, we can design the system in such a way as to prevent exploitation of vulnerabilities leading to catastrophic failure.”
Be the first to hear the latest developments in the cyber industry.
