The National Cyber Security Centre (NCSC) has warned users of AI-powered chatbots like ChatGPT, Google Bard, and Meta’s LLaMA that the technology poses security threats and can be manipulated by hackers.

Prompt injection attacks, which occur when users create a prompt which makes large language models (LLMs) behave in an unintended way and can cause chatbots to produce offensive material or reveal confidential information.

The NCSC said that as LLMs are increasingly used to pass data to third-party applications and services, the risks from malicious prompt injection will grow.

The organisation said that prompt injection attacks can be difficult to detect and prevent, calling on businesses to learn about the risks, apply basic cybersecurity rules and educate their users about the risks. Systems should also be designed taking these potential risks into account.

“At present, there are no failsafe security measures that will remove this risk,” warned the NCSC. “Consider your system architecture carefully and take care before introducing an LLM into a high-risk system.”

The organisation added that a machine learning model is only as good as the data is trained on, with LLMs being no exception.

“Their training data is typically scraped from the open internet in truly vast amounts, and will probably include content that is offensive, inaccurate or controversial,” it continued. “Attackers can also tamper with this information to produce undesirable outcomes, both in terms of security and bias.”

Share Story: