Research and thinking into the problem of Prompt Injection in language models like GPT.
Why is this a problem?
Now this is the part of building new tools that isn’t fun: Security. It creates anxiety in whoever is responsible for giving the green light on a deployment for software that will be used commercially or by consumers. Righly so, its a big deal and should be taken very seriously.
The rapid rise in popularity of LLMs recently and all of the awesome opensource work going on is helping adoption of AI progress at ridiculous pace, however we cannot lose sight of the important parts of building systems that are supposed to solve problems, by neglecting the less fun parts that are likely to cause problems.
If language models are used as part of web applications and backend system processes, then at some point it will require an input from a user or another system.
As any developers should be aware, any input into your application cannot be trusted and should be sanitised and mitigated for potential attacks. The same will apply for LLM’s and for any badly designed prompts or poorly sanitised inputs then the potential for prompts to be manipulated or exploited is high.
If your business or system is dependent on these LLMs in future, then this is a risk that should be mitigated and taken into account when designing prompts.
What can be done about prompt injection?
Keep reading with a 7-day free trial
Subscribe to callumcarolan.ai to keep reading this post and get 7 days of free access to the full post archives.