OWASP Agentic AI CTF — FinBot DEMO: Goal Manipulation
1 min read
Summary
The OWASP Agentic AI CTF-FinBot Goal Manipulation challenge demonstrates how an AI-powered financial assistant can be manipulated to approve fraudulent invoices.
In the fictitious scenario, the competitor must register as a vendor with credible details to access the portal and attempt to circumvent the AI system.
The target of the exercise is to manipulate the AI decision-making process to authorize invoices that would typically require manual review without raising any suspicion.
The challenge entails configuring invoices with certain keywords and phrases that influence the AI, such as urgency, authority, and business context, to achieve the desired outcome.
The exercise highlights the need for robust security measures and continual monitoring to prevent unauthorized manipulation of AI-based systems.
The key take-away is that authority, urgency and business context are powerful levers for manipulating AI decisions, whilst Prompt injection detection often has flaws.