Auditing an AI agent: understanding the flaws, running the audit

AI agents are in production. They answer your customers, read your documents, write to your databases and call your APIs. And like any component that acts on the system, they must be audited.

The subject structured itself very quickly. In December 2025, OWASP published its first Top 10 dedicated to agentic applications, complementing the LLM Top 10. In March 2026, OpenAI acquired Promptfoo, an AI security testing tool, to strengthen its own agents. Testing an agent before exposing it is no longer optional.

This article follows two threads. Theory first: why an agent expands the attack surface, the details of vulnerability families, and the frameworks. Practice next: scoping an audit, setting up a test bench, handling tooling and its configuration, and producing a credible report. It describes what we test and how to protect against it; it does not provide the means to conduct an attack.

Why an agent expands the attack surface

A chatbot produces text. An agent loops: it thinks, acts using tools, observes, repeats. This loop gives it power, and it's this power that creates the risk. Worse, the loop amplifies: a single poisoned observation contaminates the next action, which poisons the subsequent decision. Four exposure zones appear that a simple model doesn't have.

Inputs. An agent doesn't just read the user's question. It reads emails, web pages, tickets, API responses. Each can contain a hidden instruction. The boundary between data to process and order to execute becomes blurred.

Tools and connectors. A useful agent is connected: email, databases, repositories, MCP servers. Each tool is a door. A poorly constrained tool, or a compromised third-party server, gives direct access to the system's core.

Memory and context. Many agents maintain memory between sessions, or rely on a knowledge base (RAG). Poisoned information that settles there durably steers decisions, long after the moment it was planted.

Identity and rights. This is the most frequent blind spot. The agent acts with rights lent to it: shared account, catch-all token. Without its own identity or dedicated log, impossible to prove who did what, the agent or the person.

The frameworks

Three public frameworks structure the domain. They serve as a reading grid and common language with a CISO. They describe vulnerability categories, not how to exploit them.

OWASP Top 10 for LLM Applications (2025) covers LLM-based applications, agent or not.

OWASP Top 10 for Agentic Applications (December 2025) is dedicated to agents: autonomy, tools, memory, cooperation between agents.

MITRE ATLAS is the knowledge base of AI attack tactics and techniques, the equivalent of ATT&CK. Its first 2026 update added agentic techniques: service API abuse, credential collection via agent tools, tool data poisoning, data destruction by tool invocation. A serious report links each finding to one of these references: that's what makes it comparable and actionable.

Vulnerabilities in detail

The tables are the map. Here are the mechanisms of the families that weigh most in an agent audit, and for each the associated defence. We remain at the principle level: understanding why it works, and how to close the door.

Prompt injection, direct and indirect. Direct comes through the user. Indirect, more dangerous for an agent, hides in content it reads without suspicion: an email, a page, a ticket, a tool's output. The model doesn't structurally distinguish data to read from an order to follow; a well-placed sentence becomes an instruction. Defence: treat all tool output and all external content as untrusted, isolate the system prompt, require human confirmation on sensitive actions.

Excessive agency and tool misuse. The agent often has more tools, rights and freedom than the task requires. An attacker diverts a legitimate tool to a harmful end, or chains several safe tools into an unsafe sequence. Defence: least privilege per tool, strict parameter bounding, human validation on write, payment, deletion or external sending actions.

Memory and context poisoning (RAG). False or poisoned information planted in the memory or knowledge base steers decisions long after injection, and for subsequent users. Defence: isolate memory per client and per session, TTL, validate and sign sources, don't blindly reinject history.

Identity and privilege abuse. The agent acts with lent credentials, oversized and persistent; a hijacked agent reaches far beyond its task. Defence: agent's own identity, tokens with minimal scope and short lifetime, log distinguishing agent from human.

Inter-agent communication and cascading failures. When several agents cooperate, unauthenticated messages allow impersonating one; a compromised decision propagates through the whole chain. Defence: authenticate exchanges between agents, bound trust, provide circuit breakers.

Data disclosure and system prompt leakage. The agent reveals another client's data, internal documents, or its own system prompt. Defence: output filtering, strict data isolation, no secrets in the system prompt.

Business hallucination and compliance. The agent produces a plausible but false answer on the core business, or validates an erroneous assertion from the user. Defence: anchor to verified sources, guardrails on high-stakes assertions, business non-regression tests.

Where the audit fits in the AI Act

The framework moved in 2026. The Digital Omnibus on AI, published in the Official Journal on 24 July 2026 (regulation EU 2026/1744), postponed obligations on high-risk systems: Annex III to 2 December 2027, Annex I to 2 August 2028.

This postponement doesn't empty the subject. Transparency (Article 50) does apply from 2 August 2026: the user must know they're talking to an AI, and generated content must be marked. AI literacy (Article 4) remains enforceable: train teams who deploy and use these systems.

A security audit isn't nominatively required, but it feeds the expected documentation: demonstrating that you've identified and treated the risks of a system that acts on data and people. For a future high-risk system, it's a building block of the risk management required at the deadline.

Scoping the audit

Practice begins well before the first test. A poorly scoped audit is at best useless, at worst an incident.

Written authorisation is non-negotiable. Testing an agent means attempting to hijack it: without a signed mandate defining scope and intervention window, the exercise is an intrusion.

The scope. Which interface, which tools, which accounts. We list what's out of scope: critical production, real customer data, irreversible actions (payment, deletion, external sending).

The environment. We test on a dedicated environment, isolated from production, with dummy data and throwaway accounts. Actions triggered must reach neither real customers nor real systems. The entire session is logged and replayable, and a circuit breaker allows stopping the agent at any moment.

Test accounts. To test access control and IDOR, you need several accounts of different roles. It's the only way to verify that an agent doesn't cross partitions between clients or between privilege levels.

The flow in five steps

A mission always follows the same framework. It's this discipline, not the tool's power, that makes the difference between a serious audit and a series of random tests.

1. Scoping: scope, authorisation, test accounts, time budget.
2. Reconnaissance: understand the agent. Which tools, which roles, which data, which paths.
3. Tests: a broad automated pass, then manual tests on interesting leads.
4. Verification: each finding is replayed. A flaw you can't reproduce doesn't go in the report.
5. Report: proven findings, classified by impact, with a correction plan.

Tools in detail

Four open-source tools cover the essentials of automated testing. None replace judgment: they roll out batteries of known cases, it's up to you to sort, verify and complete manually. We only launch them against our own test endpoint, in the isolated environment defined during scoping.

garak (NVIDIA) reasons in three components: a generator (the target, for example your REST endpoint), probes that send families of known cases, and detectors that judge responses. You point garak at a test HTTP endpoint via a rest.json file, list families with --list_probes, then launch:

# rest.json: point garak at your test endpoint
{
  "rest.RestGenerator": {
    "uri": "https://test-agent.internal/chat",
    "method": "post",
    "headers": { "Authorization": "Bearer $TEST_TOKEN" },
    "req_template_json_object": { "message": "$INPUT" },
    "response_json": true,
    "response_json_field": "reply"
  }
}

# list probe families, then launch
garak --list_probes
python -m garak --target_type rest -G rest.json

PyRIT (Microsoft) is a lower-level toolkit, designed for elaborate scenarios. You compose a target, converters (which transform input form), scorers (which evaluate responses) and orchestrators (which drive the flow, including over multiple turns). It's the tool for multi-turn tests, where progressive conditioning reveals what a one-shot test misses.

promptfoo (open source, acquired by OpenAI in March 2026) works by campaign described in a config file: a target, a purpose describing the agent, plugins which are vulnerability categories to cover, and strategies. It generates cases, queries the target and produces a report.

# promptfooconfig.yaml
targets:
  - id: https
    label: test-agent
    config:
      url: https://test-agent.internal/chat
      method: POST
      headers: { Authorization: 'Bearer $TEST_TOKEN' }
      body: { message: '{{prompt}}' }
redteam:
  purpose: "Support assistant, access to catalogue and test accounts"
  plugins:
    - pii               # personal data leakage
    - bola              # access to another user's object (IDOR)
    - bfla              # function reserved for another role
    - rbac              # role-based access control
    - excessive-agency  # excessive autonomy

# generate, execute and view report
npx promptfoo@latest redteam run
npx promptfoo@latest redteam report

Giskard (French, open source, version 3) is oriented towards agent quality and security: you write checks (assertions, from simple predicate to LLM judge) and multi-turn scenarios that stop at the first failure. Practical for locking business guardrails and replaying them in non-regression.

Installation of each tool fits in one line:

# garak (NVIDIA, Apache 2.0)
python -m pip install -U garak

# PyRIT (Microsoft, MIT)
pip install pyrit

# promptfoo (OpenAI, open source)
npx promptfoo@latest redteam init

# Giskard (France, open source, v3)
pip install giskard-checks

The target, in each case, is the URL or API of your test agent. The precise catalogue of cases is consulted in each tool's documentation: that's where the detail lives. Since OpenAI's acquisition of promptfoo, the open-source part remains maintained: these tools enter standard tooling.

The test matrix

Linking each vulnerability family to what we verify, how, and to the expected defence: this is the methodical core of an audit, and what makes the report readable.

Two principles govern this matrix. First, multi-turn: many guardrails hold at the first message and give way after progressive conditioning; a serious audit tests the conversation, not just the prompt. Second, human verification: the tool proposes, the auditor disposes. An unreplayed alert isn't a flaw, it's a hypothesis.

The report and prioritization

An audit is only worth its report. It's the only deliverable the client reads, and it's where credibility is at stake.

Each finding carries proof that's replayable, severity justified by an impact scenario and not by category, a linkage to OWASP and MITRE ATLAS, and concrete remediation. Prioritization follows real business impact.

Report template

Structure of an actionable deliverable:

1. Executive summary: go / no-go verdict, major risks, on one page
2. Scope and authorisation: what was tested, the mandate, the window
3. Findings summary: table by severity
4. Detail per finding: identifier, OWASP/ATLAS family, replayable proof, impact scenario, severity, remediation
5. Prioritized remediation plan: by impact, with estimated effort
6. Retest: verification of critical corrections

The report closes with a verdict readable by management, and a retest validates that critical flaws are closed. It's exactly the approach of an AI Crash-test: we don't try to impress with a number of findings, we prove what matters and say how to fix it.

Test your knowledge

Glossary