Promptfoo
App : A product you use directly
Prompt, agent, and RAG testing and red-teaming, CI-integrable.
The testing tool to adopt before putting an AI into production; acquired by OpenAI, still MIT licensed.
Security reflects where data goes, requested permissions and incidents on record at the verification date. It is not a guarantee of zero risk: check the deployment mode and the connections you enable.
What it does, concretely
Deploying an AI application without prior testing exposes it to real risks: prompt injections, personal data leaks, guardrail bypasses, and unexpected agent behaviors. Promptfoo addresses this by automating security and evaluation testing before deployment.
In practice, the tool generates thousands of context-aware attacks tailored to your application (direct and indirect injections, jailbreaks, business rule violations, insecure tool use in agents), compares responses from different models side by side, and produces vulnerability reports. It integrates into CI/CD pipelines (GitHub, GitLab, Jenkins) and can scan pull requests to detect AI-related security issues.
Promptfoo is aimed at developers and security teams building applications on top of LLMs, agents, or RAG systems. It is used by 156 Fortune 500 companies across sectors such as healthcare, telecommunications, and retail. Evaluations run entirely locally, meaning prompts never leave your machine.
However, the tool requires Node.js 22.22.0 or higher (Node.js 24 LTS recommended) and a degree of familiarity with the command line and YAML configuration files. It is not designed for non-technical end users.
What it offers
- ·Automated prompt and model evaluation with side-by-side comparison across providers (OpenAI, Anthropic, Azure, Bedrock, Ollama, and more).
- ·Automated red teaming that generates custom attacks to detect over 50 vulnerability types, including prompt injections and jailbreaks.
- ·Security testing for agents and RAG systems, covering data leaks, business rule violations, and insecure tool use.
- ·Integration into CI/CD pipelines (GitHub, GitLab, Jenkins) for continuous checks at every deployment.
- ·Code and pull request scanning to detect AI-related security and compliance issues.
- ·Vulnerability reports with remediation steps delivered directly into developer workflows.
- ·100% local execution of evaluations: prompts and data never pass through external servers.
- ·Real-time threat intelligence powered by a community of over 300,000 users.
In practice
- Price
- Freemium
- Commercial use
- Commercial use: yes
- Available on
- cli, docker
- Licence
- MIT
Security and data
- Rating
- Security: nothing to flag
- Your data
- Data stays local
- Known incidents
- Rachat par OpenAI en mars 2026.
- Maintenance
- Active
- Last release
- 25 October 2026
Links and repository checked on 25 October 2026 · added on 1 May 2023
How to install Promptfoo
npm install -g promptfoo