Promptfoo provides a structured approach to developing Large Language Model (LLM) applications, moving beyond manual trial-and-error. This open-source CLI tool and library focuses on systematic evaluation and red teaming, ensuring LLM performance, accuracy, and security.
Automated Red-Team Testing for LLM Applications
LLM applications require rigorous testing to ensure they perform as expected and remain secure. Promptfoo addresses this by enabling developers to define thorough test cases for prompts, models, and Retrieval-Augmented Generation (RAG) pipelines. It evaluates performance, accuracy, and reliability, and identifies critical security vulnerabilities like prompt injections, jailbreaks, and Personally Identifiable Information (PII) leaks. This test-driven development approach helps maintain quality and prevents costly issues in production.
Define Test Cases, Run Evaluations, Compare Outputs
Promptfoo integrates into the LLM development lifecycle, allowing users to define test cases using YAML, CSVs, or Google Sheets. It supports over 60 LLM providers, including OpenAI, Anthropic, Google, Azure, and local models via Ollama. The tool manages LLM provider rate limits automatically through retries and adaptive concurrency. Evaluations run locally by default, ensuring data privacy, and generate matrix views for side-by-side comparisons and detailed vulnerability reports.
Key technical features include:
- Test Case Definition: Use YAML, CSVs, or Google Sheets to define tests.
- Broad LLM Support: Integrates with over 60 providers, including major cloud and local options.
- Automated Assertions: Supports regex, semantic similarity, custom JavaScript, and LLM-as-a-judge for output scoring.
- CI/CD Integration: Works with GitHub Actions, GitLab CI, and Jenkins for continuous testing.
- Rate Limit Management: Automatically handles LLM provider rate limits.
- Local-First Evaluation: Ensures data privacy by running tests locally.
AI Engineers, QA Teams, and Security Researchers
Promptfoo serves a diverse group of professionals involved in LLM development and deployment. Developers and machine learning engineers use it for evaluating and comparing prompts across various models. Security teams and QA engineers use its automated red teaming capabilities to uncover vulnerabilities and ensure quality standards. AI product managers can use it for regression testing and chatbot evaluation. Its integration into CI/CD pipelines makes it valuable for teams aiming for continuous security and quality assurance.
Free and Open-Source (MIT); Cloud Platform Available
Promptfoo offers a free, open-source Community Plan that includes all core LLM evaluation features. This plan supports all model providers and integrations, along with red teaming, limited to 10,000 probes per month. It operates locally or can be self-hosted.
For advanced needs, custom-priced Enterprise and On-Premise plans are available:
| Plan | Price | Key Details |
|---|---|---|
| Community | Free | All core LLM evaluation features, all model providers, 10,000 red teaming probes/month, local/self-hosted. |
| Enterprise | Custom | Custom red teaming limits, team collaboration, continuous monitoring, centralized security dashboard, customizable attack profiles, SSO, granular permissions, Promptfoo API access, managed cloud deployment, priority support. |
| On-Premise | Custom | All Enterprise features, deployed on user’s infrastructure for complete data isolation. |
CLI-First, Requires Test Case Design Expertise
promptfoo has limitations around its CLI-heavy interface:
- The open-source version lacks built-in production monitoring, dashboards for test organization, and native cost tracking.
- Its YAML-heavy workflow can feel verbose for simple test cases, with a learning curve for configuration syntax.
- Primarily CLI-based with no official GUI, which may not suit teams wanting visual test management.