promptfoo/promptfoo
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.
TypeScript25K stars2.3K forks+1.7K today
Topics
#llm#prompt-engineering#prompts#llmops#prompt-testing#testing#rag#evaluation#evaluation-framework#llm-eval#llm-evaluation#llm-evaluation-framework#ci#ci-cd#cicd#pentesting#red-teaming#vulnerability-scanners
What it does
Promptfoo is a CLI tool and library for evaluating and securing Large Language Models (LLMs) through automated testing, red teaming, and vulnerability scanning. It supports multiple LLM providers and integrates with CI/CD pipelines.
Star history
Tracking
- Last trending
- 2026-04-25
Creator kit
Hook
🚀 Evaluate & secure your AI apps like a pro with Promptfoo! #AI #DevSecOps
Content angles
- How to use Promptfoo for comprehensive LLM testing and security assessments.
- A deep dive into integrating Promptfoo with CI/CD pipelines for automated evaluations.
- Comparative analysis of different LLM providers using Promptfoo's evaluation framework.
Who should care
Developers, DevOps engineers, AI researchers, cybersecurity professionals