promptfoo/promptfoo — Test your prompts, agents, and RAGs. AI Red teaming, pentesting, and vulnerabili

11,737 stars | 1,082 forks | TypeScript

Test your prompts, agents, and RAGs. AI Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.

What it does

Promptfoo is a CLI tool and library for evaluating and securing Large Language Models (LLMs) through automated testing, red teaming, and vulnerability scanning. It supports multiple LLM providers and integrates with CI/CD pipelines.

Why it matters: 🚀 Evaluate & secure your AI apps like a pro with Promptfoo! #AI #DevSecOps

View on GitHub

Trending today with 632 new stars

---

Want to create content about this repo? Use Nemati AI tools to generate articles, tutorials, and social posts.