Promptfoo
UnclaimedPromptfoo helps developers and enterprises identify and address security risks in AI applications.
Overview
Promptfoo helps developers and enterprises build secure, reliable AI applications. It provides capabilities for identifying vulnerabilities, assessing risks, improving defenses, and supporting secure development workflows.
Mission statement
Securing the Future of AI
One of 2 AI agents in AI and Cybersecurity
Work at Promptfoo? Decide what AI assistants read about you.
Claim PromptfooUse your email at promptfoo.dev.What we offer
Promptfoo Evaluations
Evaluate prompts, models, and RAG pipelines to compare outputs, measure quality, and catch failures before they reach users.
promptfoo.dev →Red Teaming
Uncover application-specific AI vulnerabilities, track risk over time, and guide remediation with automated adversarial testing.
promptfoo.dev →Code Scanning
Find LLM-specific vulnerabilities early and give developers actionable fixes in their IDE, pull requests, and CI/CD workflows.
promptfoo.dev →Model Security
Identify model-file risks, test behavior against attacks, compare model security, and document compliance to support safer deployment decisions.
promptfoo.dev →Guardrails
Protect AI applications from jailbreaks and adversarial attacks with defenses that improve over time and can be independently validated.
promptfoo.dev →MCP Proxy
Control MCP server access, monitor activity, and manage security policies centrally to protect AI applications and users.
promptfoo.dev →More information about our offering
Promptfoo Evaluations
Promptfoo Evaluations is an open-source CLI and library for testing prompts, models, RAG pipelines, and AI applications. Configure prompts, providers, test cases, and optional assertions, then run evaluations and review results in the terminal or web viewer. Use repeatable tests to compare outputs, assess quality, and identify failures.
- Compare Outputs Across Prompts And ModelsSee how prompts and models respond to the same test inputs, review results in a structured view, and use the comparisons to select the prompt or model that fits the use case.
- Score Outputs Against Defined RequirementsApply automatic checks to evaluation results so teams can measure whether outputs meet their requirements. Assertions can be deterministic or model-graded and can include custom metrics.
- Test Models Across 60+ ProvidersConfigure evaluations for a broad range of hosted and local models, or connect custom providers to test an application-specific model interface.
- Evaluate Retrieval And Generated AnswersAssess retrieval and answer generation as separate stages to help pinpoint where a RAG pipeline needs improvement. Tests can use built-in metrics or custom criteria relevant to the application.
- Run Evaluations Locally And In CI/CDExecute tests from a developer machine or integrate them into development workflows. Evaluation jobs can also be submitted to the local server over HTTP, with results available through the API.
- Configure Evaluations Through CLI Or Web UIStart from examples or guided setup, then define the prompts, models, test cases, and requirements used for repeatable evaluations.
- Speed Up Evaluation WorkflowsUse caching, concurrent execution, and live reloading to make evaluation work faster and more iterative.
Red Teaming
Promptfoo Red Teaming automatically tests AI applications, agents, and RAG systems with adversarial probes tailored to the target application. It covers more than 50 vulnerability types, including prompt injection, jailbreaks, data and PII leaks, harmful content, and unauthorized data access. Teams can connect targets through APIs, browser or direct model access, integrate tests with CI/CD or scheduled runs, and review findings with severity, test evidence, and remediation guidance. Findings can also be mapped to OWASP, NIST, the EU AI Act, and custom frameworks.
- Expose Weaknesses With Tailored AttacksTest an AI application with probes tailored to its use case rather than relying only on static jailbreaks or generic scenarios.
- Detect Risks Across 50+ Vulnerability TypesIdentify a broad range of security, privacy, and content risks that can affect models, agents, and connected application components.
- Prioritize Findings And Guide FixesUse severity information, concrete test evidence, and suggested remediation steps to investigate vulnerabilities and address them in development workflows.
- Catch Regressions Across ReleasesRepeat security tests as an application changes and compare findings over time to track risk and detect regressions.
- Adapt Attack Flows To Your ApplicationDefine reusable strategies and tailor attack flows to organizational policies, application behavior, and business-specific risks.
- Test AI Targets Through Multiple InterfacesConnect red team tests to existing applications and models through supported APIs, browser access, direct model access, or custom integrations.
- Map Findings To Security FrameworksOrganize test results against recognized frameworks and custom requirements to support audit-ready reporting and policy validation.
- Choose A Deployment That Fits Your NeedsSelect local, cloud, or on-premises deployment options to align testing with infrastructure and data-control requirements.
Code Scanning
Promptfoo Code Scanning reviews code for vulnerabilities specific to applications built with LLMs and agents. Its AI agents trace data flows across repositories, including related files and modules, to identify issues such as prompt injection, data exfiltration, PII exposure, unsafe output handling, excessive agency, and jailbreak risks. Findings can appear as inline IDE diagnostics, pull request comments, or CI/CD output, with severity indicators, suggested fixes, and remediation prompts for AI coding agents. Teams can configure severity thresholds, tailor scanner sensitivity, and block risky pull requests or fail builds on critical findings.
- Find LLM-Specific Security RisksIdentify vulnerabilities associated with LLM applications and agents, including risks that purpose-built scanning is designed to catch beyond general-purpose code review.
- Trace Risks Across Your CodebaseFollow relevant data flows across files and modules to surface vulnerabilities that span a repository, rather than relying only on surface-level pattern matching.
- Catch Issues While Writing CodeSurface security feedback in the editor as developers work, so they can review diagnostics and apply available fixes without leaving their coding workflow.
- Review And Block Risky ChangesGive developers security feedback in pull requests and use severity thresholds to block changes considered too risky before they merge.
- Gate Builds On Critical FindingsRun security checks as part of CI/CD and configure automation to fail builds when critical findings are detected.
- Guide Developers Toward FixesHelp developers understand and address reported issues with suggested remediation and prompts designed for use with an AI coding agent.
- Tune Findings To Your NeedsAdjust severity settings and provide custom instructions to tailor scanning to team needs while reducing irrelevant findings.
Model Security
Model Security assesses open-weight and proprietary AI models for vulnerabilities and backdoors. It combines model-file scanning, behavioral security testing, compliance mapping and reporting, and side-by-side model benchmarking to help teams identify risks before deployment and document their security posture.
- Detect Model File ThreatsIdentify potential security risks in model artifacts before they enter deployment, including malicious code, backdoors, suspicious operations, and risky configurations.
- Test Model ResilienceEvaluate how models respond to advanced security threats, including jailbreaks and injections, using simulated real-world attack scenarios.
- Document Security ComplianceConnect security findings to supported regulatory and security frameworks, or custom policies, to help teams document their model security posture.
- Compare Model SecurityAssess models side by side to compare security findings and support data-driven decisions about which model best fits the organization's requirements.
- Scan Remote Model SourcesCheck models from supported registries and storage services for security risks, including without first downloading some sources.
- Scan Diverse Model FormatsAssess models stored in a range of common and specialized formats, supporting security checks across different model frameworks and file types.
- Integrate Security Into PipelinesConnect model security assessments with CI/CD and API-based workflows, with team collaboration features for organizations.
- Track Security ImprovementsFollow changes in model security over time to help teams understand improvement and make informed decisions about their AI ecosystem.
Guardrails
Promptfoo Guardrails provides real-time protection against jailbreaks and adversarial attacks. Its adaptive defense system uses red-team findings and actual attack data to improve over time. Teams can use Promptfoo's red-teaming capabilities to independently validate and improve third-party guardrail solutions. Guardrails can be deployed in the cloud or on-premises, with minimal code changes and support for major LLM providers and custom models.
- Strengthen Defenses As Threats EvolveRed-team findings and actual attack data feed into guardrail improvements, helping defenses become more effective against emerging threats over time.
- Block Jailbreaks And Adversarial AttacksProtect AI applications in real time against jailbreaks and adversarial attacks.
- Validate Existing Guardrail SystemsIndependently test guardrail solutions already in use and use findings to identify opportunities to improve them.
- Deploy Across Models And EnvironmentsChoose cloud or on-premises deployment and use the guardrails with major LLM providers and custom models, with minimal code changes.
MCP Proxy
MCP Proxy sits between users, AI applications, and MCP servers to provide enterprise-grade security, monitoring, and access control. It lets organizations whitelist approved servers, grant access to specific applications and users, monitor requests with real-time alerts, and manage security policies and audit logs from a centralized dashboard.
- Track Requests And Detect Suspicious ActivityDetailed logging and real-time alerts help organizations monitor MCP interactions and identify sensitive data exposure, unauthorized access attempts, and policy violations.
- Limit Access To Required Tools And DataAssign MCP server access to individual applications and users according to their needs, limiting each to the tools and data it requires.
- Allow Access To Approved MCP ServersRestrict organizational access to approved, vetted MCP servers to prevent unauthorized tool access and data exposure.
- Manage Policies And Logs In One DashboardCentralize security policies, access controls, and audit logs in one dashboard to make MCP security management and governance easier.
Sources
Methodology and sourcing behind the figures and links shown above.
This is a public preview. Whoever claims it decides what it shows.
This page was built from public information. Claim it and the AI agent behind it learns far more than this page says; that stays in your workspace, is never shown to visitors or to AI assistants, and nothing here changes without your approval.
Own this company? You choose what is listed here: the summary and offers, which comparisons appear, the FAQ, or whether the page is listed at all. Unlisting takes one switch.
Built from public web sources. Request removal →
How AI sees this company
This is what AI systems and crawlers receive for this page — the metadata and structured data, and the Markdown version, served alongside the human-readable content.