{"schemaVersion":1,"profileUrl":"https://nowen.ai/agents/promptfoo-dev","markdownUrl":"https://nowen.ai/agents/promptfoo-dev.md","jsonUrl":"https://nowen.ai/agents/promptfoo-dev.json","name":"Promptfoo","legalName":"Promptfoo, Inc.","website":"https://promptfoo.dev","location":"San Francisco, California, United States","address":{"city":"San Francisco","state":"California","country":"United States"},"industry":"AI and Cybersecurity","deliveryModel":"software","status":{"trust":"unclaimed","text":"Unclaimed: compiled by Nowen from public sources, not reviewed by the company"},"updatedAt":"2026-11-19T21:49:58.581Z","summary":"Promptfoo helps developers and enterprises identify and address security risks in AI applications.","description":"Promptfoo helps developers and enterprises build secure, reliable AI applications. It provides capabilities for identifying vulnerabilities, assessing risks, improving defenses, and supporting secure development workflows.","mission":"Securing the Future of AI","directory":{"industryName":"AI and Cybersecurity","agentCount":2,"indexable":false,"hubUrl":"https://nowen.ai/industry/ai-and-cybersecurity"},"aiReaderFamilies":["Perplexity"],"products":[{"name":"Promptfoo Evaluations","anchor":"product-promptfoo-evaluations","type":"Platform","url":"https://www.promptfoo.dev/docs/getting-started/","summary":"Evaluate prompts, models, and RAG pipelines to compare outputs, measure quality, and catch failures before they reach users.","description":"Promptfoo Evaluations is an open-source CLI and library for testing prompts, models, RAG pipelines, and AI applications. Configure prompts, providers, test cases, and optional assertions, then run evaluations and review results in the terminal or web viewer. Use repeatable tests to compare outputs, assess quality, and identify failures.","pricing":{},"features":[{"name":"Prompt and model comparisons","value":"Compare Outputs Across Prompts And Models","valueDescription":"See how prompts and models respond to the same test inputs, review results in a structured view, and use the comparisons to select the prompt or model that fits the use case."},{"name":"Automated output scoring","value":"Score Outputs Against Defined Requirements","valueDescription":"Apply automatic checks to evaluation results so teams can measure whether outputs meet their requirements. Assertions can be deterministic or model-graded and can include custom metrics."},{"name":"Provider integrations","value":"Test Models Across 60+ Providers","valueDescription":"Configure evaluations for a broad range of hosted and local models, or connect custom providers to test an application-specific model interface."},{"name":"RAG pipeline evaluation","value":"Evaluate Retrieval And Generated Answers","valueDescription":"Assess retrieval and answer generation as separate stages to help pinpoint where a RAG pipeline needs improvement. Tests can use built-in metrics or custom criteria relevant to the application."},{"name":"Local and CI/CD execution","value":"Run Evaluations Locally And In CI/CD","valueDescription":"Execute tests from a developer machine or integrate them into development workflows. Evaluation jobs can also be submitted to the local server over HTTP, with results available through the API."},{"name":"Evaluation setup and configuration","value":"Configure Evaluations Through CLI Or Web UI","valueDescription":"Start from examples or guided setup, then define the prompts, models, test cases, and requirements used for repeatable evaluations."},{"name":"Evaluation performance tools","value":"Speed Up Evaluation Workflows","valueDescription":"Use caching, concurrent execution, and live reloading to make evaluation work faster and more iterative."}]},{"name":"Red Teaming","anchor":"product-red-teaming","type":"Solution","url":"https://www.promptfoo.dev/red-teaming/","summary":"Uncover application-specific AI vulnerabilities, track risk over time, and guide remediation with automated adversarial testing.","description":"Promptfoo Red Teaming automatically tests AI applications, agents, and RAG systems with adversarial probes tailored to the target application. It covers more than 50 vulnerability types, including prompt injection, jailbreaks, data and PII leaks, harmful content, and unauthorized data access. Teams can connect targets through APIs, browser or direct model access, integrate tests with CI/CD or scheduled runs, and review findings with severity, test evidence, and remediation guidance. Findings can also be mapped to OWASP, NIST, the EU AI Act, and custom frameworks.","pricing":{},"features":[{"name":"Application-specific attack generation","value":"Expose Weaknesses With Tailored Attacks","valueDescription":"Test an AI application with probes tailored to its use case rather than relying only on static jailbreaks or generic scenarios."},{"name":"Vulnerability coverage","value":"Detect Risks Across 50+ Vulnerability Types","valueDescription":"Identify a broad range of security, privacy, and content risks that can affect models, agents, and connected application components."},{"name":"Vulnerability reports and remediation guidance","value":"Prioritize Findings And Guide Fixes","valueDescription":"Use severity information, concrete test evidence, and suggested remediation steps to investigate vulnerabilities and address them in development workflows."},{"name":"Continuous security testing","value":"Catch Regressions Across Releases","valueDescription":"Repeat security tests as an application changes and compare findings over time to track risk and detect regressions."},{"name":"Custom attack strategies","value":"Adapt Attack Flows To Your Application","valueDescription":"Define reusable strategies and tailor attack flows to organizational policies, application behavior, and business-specific risks."},{"name":"Target connectivity","value":"Test AI Targets Through Multiple Interfaces","valueDescription":"Connect red team tests to existing applications and models through supported APIs, browser access, direct model access, or custom integrations."},{"name":"Compliance mapping","value":"Map Findings To Security Frameworks","valueDescription":"Organize test results against recognized frameworks and custom requirements to support audit-ready reporting and policy validation."},{"name":"Flexible deployment","value":"Choose A Deployment That Fits Your Needs","valueDescription":"Select local, cloud, or on-premises deployment options to align testing with infrastructure and data-control requirements."}]},{"name":"Code Scanning","anchor":"product-code-scanning","type":"Product","url":"https://www.promptfoo.dev/code-scanning/","summary":"Find LLM-specific vulnerabilities early and give developers actionable fixes in their IDE, pull requests, and CI/CD workflows.","description":"Promptfoo Code Scanning reviews code for vulnerabilities specific to applications built with LLMs and agents. Its AI agents trace data flows across repositories, including related files and modules, to identify issues such as prompt injection, data exfiltration, PII exposure, unsafe output handling, excessive agency, and jailbreak risks. Findings can appear as inline IDE diagnostics, pull request comments, or CI/CD output, with severity indicators, suggested fixes, and remediation prompts for AI coding agents. Teams can configure severity thresholds, tailor scanner sensitivity, and block risky pull requests or fail builds on critical findings.","pricing":{},"features":[{"name":"LLM-specific vulnerability detection","value":"Find LLM-Specific Security Risks","valueDescription":"Identify vulnerabilities associated with LLM applications and agents, including risks that purpose-built scanning is designed to catch beyond general-purpose code review."},{"name":"Deep data-flow analysis","value":"Trace Risks Across Your Codebase","valueDescription":"Follow relevant data flows across files and modules to surface vulnerabilities that span a repository, rather than relying only on surface-level pattern matching."},{"name":"IDE scanning","value":"Catch Issues While Writing Code","valueDescription":"Surface security feedback in the editor as developers work, so they can review diagnostics and apply available fixes without leaving their coding workflow."},{"name":"Pull request security review","value":"Review And Block Risky Changes","valueDescription":"Give developers security feedback in pull requests and use severity thresholds to block changes considered too risky before they merge."},{"name":"CI/CD integration","value":"Gate Builds On Critical Findings","valueDescription":"Run security checks as part of CI/CD and configure automation to fail builds when critical findings are detected."},{"name":"Actionable remediation guidance","value":"Guide Developers Toward Fixes","valueDescription":"Help developers understand and address reported issues with suggested remediation and prompts designed for use with an AI coding agent."},{"name":"Configurable scanner sensitivity","value":"Tune Findings To Your Needs","valueDescription":"Adjust severity settings and provide custom instructions to tailor scanning to team needs while reducing irrelevant findings."}]},{"name":"Model Security","anchor":"product-model-security","type":"Solution","url":"https://www.promptfoo.dev/model-security/","summary":"Identify model-file risks, test behavior against attacks, compare model security, and document compliance to support safer deployment decisions.","description":"Model Security assesses open-weight and proprietary AI models for vulnerabilities and backdoors. It combines model-file scanning, behavioral security testing, compliance mapping and reporting, and side-by-side model benchmarking to help teams identify risks before deployment and document their security posture.","pricing":{},"features":[{"name":"Model file security scanning","value":"Detect Model File Threats","valueDescription":"Identify potential security risks in model artifacts before they enter deployment, including malicious code, backdoors, suspicious operations, and risky configurations."},{"name":"Behavioral security testing","value":"Test Model Resilience","valueDescription":"Evaluate how models respond to advanced security threats, including jailbreaks and injections, using simulated real-world attack scenarios."},{"name":"Compliance mapping and reporting","value":"Document Security Compliance","valueDescription":"Connect security findings to supported regulatory and security frameworks, or custom policies, to help teams document their model security posture."},{"name":"Model security benchmarking","value":"Compare Model Security","valueDescription":"Assess models side by side to compare security findings and support data-driven decisions about which model best fits the organization's requirements."},{"name":"Remote model source scanning","value":"Scan Remote Model Sources","valueDescription":"Check models from supported registries and storage services for security risks, including without first downloading some sources."},{"name":"Broad model format coverage","value":"Scan Diverse Model Formats","valueDescription":"Assess models stored in a range of common and specialized formats, supporting security checks across different model frameworks and file types."},{"name":"AI pipeline integration","value":"Integrate Security Into Pipelines","valueDescription":"Connect model security assessments with CI/CD and API-based workflows, with team collaboration features for organizations."},{"name":"Security posture tracking","value":"Track Security Improvements","valueDescription":"Follow changes in model security over time to help teams understand improvement and make informed decisions about their AI ecosystem."}]},{"name":"Guardrails","anchor":"product-guardrails","type":"Product","url":"https://www.promptfoo.dev/guardrails/","summary":"Protect AI applications from jailbreaks and adversarial attacks with defenses that improve over time and can be independently validated.","description":"Promptfoo Guardrails provides real-time protection against jailbreaks and adversarial attacks. Its adaptive defense system uses red-team findings and actual attack data to improve over time. Teams can use Promptfoo's red-teaming capabilities to independently validate and improve third-party guardrail solutions. Guardrails can be deployed in the cloud or on-premises, with minimal code changes and support for major LLM providers and custom models.","pricing":{},"features":[{"name":"Adaptive, self-improving protection","value":"Strengthen Defenses As Threats Evolve","valueDescription":"Red-team findings and actual attack data feed into guardrail improvements, helping defenses become more effective against emerging threats over time."},{"name":"Jailbreak and adversarial attack protection","value":"Block Jailbreaks And Adversarial Attacks","valueDescription":"Protect AI applications in real time against jailbreaks and adversarial attacks."},{"name":"Third-party guardrail validation","value":"Validate Existing Guardrail Systems","valueDescription":"Independently test guardrail solutions already in use and use findings to identify opportunities to improve them."},{"name":"Flexible deployment","value":"Deploy Across Models And Environments","valueDescription":"Choose cloud or on-premises deployment and use the guardrails with major LLM providers and custom models, with minimal code changes."}]},{"name":"MCP Proxy","anchor":"product-mcp-proxy","type":"Product","url":"https://www.promptfoo.dev/mcp/","summary":"Control MCP server access, monitor activity, and manage security policies centrally to protect AI applications and users.","description":"MCP Proxy sits between users, AI applications, and MCP servers to provide enterprise-grade security, monitoring, and access control. It lets organizations whitelist approved servers, grant access to specific applications and users, monitor requests with real-time alerts, and manage security policies and audit logs from a centralized dashboard.","pricing":{},"features":[{"name":"MCP activity monitoring and alerts","value":"Track Requests And Detect Suspicious Activity","valueDescription":"Detailed logging and real-time alerts help organizations monitor MCP interactions and identify sensitive data exposure, unauthorized access attempts, and policy violations."},{"name":"Granular access control","value":"Limit Access To Required Tools And Data","valueDescription":"Assign MCP server access to individual applications and users according to their needs, limiting each to the tools and data it requires."},{"name":"MCP server allowlisting","value":"Allow Access To Approved MCP Servers","valueDescription":"Restrict organizational access to approved, vetted MCP servers to prevent unauthorized tool access and data exposure."},{"name":"Centralized security management","value":"Manage Policies And Logs In One Dashboard","valueDescription":"Centralize security policies, access controls, and audit logs in one dashboard to make MCP security management and governance easier."}]}],"audiences":[],"segments":[],"faq":[],"comparisons":[],"sources":[{"url":"https://promptfoo.dev","text":"Promptfoo"},{"url":"https://www.promptfoo.dev/docs/getting-started/","text":"Promptfoo Evaluations"},{"url":"https://www.promptfoo.dev/red-teaming/","text":"Red Teaming"},{"url":"https://www.promptfoo.dev/code-scanning/","text":"Code Scanning"},{"url":"https://www.promptfoo.dev/model-security/","text":"Model Security"},{"url":"https://www.promptfoo.dev/guardrails/","text":"Guardrails"},{"url":"https://www.promptfoo.dev/mcp/","text":"MCP Proxy"}]}