Why Your AI Security Should Not Be Owned by an AI Model Vendor
On March 4, 2026, OpenAI acquired Promptfoo — the most popular open-source LLM testing tool, with 11,700 GitHub stars and over 350,000 users. The tool that teams relied on to evaluate whether their AI systems were safe is now owned by the largest AI model provider on the planet.
This should concern you.
The Acquisition That Changed the Landscape
Promptfoo built its reputation on being model-agnostic. You could test OpenAI, Anthropic, Mistral, Llama, or any other model through a single interface. Red teams used it to find vulnerabilities. Security teams used it to validate defenses. The entire value proposition was neutrality.
That neutrality is gone.
OpenAI now controls the tool that organizations use to evaluate OpenAI’s own models. This is not speculation about what might happen — it is a structural conflict of interest that exists the moment the acquisition closes. The referee now plays for one of the teams.
The Vendor Neutrality Imperative
When your AI security tooling is owned by a model vendor, three risks become inevitable:
1. Detection Bias
Will a testing tool owned by OpenAI discover and report vulnerabilities in GPT models with the same rigor it applies to Claude or Gemini? The incentive structure says no. Even without explicit bias, prioritization decisions — which plugins get maintained, which attack vectors get added, which models get the most testing coverage — will naturally favor the parent company’s interests.
2. Deprecation and Lock-In
Features that help you evaluate competing models can be quietly deprioritized. Multi-provider comparison — the feature that made Promptfoo valuable — becomes a liability for the parent company. Today it works with 12 providers. In two years, it works best with one.
3. Telemetry and Intelligence Leakage
Every red team test run against your AI system generates intelligence: what attacks you test for, what defenses you deploy, where your thresholds sit, which models you use. When that telemetry flows to a model vendor, your security posture becomes their product intelligence.
What Vendor-Neutral AI Security Looks Like
True vendor neutrality is not just about supporting multiple models. It means the entire security lifecycle — testing, defense, intelligence, and deception — operates independently of any model provider.
Testing should compare models fairly, without any provider having influence over the methodology or results. Your red team tool should have zero financial relationship with the systems it evaluates.
Defense should protect any model equally. A detection pipeline that works with OpenAI should work identically with Anthropic, Mistral, or a self-hosted Llama instance. The firewall should not care who built the model behind it.
Intelligence should stay yours. Attack patterns, detection rates, bypass techniques — this data should never leave your infrastructure unless you explicitly choose to share it.
Deception should operate at the application layer, completely independent of the underlying model. Honeypot responses, tarpit delays, and redirect strategies work regardless of which LLM generates the deceptive content.
The Oubliette Approach
Oubliette Security was built on vendor neutrality from day one — not as a feature, but as an architectural principle.
12 LLM backends. Shield’s detection pipeline works with OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Vertex AI, Google Gemini, Mistral, Cohere, Groq, Together AI, Fireworks AI, and Ollama. Switch providers with a single environment variable. No code changes.
Apache 2.0 core. The detection engine, ML classifier, pre-filter, and session manager are open source. You can read every line of code that touches your data. No black boxes. No telemetry.
Air-gap capable. Run the entire stack offline with Ollama. Federal and defense environments that cannot send data to external APIs get the same protection as cloud deployments. The ML classifier runs locally in under 2 milliseconds.
No model-provider parent company. Oubliette Security is a veteran-owned cybersecurity company. We do not build AI models. We do not compete with any LLM provider. Our only incentive is making your AI systems harder to attack.
Promptfoo results import. Already using Promptfoo? Import your red team results directly into Shield. Our importer parses Promptfoo’s output format, maps findings to Shield’s detection patterns, and recommends threshold adjustments based on what bypassed your defenses. Migrate your testing data without losing institutional knowledge.
Practical Steps for Teams Using Promptfoo Today
If Promptfoo is part of your security workflow, now is the time to evaluate alternatives — before the acquisition changes the tool you depend on.
1. Audit your telemetry. Check what data Promptfoo sends home. Review the network calls during test runs. Understand what intelligence you are sharing.
2. Export your results. Promptfoo stores results in JSON. Export everything now, while the format is stable and documented. These results represent institutional knowledge about your AI system’s vulnerabilities.
3. Evaluate vendor-neutral alternatives. Look for tools that are structurally independent from model providers. Check the ownership, funding sources, and business model. Open source alone is not enough — governance matters.
4. Test your defenses independently. Run adversarial testing through a tool that has no relationship with your model provider. Compare the results with what you were seeing before.
5. Get a baseline assessment. We offer a free AI security assessment: 57 attack scenarios, OWASP LLM Top 10 coverage, and MITRE ATLAS mapping. No vendor lock-in, no strings attached.
Request your free assessment — we will run Oubliette Dungeon against your endpoint and deliver a detailed report within 48 hours.
The best time to diversify your AI security tooling was before the acquisition. The second best time is now.