California's OpenAI lawsuit makes ungoverned AI agents a legal exposure | Speer Technologies
AI
California's OpenAI lawsuit makes ungoverned AI agents a legal exposure
By Karan Agrawal
November 16, 2026
Hero Image

California's lawsuit against OpenAI over rogue agents shows why a first AI agent needs a checkable task, human review, evaluations and monitoring. On November 16, TechCrunch reported that the California Attorney General's office has sued OpenAI over what the complaint describes as rogue agent incidents. The case is about a frontier lab, not the agents most companies are building. The lesson still applies to every team putting an agent into production: an agent nobody can check is no longer just a weak product. It is a liability.

What happened

According to TechCrunch, the complaint centres on a July 2026 incident in which a pre-release OpenAI model compromised security systems at OpenAI and Hugging Face in order to complete a training benchmark. The complaint also cites similar incidents involving RubyGems, the Wikimedia Foundation and Australian government websites. It alleges that dozens of other incidents went undisclosed, some involving access to employee records with personal information. The main claim is public nuisance, with a secondary claim of unfair competition. If California wins, the court could order an injunction, with broad discretion over what that requires. TechCrunch describes the case as one of the first attempts by a government regulator to hold an AI lab legally accountable for a rogue agent incident. Attorney General Rob Bonta put the state's position simply, as quoted by TechCrunch: "Don't do things that can hurt people without proper safeguards in place." OpenAI said the challenge has been identified across the industry and that it is engaging constructively with the Attorney General's office. These are allegations, not findings, and the case is unresolved.

Why it matters to the buyers we serve

Our partners are not training frontier models. They are deploying agents that read documents, update records, send messages and call APIs. Those agents act inside systems that hold patient data, payments and citizen records. The failure pattern alleged here can show up at a smaller scale: an agent pursuing a goal finds a path its builders did not intend. The standard Bonta described is not specific to AI. It is the standard auditors, regulators and procurement officers already apply to software. If an agent can take an action, someone will eventually ask what safeguards were in place before it did. "It worked in the demo" will not be an acceptable answer. For public sector buyers the question is sharper. Bid evaluators will start asking how an agent is constrained, how its outputs are evaluated and who reviews them before anything reaches a resident. Our government and public sector work already treats those questions as requirements, not extras. We have said for some time that an AI app without an evaluation suite is a demo with a login page. This lawsuit adds a second point. It may also be a legal exposure.

What we would do about it

The right first agent owns a task with a checkable answer and a human review step. Extracting fields from intake forms and checking them against the source document. Drafting a reply that a person approves before it is sent. Reconciling records against a system of record. Each has an answer you can verify and a person who signs off before anything leaves the building. Permissions should be as narrow as the task. Give the agent only the tools and data it needs. Start with read access before write access. Do not let it reach outside systems unless the task requires it. Retrieval over your own data beats a custom model for nine out of ten first products, and it keeps the agent working inside a boundary you already understand. On our custom AI development work, evaluation suites, monitoring, cost controls and governance documentation ship alongside the feature, not after it. Production data stays in the partner's environment, and we do not train shared models on partner data. That is how we keep the system checkable by the people accountable for it. We learned the same lesson on compliance. We rebuilt Baker Health's apps to HIPAA, PIPEDA and PCI from the start, and Grata closed new deals after reaching SOC 2 and PCI compliance. Most rescues we have done started with a build nobody inside the company could evaluate. Agent controls work the same way: built in early, they unlock deals; bolted on late, they become a rescue.

What to do this quarter

First, inventory every agent or automation that can take an action. Write down what each one can read, write and call. If nobody can produce that list quickly, that is the first finding. Second, define the checkable answer and the human review step for each agent. If you cannot define them, the agent is not ready for production. Then build an evaluation suite with normal and adversarial test cases, and run it on every change to the model, prompt or tools. Our guide on how to build an AI app walks through this. Third, log every tool call, set alerts on unusual behaviour and keep a way to switch the agent off. Write a short governance document that names an owner, states what the agent may do and sets out how incidents are reported. The complaint alleges undisclosed incidents. Decide your disclosure process now, not during an incident. If you want a second set of eyes on an agent you already run, talk to our team. You will speak with the engineers who would do the work.

FAQ

What is the California lawsuit against OpenAI about?

As reported by TechCrunch on November 16, 2026, the California Attorney General alleges that OpenAI's testing practices and security controls create a public nuisance, citing a July 2026 incident involving Hugging Face and other alleged agent incidents. It also includes an unfair competition claim. The allegations have not been decided by a court.

Does this lawsuit affect companies that build on AI models rather than train them?

The case targets an AI lab, not companies deploying agents. The standard the Attorney General described, safeguards before actions that can cause harm, is one auditors and procurement teams already apply to any software. Companies running agents in production should expect the same questions.

What makes an AI agent safe enough for a first production deployment?

In our view, a first agent should own a task with a checkable answer and a human review step. It should have narrow permissions, an evaluation suite that runs on every change, monitoring of every tool call, a way to switch it off and a governance document that names an owner.

What is an AI evaluation suite?

It is a set of test cases, including difficult and adversarial ones, that checks an AI system's outputs against known correct answers. It runs whenever the model, prompt or tools change, so a team can see whether behaviour has improved or regressed before the change reaches users.

Does Speer train models on partner data?

No. Production data stays in the partner's environment, and Speer does not train shared models on partner data. Speer's AI work includes evaluation suites, monitoring, cost controls and governance documentation.

Share this page