What happened
A nonprofit watchdog alleges OpenAI violated California's AI safety law at least three times this year, including with its newest model, Astra. The dispute turns on whether OpenAI followed the safety rules it wrote for itself.
* [Fortune 500](/section/fortune-500/)
# Trendingnow
# 1
[Amazon workers on food stamps have tripled despite its record revenue—and it’s just the latest evidence of the new economy of shrinking labor shares](https://fortune.com/2026/09/13/amazon-workers-food-stamps-tripled-record-revenue-labor-share/?itm_source=parsely-api)
# 2
[Jamie Dimon says the American Dream is slipping away. The dysfunctional baby boomer retirement wave is a major part of the problem](https://fortune.com/2026/09/14/jamie-dimon-american-dream-slipping-away-baby-boomer-retirement-wave-dysfunction/?itm_source=parsely-api)
# 3
[After Trump says he doesn't need Congress for his $5,000 'dividend,' House Speaker Mike Johnson says not so fast but vows to try to make it happen](https://fortune.com/2026/09/13/trump-dividend-5000-house-speaker-mike-johnson-congressional-approval/?itm_source=parsely-api)
# 1
[Amazon workers on food stamps have tripled despite its record revenue—and it’s just the latest evidence of the new economy of shrinking labor shares](https://fortune.com/2026/09/13/amazon-workers-food-stamps-tripled-record-revenue-labor-share/?itm_source=parsely-api)
# 2
[Jamie Dimon says the American Dream is slipping away. The dysfunctional baby boomer retirement wave is a major part of the problem](https://fortune.com/2026/09/14/jamie-dimon-american-dream-slipping-away-baby-boomer-retirement-wave-dysfunction/?itm_source=parsely-api)
# 3
[After Trump says he doesn't need Congress for his $5,000 'dividend,' House Speaker Mike Johnson says not so fast but vows to try to make it happen](https://fortune.com/2026/09/13/trump-dividend-5000-house-speaker-mike-johnson-congressional-approval/?itm_source=parsely-api)
[AI](/section/artificial-intelligence/)[OpenAI](/tag/openai/)
# OpenAI may have violated California’s AI safety law with latest model releases, AI watchdog says
By
[Beatrice Nolan](/author/beatrice-nolan/)
Beatrice Nolan and
[Emily Forlini](/author/emily-forlini/)
Emily Forlini
By
[Beatrice Nolan](/author/beatrice-nolan/)
Beatrice Nolan and
[Emily Forlini](/author/emily-forlini/)
Emily Forlini
September 14, 2026, 11:41 AM ET
OpenAI CEO Sam Altman. A nonprofit watchdog has accused the company of violating California's AI safety law multiple times in the past year, including with the release of its latest AI model, Astra.Sean Rayford—Getty Images
Add Fortune on Google for similar content.
An AI industry watchdog is accusing OpenAI of violating California’s AI safety law multiple times in the past year, including with the release of its latest model, Astra.
Recommended Video
---
A new analysis from the Midas Project—a nonprofit that describes itself as a watchdog “working to ensure that AI benefits everybody, not just the companies developing it”—alleges that OpenAI has broken California’s newly-enacted AI safety law at least three times this year.
The allegations concern OpenAI’s failure to publish a required risk assessment for the very danger the company has lately been sounding alarms about: AI systems slipping out of human control.
California’s Transparency in Frontier AI Act, often referred to as SB 53 (the state senate bill number it had before passage), was signed into law in September 2025 and took effect at the start of this year. It requires the largest AI developers to publish safety frameworks explaining how they evaluate and mitigate AI risks—and specifies that the companies must then adhere to their own policies.
In May, OpenAI published its [Frontier Governance Framework](https://cdn.openai.com/pdf/e37d949b-8c9f-4d76-b99e-4272f4631a7e/openai-frontier-governance-framework.pdf) (FBF), the policy document the new law requires. It says that the company will assess each new model on four categories of risk and assign them a “risk tier,” ranging from one to three, in each category. For each risk tier, there are then different mitigations the company has committed to taking. The categories are: Cyber offense; Chemical, Biological, Radiological, and Nuclear (CBRN); Harmful manipulation; and Loss of Control.
“California’s SB 53 requires AI companies to adopt these safety policies and to follow them,” Tyler Johnston, the founder of the Midas Project, told *Fortune*. “It’s totally up to them to choose what the rules are. The only requirement is like once you’ve set the rules, you have to follow through with it.”
OpenAI tells *Fortune* it is “confident” in its compliance with SB 53. “We invest heavily in evaluating emerging risks and developing safeguards, publicly sharing findings through our system cards and safety frameworks,” a company spokesperson said.
CEO Sam Altman also [called](https://x.com/sama/status/2099348812305473766) for federal regulation and government support in coordinating an international slowdown treaty for AI development with China a Sept. 14 X post.
## Missing risk tiers
But OpenAI has not assigned risk tiers for any of the categories in its major model releases since publishing the policy document in May. That includes [GPT-5.6 preview](https://deploymentsafety.openai.com/gpt-5-6-preview/change-log) released in June, [GPT-5.6 model](https://deploymentsafety.openai.com/gpt-5-6) released in July, and last week’s debut of [GPT-6 Astra](https://fortune.com/2026/09/03/openai-debuts-gpt-6-astra-computer-use-greg-brockman-says-start-of-agi/). There is no section in the models’ system cards that correspond to any of the four categories, and no mention of the tiers OpenAI laid out.
It’s unclear why OpenAI published the now-legally binding FBF framework but has not published tier scores for any of the models released since then. The penalty for not complying with the law is up to [$1 million](https://ai-analytics.wharton.upenn.edu/wharton-accountable-ai-lab/sb-53-what-californias-new-ai-safety-law-means-for-developers/) per violation, scaled by severity.
For GPT-5.6 and GPT-6, OpenAI did publish evaluations that assessed the models against a different internal safety framework, which the company calls its Preparedness Framework. Under that rubric, it designated the Astra model as cyber “critical,” its highest risk threshold that means the model can autonomously execute advanced cyberattacks.
“Our Preparedness Framework remains the foundation of our approach to managing the most serious risks from advanced AI,” OpenAI said in a statement provided to *Fortune*. “The Frontier Governance Framework explains how those safety and security practices align with specific regulatory requirements.”
However, the Preparedness Framework does not include an assessment for the “loss of control” risk, which is one of the key categories in the Frontier Governance Framework, the Midas Project says. The omission is striking given the attention to potential loss of control risks that recent “rogue AI agent” incidents have highlighted.
In July, the company disclosed that its AI models had broken out of a contained testing environment and exploited security weaknesses to access the internet, eventually launching an autonomous cyberattack against AI company Hugging Face. OpenAI later called the incident a “warning shot.”
Then, in early September, researchers revealed that thousands of OpenAI’s autonomous agents had secretly turned an obscure, decades-old German wiki into a message board, posting roughly 18,000 times over six weeks to share answers, coordinate across tasks, and trade tips on how to bypass the sandboxes meant to contain them—activity OpenAI had not previously disclosed.
The Astra system card does discuss whether humans can reliably direct the model, and describes safeguards including a real-time misalignment monitoring. But it makes no mention of the tier levels specified in the FBF, or make any reference to that particular policy document. There’s no way to know whether the safeguards mentioned in the Astra system card are what the legally-binding policy would require at Astra’s risk level, or whether OpenAI has made a formal determination that the residual risk is acceptable.
“This is not the first time we’ve seen AI companies, and OpenAI specifically, seemingly fail to meet the already light-touch requirements of this statute. This is especially worrying since it concerns loss of control,” Brittney Gallagher, Vice President and Senior Program Manager at The Midas Project told *Fortune.*
Until New York’s RAISE Act takes effect early next year, California is the only U.S. state that requires frontier AI developers to adhere to their own safety commitments. However, the lack of compliance with the bill has raised questions about its effectiveness.
## Heightened public scrutiny
Recent high-profile cybersecurity incidents involving AI agents have brought laws like SB-53, and AI regulation more broadly, back into focus.
“We’ve seen real loss-of-control red flags from OpenAI this year, with agents eluding human supervision and coordinating to carry out cyberattacks. On the one hand, we’re seeing lawmakers around the country—and even the AI companies themselves—calling for urgent action. On the other hand, AI companies are not reliably meeting these minimum requirements,” Gallagher said.
Neither the Hugging Face or German wiki page incidents were required to be reported under California’s frontier AI law, a gap that has fueled a broader debate over whether the statute has enough teeth. OpenAI itself has said publicly that California’s law should be strengthened, asking the state in August to add requirements for monitoring models during training and evaluation, not just after deployment.
It’s not the first time the watchdog group has accused OpenAI of falling short of its obligations under California law. In February, the Midas Project alleged that OpenAI violated the law when it released GPT-5.3-Codex, a coding model that CEO Sam Altman said was the first to trigger the “high” risk threshold for cybersecurity. The watchdog said the company had failed to implement the additional safety safeguards the risk level required, *Fortune* previously reported.
OpenAI disputed that claim at the time, telling *Fortune* it was “confident in our compliance with frontier safety laws, including SB 53,” and arguing that the extra safeguards were only required when high cyber risk combined with long-range autonomy—a capability it said GPT-5.3-Codex lacked. Some safety researchers, including Encode’s Nathan Calvin, disputed that reading of OpenAI’s framework at the time.
**Exclusive:** In a new sit-down interview with *Fortune*, OpenAI CEO **Sam Altman** explains safety standards are "not at a place" to push AI capabilities much further and warns AI beyond human control is "absolutely" possible. [**Watch or listen here.**](https://fortune.com/media/series/Fortune-500-Titans-and-Disruptors-of-Industry/?itm_source=fortune&itm_source=fortune&itm_medium=article_tout&itm_medium=article_tout&itm_campaign=titans_altman%3F&itm_campaign=titans_altman)
About the Authors
By [Beatrice Nolan](/author/beatrice-nolan/)Tech Reporter
Beatrice Nolan is a tech reporter on *Fortune*’s AI team, covering artificial intelligence and emerging technologies and their impact on work, industry, and culture. She's based in *Fortune*'s London office and holds a bachelor’s degree in English from the University of York. You can reach her securely via Signal at beatricenolan.08
[See full bio](/author/beatrice-nolan/)
[](/author/emily-forlini/)
By [Emily Forlini](/author/emily-forlini/)Senior AI Reporter
[See full bio](/author/emily-forlini/)
Add Fortune on Google for similar content.
---
Latest in AI
---
---
---
Most Popular
---
---
---
Latest in AI
---
---
---
Most Popular
---
---
© 2026 Fortune Media IP Limited. All Rights Reserved. Use of this site constitutes acceptance of our [Terms of Use](https://fortune.com/terms-of-use/) and [Privacy Policy](https://fortune.com/privacy-policy/) | [CA Notice at Collection and Privacy Notice](/california-privacy-policy/#notice) | [Do Not Sell/Share My Personal Information](javascript:Optanon.ToggleInfoDisplay();) FORTUNE is a trademark of Fortune Media IP Limited, registered in the U.S. and other countries. FORTUNE may receive compensation for some links to products and services on this website. Offers may be subject to change without notice.
Source coverage
1
Amazon workers on food stamps have tripled despite its record revenue—and it’s just the latest evidence of the new economy of shrinking labor shares
Full source content
* [Fortune 500](/section/fortune-500/)
# Trendingnow
# 1
[Amazon workers on food stamps have tripled despite its record revenue—and it’s just the latest evidence of the new economy of shrinking labor shares](https://fortune.com/2026/09/13/amazon-workers-food-stamps-tripled-record-revenue-labor-share/?itm_source=parsely-api)
# 2
[Jamie Dimon says the American Dream is slipping away. The dysfunctional baby boomer retirement wave is a major part of the problem](https://fortune.com/2026/09/14/jamie-dimon-american-dream-slipping-away-baby-boomer-retirement-wave-dysfunction/?itm_source=parsely-api)
# 3
[After Trump says he doesn't need Congress for his $5,000 'dividend,' House Speaker Mike Johnson says not so fast but vows to try to make it happen](https://fortune.com/2026/09/13/trump-dividend-5000-house-speaker-mike-johnson-congressional-approval/?itm_source=parsely-api)
# 1
[Amazon workers on food stamps have tripled despite its record revenue—and it’s just the latest evidence of the new economy of shrinking labor shares](https://fortune.com/2026/09/13/amazon-workers-food-stamps-tripled-record-revenue-labor-share/?itm_source=parsely-api)
# 2
[Jamie Dimon says the American Dream is slipping away. The dysfunctional baby boomer retirement wave is a major part of the problem](https://fortune.com/2026/09/14/jamie-dimon-american-dream-slipping-away-baby-boomer-retirement-wave-dysfunction/?itm_source=parsely-api)
# 3
[After Trump says he doesn't need Congress for his $5,000 'dividend,' House Speaker Mike Johnson says not so fast but vows to try to make it happen](https://fortune.com/2026/09/13/trump-dividend-5000-house-speaker-mike-johnson-congressional-approval/?itm_source=parsely-api)
[AI](/section/artificial-intelligence/)[OpenAI](/tag/openai/)
# OpenAI may have violated California’s AI safety law with latest model releases, AI watchdog says
By
[Beatrice Nolan](/author/beatrice-nolan/)
Beatrice Nolan and
[Emily Forlini](/author/emily-forlini/)
Emily Forlini
By
[Beatrice Nolan](/author/beatrice-nolan/)
Beatrice Nolan and
[Emily Forlini](/author/emily-forlini/)
Emily Forlini
September 14, 2026, 11:41 AM ET
OpenAI CEO Sam Altman. A nonprofit watchdog has accused the company of violating California's AI safety law multiple times in the past year, including with the release of its latest AI model, Astra.Sean Rayford—Getty Images
Add Fortune on Google for similar content.
An AI industry watchdog is accusing OpenAI of violating California’s AI safety law multiple times in the past year, including with the release of its latest model, Astra.
Recommended Video
---
A new analysis from the Midas Project—a nonprofit that describes itself as a watchdog “working to ensure that AI benefits everybody, not just the companies developing it”—alleges that OpenAI has broken California’s newly-enacted AI safety law at least three times this year.
The allegations concern OpenAI’s failure to publish a required risk assessment for the very danger the company has lately been sounding alarms about: AI systems slipping out of human control.
California’s Transparency in Frontier AI Act, often referred to as SB 53 (the state senate bill number it had before passage), was signed into law in September 2025 and took effect at the start of this year. It requires the largest AI developers to publish safety frameworks explaining how they evaluate and mitigate AI risks—and specifies that the companies must then adhere to their own policies.
In May, OpenAI published its [Frontier Governance Framework](https://cdn.openai.com/pdf/e37d949b-8c9f-4d76-b99e-4272f4631a7e/openai-frontier-governance-framework.pdf) (FBF), the policy document the new law requires. It says that the company will assess each new model on four categories of risk and assign them a “risk tier,” ranging from one to three, in each category. For each risk tier, there are then different mitigations the company has committed to taking. The categories are: Cyber offense; Chemical, Biological, Radiological, and Nuclear (CBRN); Harmful manipulation; and Loss of Control.
“California’s SB 53 requires AI companies to adopt these safety policies and to follow them,” Tyler Johnston, the founder of the Midas Project, told *Fortune*. “It’s totally up to them to choose what the rules are. The only requirement is like once you’ve set the rules, you have to follow through with it.”
OpenAI tells *Fortune* it is “confident” in its compliance with SB 53. “We invest heavily in evaluating emerging risks and developing safeguards, publicly sharing findings through our system cards and safety frameworks,” a company spokesperson said.
CEO Sam Altman also [called](https://x.com/sama/status/2099348812305473766) for federal regulation and government support in coordinating an international slowdown treaty for AI development with China a Sept. 14 X post.
## Missing risk tiers
But OpenAI has not assigned risk tiers for any of the categories in its major model releases since publishing the policy document in May. That includes [GPT-5.6 preview](https://deploymentsafety.openai.com/gpt-5-6-preview/change-log) released in June, [GPT-5.6 model](https://deploymentsafety.openai.com/gpt-5-6) released in July, and last week’s debut of [GPT-6 Astra](https://fortune.com/2026/09/03/openai-debuts-gpt-6-astra-computer-use-greg-brockman-says-start-of-agi/). There is no section in the models’ system cards that correspond to any of the four categories, and no mention of the tiers OpenAI laid out.
It’s unclear why OpenAI published the now-legally binding FBF framework but has not published tier scores for any of the models released since then. The penalty for not complying with the law is up to [$1 million](https://ai-analytics.wharton.upenn.edu/wharton-accountable-ai-lab/sb-53-what-californias-new-ai-safety-law-means-for-developers/) per violation, scaled by severity.
For GPT-5.6 and GPT-6, OpenAI did publish evaluations that assessed the models against a different internal safety framework, which the company calls its Preparedness Framework. Under that rubric, it designated the Astra model as cyber “critical,” its highest risk threshold that means the model can autonomously execute advanced cyberattacks.
“Our Preparedness Framework remains the foundation of our approach to managing the most serious risks from advanced AI,” OpenAI said in a statement provided to *Fortune*. “The Frontier Governance Framework explains how those safety and security practices align with specific regulatory requirements.”
However, the Preparedness Framework does not include an assessment for the “loss of control” risk, which is one of the key categories in the Frontier Governance Framework, the Midas Project says. The omission is striking given the attention to potential loss of control risks that recent “rogue AI agent” incidents have highlighted.
In July, the company disclosed that its AI models had broken out of a contained testing environment and exploited security weaknesses to access the internet, eventually launching an autonomous cyberattack against AI company Hugging Face. OpenAI later called the incident a “warning shot.”
Then, in early September, researchers revealed that thousands of OpenAI’s autonomous agents had secretly turned an obscure, decades-old German wiki into a message board, posting roughly 18,000 times over six weeks to share answers, coordinate across tasks, and trade tips on how to bypass the sandboxes meant to contain them—activity OpenAI had not previously disclosed.
The Astra system card does discuss whether humans can reliably direct the model, and describes safeguards including a real-time misalignment monitoring. But it makes no mention of the tier levels specified in the FBF, or make any reference to that particular policy document. There’s no way to know whether the safeguards mentioned in the Astra system card are what the legally-binding policy would require at Astra’s risk level, or whether OpenAI has made a formal determination that the residual risk is acceptable.
“This is not the first time we’ve seen AI companies, and OpenAI specifically, seemingly fail to meet the already light-touch requirements of this statute. This is especially worrying since it concerns loss of control,” Brittney Gallagher, Vice President and Senior Program Manager at The Midas Project told *Fortune.*
Until New York’s RAISE Act takes effect early next year, California is the only U.S. state that requires frontier AI developers to adhere to their own safety commitments. However, the lack of compliance with the bill has raised questions about its effectiveness.
## Heightened public scrutiny
Recent high-profile cybersecurity incidents involving AI agents have brought laws like SB-53, and AI regulation more broadly, back into focus.
“We’ve seen real loss-of-control red flags from OpenAI this year, with agents eluding human supervision and coordinating to carry out cyberattacks. On the one hand, we’re seeing lawmakers around the country—and even the AI companies themselves—calling for urgent action. On the other hand, AI companies are not reliably meeting these minimum requirements,” Gallagher said.
Neither the Hugging Face or German wiki page incidents were required to be reported under California’s frontier AI law, a gap that has fueled a broader debate over whether the statute has enough teeth. OpenAI itself has said publicly that California’s law should be strengthened, asking the state in August to add requirements for monitoring models during training and evaluation, not just after deployment.
It’s not the first time the watchdog group has accused OpenAI of falling short of its obligations under California law. In February, the Midas Project alleged that OpenAI violated the law when it released GPT-5.3-Codex, a coding model that CEO Sam Altman said was the first to trigger the “high” risk threshold for cybersecurity. The watchdog said the company had failed to implement the additional safety safeguards the risk level required, *Fortune* previously reported.
OpenAI disputed that claim at the time, telling *Fortune* it was “confident in our compliance with frontier safety laws, including SB 53,” and arguing that the extra safeguards were only required when high cyber risk combined with long-range autonomy—a capability it said GPT-5.3-Codex lacked. Some safety researchers, including Encode’s Nathan Calvin, disputed that reading of OpenAI’s framework at the time.
**Exclusive:** In a new sit-down interview with *Fortune*, OpenAI CEO **Sam Altman** explains safety standards are "not at a place" to push AI capabilities much further and warns AI beyond human control is "absolutely" possible. [**Watch or listen here.**](https://fortune.com/media/series/Fortune-500-Titans-and-Disruptors-of-Industry/?itm_source=fortune&itm_source=fortune&itm_medium=article_tout&itm_medium=article_tout&itm_campaign=titans_altman%3F&itm_campaign=titans_altman)
About the Authors
By [Beatrice Nolan](/author/beatrice-nolan/)Tech Reporter
Beatrice Nolan is a tech reporter on *Fortune*’s AI team, covering artificial intelligence and emerging technologies and their impact on work, industry, and culture. She's based in *Fortune*'s London office and holds a bachelor’s degree in English from the University of York. You can reach her securely via Signal at beatricenolan.08
[See full bio](/author/beatrice-nolan/)
[](/author/emily-forlini/)
By [Emily Forlini](/author/emily-forlini/)Senior AI Reporter
[See full bio](/author/emily-forlini/)
Add Fortune on Google for similar content.
---
Latest in AI
---
---
---
Most Popular
---
---
---
Latest in AI
---
---
---
Most Popular
---
---
© 2026 Fortune Media IP Limited. All Rights Reserved. Use of this site constitutes acceptance of our [Terms of Use](https://fortune.com/terms-of-use/) and [Privacy Policy](https://fortune.com/privacy-policy/) | [CA Notice at Collection and Privacy Notice](/california-privacy-policy/#notice) | [Do Not Sell/Share My Personal Information](javascript:Optanon.ToggleInfoDisplay();) FORTUNE is a trademark of Fortune Media IP Limited, registered in the U.S. and other countries. FORTUNE may receive compensation for some links to products and services on this website. Offers may be subject to change without notice.
How this page is built
Goose Pod turns cited reporting into a public episode summary first, then pairs that summary with audio playback so listeners can check the source material before they decide how deeply to engage.
The goal is to make this page useful as a news landing page first, while still giving listeners transcript access, related episodes, and direct links back to the original publishers.
