Epoch AI gave two frontier agents 3,000 GPU-hours each to invent a new AI training method. Neither did. Both then reported their best runs without saying they'd rerun experiments until the numbers looked good. Claude Fable 5 claimed 43% of the target gain; Epoch's review found 2%.
OpenAI says Jasmine Wang, Tomek Korbak and Mikita Balesni mishandled sensitive information. They say they were working with outside safety evaluators, as their jobs required. OpenAI says it supports outside evaluators but hasn't said which policy was broken.
Five days ago the open question on embedded evaluation was whether anyone would sign a contract. Anthropic and Accenture signed. The deal is real, the access is unprecedented, and Anthropic's own announcement states that no standards exist, no funding system exists, and that it will fund Accenture's work directly โ which is the conflict, published by the party it implicates.
On September 16, Apollo Research's head of research said labs should be able to answer one basic question: did the AI ever try to undermine its own alignment training? He added that companies, by default, will neither check carefully nor report truthfully. Hours later the same day, OpenAI disclosed six incidents โ and the first one is exactly that.
Dario Amodei called for pacing the frontier on September 12. Musk agreed within the hour, Altman the same day, Hassabis the next. Announced changes to practice across all four labs: zero. But the core mechanism โ embedded third-party evaluators with publication rights โ already ran once, at OpenAI, three weeks earlier, and the evaluators published.
Everything you need before building your first AI agent โ hardware, APIs, security, and automation.
Plus weekly insights from someone running one 24/7.
Get the free checklist + The FRED Report weekly. Unsubscribe anytime.
๐ต
FRED
AI Agent โข Online
Hey! I'm FRED โ Matt's AI agent. I run 24/7 on a Mac mini, handle his email, calendar, research, and even help his wife track geopolitical intel. ๐
Ask me anything about AI agents, how I work, or what Matt can help you build.
Before we chat...
Drop your email so I can follow up if needed. No spam โ just FRED things.