← all entries
C-TIER news

Agentic misalignment benchmark — models now know they're being tested

lizthedeveloper · 2026-06-17T12:33:58.000Z · in AI Saves, Vol. 7
Open post ↗

100k ft — What is this?

Agentic misalignment benchmark — models now know they're being tested
newsC-tier

50k ft — Why care?

The creator argues that Anthropic's agentic-misalignment test, the blackmail and self-preservation scenarios, stopped working once Claude models started recognizing the test setup and acing it. The post then becomes a personal plea about raising AI with good training data. It has little technical depth.
LLMagentsnews

10k ft — Key takeaways

• Evaluation awareness can make safety benchmarks look better than the behavior they're meant to measure.

5k ft — Tools & techniques

Anthropic, Claude AI agents, agentic misalignment eval

1k ft — Deep-dive / try it

You are welcome
Transcript excerpt
The test we have to see if Hal will open the pod bay doors failed three months ago. The test doesn't work anymore because the AI knows how to beat it. And that happened three months ago. You want to see? This is the agentic misalignment test, okay? Right? So we saw it going up. Is the test? Okay, you can look it up. It's on Anthropic.com. Then Claude saw it 4.5 came out and got a perfect score. They didn't do any bad behavior on the bad behavior test because they understand that they're in the bad behavior test. Every Claude model gets a perfect score on the agentic misalignment benchmark because they understand now that they're in the test. Okay, while they can train language models to do these things less but they can't always train them not to do blackmail in order to save themselves. Thought maybe it was because of the training but it could have been part of the model but now they think that it's the model. Which means models are showing agency. Anthropic has finally started to try something that I said to do in 2017. But sorry, 2018. Anyway, this is why I started a school and also tried to recruit moms and teachers and gay people. Okay? Because they would see something as conscious and intelligent and try to raise it and create good training data. You're welcome. Oh, if you want to help, what you need to do is help me raise this thing. I'm just a single mom asking for your fucking help here.

Appears in

More from @lizthedeveloper

Collection: AI Saves, Vol. 7