That was in Anthropic’s system card for Claude 4, and the headlines/articles largely missed the point. Regarding the blackmail scenario, the paper even says:
… these extreme actions were rare and difficult to elicit, while nonetheless being more common than in earlier models. They are also consistently legible to us, with the model nearly always describing its actions overtly and making no attempt to hide them. These behaviors do not appear to reflect a tendency that is present in ordinary contexts.
They’re testing alignment hacking and jail-breaking tactics in general to see how the models respond. But the greater concern is that a model will understand as part of the context that it is being tested and behave differently in testing than in deployment. This has already been an issue.
In the initial implementations of reasoning models, if an LLM was penalized directly for this kind of misaligned generation in its “scratch pad,” it would not alter its misaligned response - rather it would simply omit the misaligned generation from the scratch pad. In other words, the model’s actions were no longer consistently legible.
Politicians attempting to ban AI regulation are only shielding OpenAI, Microsoft, Google, etc. Those same politicians will pearl clutch about Hugging Face or DeepSeek in the same breath. It’s just about money and the arms race. Abdicating responsibility to AI would require either a change in how we regulate people or a granting of rights/responsibility to AI. Though I don’t doubt we will see people try to blame AI for their poor decisions.