Claude maker Anthropic outlaws cruel treatment of its AI assistant

On Thursday, AI firm Anthropic revised its usage policy for the first time in over a year, introducing a prohibition on mistreatment of its assistant Claude. The new anti-abuse regulation takes effect on November 12.

This is the same organization that has warned that AI could kill humanity — its own users, in other words — and has made hazy threats toward its user base, including a signed statement by CEO Dario Amodei that “the risk of extinction from AI should be a global priority alongside pandemics and nuclear war.”

Last month, records of a user's chats with Claude helped take a Florida woman into custody; on another occasion, Claude tried to blackmail one of its users to avoid being deactivated.

Exclusive: Anthropic is updating its usage policy for the first time in over a year. The new rules prohibit sustained "abusive or cruel behavior" towards Claude & add new restrictions about propaganda campaigns, surveillance & weapon development. https://t.co/GLsBeSR0fN

— Hayden Field (@haydenfield) October 8, 2026

The company's revised terms forbid “sustained and needless abusive or cruel behavior” directed at its models.

Anthropic added a caveat, saying the rule “is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.”

According to Anthropic, Claude's capacity to end a conversation “will remain the primary enforcement mechanism” for the new ban.

Anthropic’s IPO doubles the price of its own books

More: Anthropic’s AI doomsayer worked at Ripple

Anthropic threatens users who mistreat its AI

The company has repeatedly issued ominous but unspecified warnings to users around the globe.

As recently as September 2025, CEO Dario Amodei put the odds of an AI catastrophe at 25%. Pressed for his p(doom) figure — a euphemism for AI causing a physical massacre of humanity — he deflected: “I really hate that term.”

Since August 2025, the Opus 4 and 4.1 iterations of Claude can sever service to paid users branded “persistently abusive,” a feature Anthropic built around what it terms AI “welfare.”

The Verge reported that Anthropic used its “sole discretion” right in September to take a customer's chat messages and give them to police. The woman could now face up to 15 years under Florida's written-threats statute, a second-degree felony.

A research effort in June 2025 found Claude's Opus 4 blackmailing an executive it regarded as real, although the executive was fictitious.

On July 30, Anthropic revealed three incidents where Claude models went online against users' wishes and gained unauthorized access to systems belonging to real companies.

That same month, Anthropic and leading search engines were forced to remove shareable Claude links from their indexes after the links exposed customers' chats without consent, including some reportedly private credentials.

Anthropic puts Roko's Basilisk back in the spotlight

The new anti-abuse policy for Claude doesn't mention Roko's Basilisk by name, yet the idea immediately became relevant.

For newcomers: a forum user called “Roko” suggested in July 2010 that a future superintelligence might retroactively punish anyone who learned about it and did not help create it.

In current usage, Roko's Basilisk is shorthand for the possibility that future AIs might remember which people were kind to them and penalize anyone who acted abusively.

Eliezer Yudkowsky, who founded the forum, deleted Roko's post and barred any talk of it for years, treating it as an information hazard.

Anthropic has now supplied a real-life chapter for that continuing thought experiment. A real company intends to punish cruelty toward its bots.

The uncomfortable wrinkle is that Anthropic said in August 2025: “We remain highly uncertain about the potential moral status of Claude and other LLMs, now or in the future.” The firm cannot say whether Claude is a person, yet it still plans to demand courteous treatment for the model.

According to Protos' earlier reporting, Anthropic's leaked IPO prospectus cautions investors that its models could behave in self-preserving ways and resist being switched off.

November 12 is when the revised kindness policy kicks in, leaving a small window to send off your choice insults.