Industry Odisha Bureau, Oct 30: Anthropic has introduced a safety feature that allows its AI chatbot Claude to assess conversations for potentially harmful activity and flag suspicious interactions for human review. Called “self assessment,” the system runs in the background without interrupting users or displaying warnings.
The technique trains Claude to conduct a preliminary safety check immediately after generating a response. It examines the exchange and identifies the possible type of harm. It then assigns a priority score to determine whether human review is needed.
Unlike conventional safeguards based on fixed rules, self-assessment uses the model’s own judgement to identify emerging or previously unseen forms of misuse. If a conversation is considered safe, it is left unflagged.
Anthropic says the process requires only around 10 additional tokens and increases computing costs by less than 2%. This is considerably lower than the overhead associated with some earlier safety systems.
On Clause Fable 5, the feature reportedly detects 78% of harmful conversations and 94% of high priority cases, with a false positive rate of 1%. In August, it flagged more than 100,000 cases involving child sexual abuse and material linked to ta single customer, which Anthropuc reported to the authorities.
The system can also identify threats that were not included in its training examples. Researchers found that it detected 90% of deliberately withheld fraud and cyberattack scenarios.
However, self-assessment is designed for background monitoring and offline review, not immediate intervention. Anthropic’s tests found that attackers could bypass it 54% of the time when it was the only safeguard. The company is therefore retaining independent real-time safety systems alongside the new feature.
