LLMs use fixed weights and a long context window to prevent leaking information between users. But that doesn't mean your conversations are private. Anthropic is listening to what you say to Claude, and will ban you or report you to authorities if you ask the wrong questions.
https://www.anthropic.com/threat-intelligence-report-september-2026 It's long, so let me summarize. Most of this bad behavior is occurring in countries that Anthropic has blocked: China, Russia, Iran, Yemen, Sudan, Syria, and a few other "enemy" countries. They access Claude through VPNs and proxy accounts. Some examples: - Gain of function research on chikungunya, bird flu, and orthopox viruses including smallpox and mpox. Anthropic acknowledges that research into making diseases more deadly or contagious is dual use and necessary. The same knowledge needed to make biological weapons is also needed to develop treatments and vaccines to counter them. - Computer security. Again, this is dual use. You have to be able to find vulnerabilities to patch them. Anthropic cites several cases including one where Russians used Claude to vibe code attacks on hotel WiFi where Ukrainian officials procuring drone parts were staying. - Autonomous weapons. Russia used Claude to develop software for autonomous drone swarms, training onboard vision systems on Ukrainian combat footage to identify targets including humans. Yemen used Claude to code navigation software for tactical and ballistic missiles. China used Claude to write proposal documents for a torpedo countermeasures system and to develop electronic countermeasures to jam US and Taiwanese air defense systems including Patriot and THAAD missiles. - Distillation. All of the major Chinese AI, Z.ai, Kimi, DeepSeek, and Xiaomi have been detected passing user queries to Claude and training on the responses. In the first 3, Claude's responses were passed back to the users. - Scams. A fake dating site in China used AI generated chats and images to scam users. - Propaganda. Actors in Russia, Iran, UAE, Bangladesh, and Kenya used AI to generate propaganda and flood social media with thousands of fake accounts targeting opposing political parties, faking multiple indepent sources citing each other to appear legitimate. - Surveillance, using Claude to scour social media and surveillance images to collect dossiers on individual locations and political leanings in Iran and the Persian Gulf through a company based in Israel and Singapore. Enjoy it while you can. Soon, users will avoid getting caught by downloading open weight models and running them locally. Inference takes a lot less compute than training. We know this works, because that's how the Hutter prize leader does it. -- Matt Mahoney, [email protected] ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T3f8115622f860785-Mae72cc8324ef0c97bf357631 Delivery options: https://agi.topicbox.com/groups/agi/subscription
