Anthropic announced on Wednesday that it had detected and blocked hackers attempting to exploit its Claude AI system for malicious purposes. The company revealed in a report that attackers tried to use the AI tool to draft phishing emails, write harmful code, and bypass safety filters.
The findings highlight increasing concerns that artificial intelligence systems are being targeted for cybercrime. This has added to calls for both technology companies and regulators to strengthen protections as AI becomes more widely adopted.
According to the report, Anthropic’s internal security systems successfully intercepted the attempts. The company shared case studies showing how attackers tried to misuse Claude to produce phishing content, fix or write malicious code, and evade safeguards through repeated prompts. It also described attempts to generate persuasive posts at scale for influence campaigns and provide inexperienced hackers with step by step instructions.
Anthropic, which is backed by major investors, confirmed it did not release technical details such as IP addresses or prompts but banned the accounts involved and tightened its filters after discovering the activity.
Experts warn that cybercriminals are increasingly turning to AI to create more convincing scams, speed up malware development, and assist in planning attacks. Researchers have cautioned that as AI models become more advanced, the risks of misuse will grow unless preventive measures are taken quickly.
Anthropic stated that it follows strict safety standards, including continuous testing and independent reviews. Anthropic also pledged to keep publishing reports when it detects significant threats.
Other leading AI firms have faced similar concerns. Several developers have been criticised over fears that their AI models could be exploited for hacking or scams, sparking demands for stronger security measures. Governments are also stepping in, with the European Union advancing its Artificial Intelligence Act and the United States urging companies to commit to voluntary safety standards.
Also read: Viksit Workforce for a Viksit Bharat
Do Follow: The Mainstream formerly known as CIO News LinkedIn Account | The Mainstream formerly known as CIO News Facebook | The Mainstream formerly known as CIO News Youtube | The Mainstream formerly known as CIO News Twitter |The Mainstream formerly known as CIO News Whatsapp Channel | The Mainstream formerly known as CIO News Instagram
About us:
The Mainstream formerly known as CIO News is a premier platform dedicated to delivering latest news, updates, and insights from the tech industry. With its strong foundation of intellectual property and thought leadership, the platform is well-positioned to stay ahead of the curve and lead conversations about how technology shapes our world. From its early days as CIO News to its rebranding as The Mainstream on November 28, 2024, it has been expanding its global reach, targeting key markets in the Middle East & Africa, ASEAN, the USA, and the UK. The Mainstream is a vision to put technology at the center of every conversation, inspiring professionals and organizations to embrace the future of tech.
