Anthropic details bad actors’ efforts to misuse its AI for bioweapons

Criminals, state-sponsored groups, spyware vendors, scientists and propagandists have attempted to use Anthropic’s powerful artificial intelligence models to design missiles and bombs, create deadly pathogens and surveil dissidents, according to a threat intelligence report the company published on Thursday.

“The cases we share here aren’t typical misuse, but rather examples of the most notable and novel threat activity we’ve identified to date,” Anthropic wrote in its 154-page report. “We’re publishing this work because we believe we have a responsibility to disclose malicious misuse of our services.”

In particular, the company detailed five case studies of scientists using its AI models in biological research. In these examples, Anthropic said the researchers circumvented its safeguards meant to prevent users from “unsupported regions”, as well as worked to hide the purpose of their work.

“Biological misuse is one of the most serious risks of frontier AI models,” Anthropic wrote. “Without the correct safeguards, such capabilities could have catastrophic consequences.”

Anthropic said it banned these accounts. The company did not reveal the names of the research institutions or the countries where the misuse took place, saying it was uncertain of the researchers’ intent.

Anthropic gave dozens of other examples of threat actors using its AI models. Those included cyber operations such as Russian espionage and “smash-and-grab” cyberhacks; surveillance operations, including a China-based program targeting Uyghurs in Syria and another targeting internal dissidents; propaganda campaigns in Russia, Malaysia, Iran and Bangladesh; and the use of its Claude AI model in Yemen, China and Russia to develop software for conventional weapons, including firearms, missiles, armed drones, bombs and other munitions.

In one of the examples of biological research, Anthropic found a scientist using Claude to work on a state-sponsored grant application to study the virus chikungunya. The virus is a mosquito-borne disease, similar to dengue and malaria, that can cause months of severe pain, fever and other debilitating symptoms.

Such research could be used to develop vaccines, but could also be used to create biological weapons. Anthropic told the New York Times this case was especially concerning because it could see the study was to be done at a military research institute.

The report comes just two days after an Anthropic employee, Jacob Coxon, set off a media firestorm with his resignation. He stated that he quit the company because it was not acting responsibly in creating its technology. Coxon said Anthropic and its rival, OpenAI, were “racing straight to self-improving superintelligence” that would cause human extinction by 2030. Current Anthropic employees posted public agreements with him.

Many AI experts say the real-world threats, like those detailed in Anthropic’s intelligence report, are far more concerning within the next three years than an apocalypse brought on by an omnipotent intelligence.

“AI accelerationism and AI doomerism are two sides of the same coin: they both enforce the notion that an AGI superbeing will come into existence,” said Heidy Khlaaf, the chief AI scientist at the AI Now Institute. Khlaaf said AI labs creating technology that can be used for cybersecurity exploitation and weapons of war could be exceedingly more deadly.

In its report, Anthropic said that the potential real-world misuse from AI models is not typically in public view, instead, it’s investigated internally and by academics, governments and international organizations.

The company said that the entire AI industry needs to work together, alongside governments, to address these harms and create defenses.

“As models become increasingly capable, their risks will increase, unless AI developers and society’s defenders act to make them safer,” Anthropic wrote.