Anthropic's distillation battle turns to the dark web as China concerns swell

Anthropic's head of threat intelligence, Jacob Klein, says his company welcomes competition. But what's coming out of the Chinese market, he says, is something much closer to theft.

Foreign adversaries, Klein says, are accessing Anthropic's Claude models — a process known as distillation — to train competing technology and sell copycat versions at a lower price. While distillation can be done legally, Klein says that's not what's happening here.

"There's an entire illicit ecosystem to try to gain access to Claude and other models," Klein told CNBC. "This ecosystem goes through any means necessary to evade our controls, so they can spin up accounts at extreme scale."

Distillation has become a controversial topic across the artificial intelligence landscape. Depending on how it's conducted, the practice can allow a model developer to use the output from another company's technology to create a competitive offering at a tiny fraction of the cost. In the U.S., some factions in the tech sector have urged policymakers to steer clear of regulations so that the best and most cost-effective AI can win, while others are lobbying for a crackdown on what they see as theft of intellectual property.

In an April memo, the Trump administration wrote distillation that undermines American research and proprietary information is "unacceptable," and said it would explore "a range of measures to hold foreign actors accountable."

The threat is intensifying at a pivotal moment for Anthropic. The 5-year-old company has soared to a private market valuation of close to $1 trillion and is expected to go public as soon as October, CNBC has reported.

Anthropic is singling out Chinese AI lab Moonshot AI as one of the companies it says is ripping off its technology. Moonshot's Kimi K3 model took the tech world by storm in July with its cheaper, frontier-level AI offering. It's been widely adopted in Silicon Valley, thanks in part to its lower price point and ability for companies to tailor it more easily.

Klein said Kimi K3 was illegally trained off the newest version of Claude.

"We've seen a fair amount of this from China," Klein said. "This is something that the industry writ large is dealing with."

Earlier this year, Anthropic alleged Moonshot and two other Chinese AI labs – DeepSeek and MiniMax – distilled its frontier AI models. Anthropic has also accused Alibaba, which makes the Qwen family of models, of conducting a massive "distillation attack" to illegally capture capabilities from Claude. OpenAI and Google have both published reports on distillation and claim they're fighting the same issue.

Alibaba, DeepSeek, Moonshot and MiniMax didn't respond to requests for comment.

## 'Fraudulent means'

Cybersecurity experts told CNBC that, in addition to China, the threat is also coming from countries like Iran, Russia and North Korea, where use of Claude, Google's Gemini and OpenAI's ChatGPT are restricted by the companies due to sanctions.

Klein said many labs in those regions "go through illicit means and fraudulent means to try to gain access to a model."

One way people are getting around those restrictions is by turning to the dark web, where they can find marketplaces of stolen credit card information and compromised AI accounts. Klein said companies like Moonshot are "spinning up tens of thousands, if not hundreds of thousands of fraudulent accounts."

Once they've accessed Anthropic's systems, they're able to ask the models questions and collect responses, which they can use to train their own model, often called the student, Klein said.

A clear sign that distillation is taking place is that a user could be asking thousands of questions, rather than dozens and potentially even creating thousands of accounts to do the same, producing a whack-a-mole scenario for the AI labs, Klein said.

"It's very hard to fully stop this as a problem, but I think slowing it down is good and worthwhile," Klein said, adding that foreign companies are able to use the technology with few guardrails.

He pointed to fears like surveillance and possible use in a biological weapons program, and noted what he described as a specific campaign from a China-based entity that was conducting espionage at scale using Anthropic's technology.

"There is a national security concern at play if malicious actors, bad actors who we don't trust are gaining access to a more capable models than they could have otherwise through the act of distillation."

Travis Lanham, technology chief at cybersecurity firm Armadin and a former Google engineer, said bad actors often go undetected because AI companies are under pressure to make their platforms as accessible as possible as they race against the competition.

"These companies are serving billions of requests," Lanham said, about the big AI labs. "The millions are relatively small compared to everything and it's just sneaking in and trying to look like the rest of the crowd."

Klein acknowledges that, for Anthropic, widespread competition is to be expected and that there are legal methods of distillation. That generally means gaining permissions and following the law on matters like IP and export controls.

"I think competition is great," Klein said. "The concern here is if you are taking our model, distilling it through fraudulent means, creating millions of fake accounts using stolen credit cards and stolen infrastructure, to then produce a model that doesn't have safeguards in place."