Chinese Firms Accused of Stealing Billions of Tokens from US AI Models: What LLM Builders Need to Know

US Intelligence Agencies Warn of Large-Scale AI Model Theft

According to reporting from BleepingComputer, U.S. cybersecurity and intelligence agencies have identified six Chinese AI companies engaged in industrial-scale distillation attacks targeting American frontier AI models. The attacks have been ongoing since at least late 2024, raising serious concerns about intellectual property theft and the security of cutting-edge AI systems.

This revelation highlights a growing vulnerability in the AI landscape: frontier models—the most advanced language models available—are increasingly targeted for their knowledge and capabilities. The sheer scale of these attacks, involving billions of tokens extracted from proprietary systems, suggests a coordinated and well-resourced effort to reverse-engineer some of the world's most valuable AI technology.

Understanding Model Distillation Attacks

Model distillation, in this context, refers to techniques where attackers systematically query a frontier AI model to extract its knowledge and capabilities, then use that information to create competing models. While distillation itself is a legitimate machine learning technique, weaponizing it as an attack vector represents a serious threat.

When conducted at industrial scale, distillation attacks can:

• Compress months or years of expensive model training into weeks
• Replicate proprietary safety guardrails and alignment techniques
• Allow competitors to bypass the massive R&D investment required to build frontier models
• Potentially capture sensitive behavioral patterns embedded in the original model

The Risks to LLM Applications and Guardrails

For companies building applications on top of frontier AI models, this threat carries multiple implications. Stolen models can be weaponized without the safety guardrails and ethical constraints that reputable AI companies invest in heavily. This means adversaries may have access to more capable—but less controlled—versions of advanced AI systems.

Additionally, if attackers can extract the underlying knowledge and reasoning patterns of frontier models, they may also capture:

• Techniques used to prevent harmful outputs
• Methods for detecting adversarial prompts
• Security mechanisms designed to prevent misuse
• Behavioral patterns that reveal how the model makes decisions

This creates a downstream risk for all applications relying on these models. If guardrails can be studied and potentially circumvented, the security assumptions that builders rely on become weaker.

What Should AI Builders Do Now?

Monitor API Usage Patterns: Implement strict logging and anomaly detection for your API calls. Unusual query patterns—particularly large-scale, systematic requests—could indicate distillation attempts.

Implement Rate Limiting: Aggressive rate limiting and request throttling make large-scale distillation attacks more difficult and expensive to execute.

Use Differential Privacy: If you control the underlying model, consider techniques that make extracted knowledge less useful through noise injection and privacy-preserving methods.

Diversify Models: Rather than relying entirely on a single frontier model, evaluate a portfolio of providers and approaches. This reduces your exposure if one system's security is compromised.

Stay Informed on Guardrails: Keep your safety measures up to date. As new attack vectors emerge, the industry will develop better defenses—make sure your systems incorporate them.

Report Suspicious Activity: If you detect signs of distillation attacks or unusual access patterns, report them to both the model provider and relevant authorities.

The Bottom Line

The report from U.S. agencies underscores a critical reality: frontier AI models are high-value targets, and their security cannot be taken for granted. For LLM app builders, this means moving beyond assuming your underlying model is secure and taking active steps to protect your application layer. Rate limiting, monitoring, diversification, and staying current on security best practices are no longer optional—they're essential components of responsible AI deployment. The race for AI dominance is intensifying, and your security posture needs to keep pace.