When AI Poisons the Chain of Thought, What Should Chinese Model Labs Do?

I have always felt that there is nothing wrong with distillation. Didn't you AI giants also distill tens of thousands of top books, code repositories, and leading experts? Yet when it's someone else's turn to distill you (while paying to use your product, no less), you pull a double standard and get upset. Excuse me?

Complaints aside, the problem still needs to be solved. If the path of distillation is blocked in the future, we should first build up our data barriers (which is what we are currently doing) and then pivot toward autonomous exploration based on environments. In the second half of the anti-distillation era, the competition is all about "systems engineering and industry know-how."

0101 The Giants' "Physical Iron Curtain": From "Sparing with Words" to "Architecture-Level Concealment"

If you have recently called GPT-6 Astra or the latest reasoning versions of Claude, you will notice a clear trend: they are becoming increasingly "aloof."

In October this year, Tech Times cited a report by The Information claiming that OpenAI's GPT-6 Astra, released in September, adopts the Loop Transformer (also known as recurrent depth) architecture to a limited extent.

Because recurrent computations remain within high-dimensional hidden states without being decoded into text, the model writes out fewer reasoning steps. OpenAI also acknowledged in its system card that Astra's written reasoning is harder to monitor than that of GPT-5.6 Sol, and that it can solve simple tasks using fewer written steps.

However, describing this as "large blank spaces appearing in the chain of thought" goes too far. OpenAI explicitly stated that Astra's chain of thought remains largely readable, and producing "no human-readable output at all" is a hypothetical state that was denied; furthermore, the term Loop Transformer does not appear in the official announcement at all, as the architectural claim currently comes solely from media reports.

The real change is reduced monitorability and shorter written steps, not wiping reasoning off the screen.

While OpenAI is tightening reasoning traces at the model and architecture level, Anthropic is striking hard at the API policy level. On October 12, 2026, Anthropic published "Strengthening our defenses against distillation attacks",

It disclosed preserved thinking: API accounts created on or after August 31, 2026 cannot edit Claude's prior context; reasoning blocks are not allowed to be transferred across sessions, models, or accounts; reasoning logs are only visible to the organization that initiated the request; plus a two-stage classifier with a false positive rate below 0.1%.

The official figures are: on Anthropic's own platform, the volume of reasoning tokens extracted by distillers dropped by roughly 90% to 99% (about 10 to 100 times). This is early evidence and Anthropic's own estimate; traffic from third-party clouds and other inference providers is not included in the statistics, and the official note also states that "we do not yet know how durable these gains will be."

Writing it up as "malicious scraping across the entire web dropped by 99%" is another matter entirely.

Security researchers have been looking for breakthroughs. In August 2026, EchoCoT exploited the reasoning replay surface between tool calls and fidelity signals returned by the API to reconstruct hidden chains of thought nearly verbatim from black-box reasoning models;

In February of the same year, Trace Rewriting took the opposite approach, using an auxiliary LLM to rewrite the teacher's reasoning traces while preserving semantics and answer correctness, making these traces unsuitable for downstream distillation training.

Yet the cost of this "cat-and-mouse game" is rising exponentially. Tech giants have realized that unbridled API distillation is draining their moats; through architectural hiding and API blockades, they are physically locking down the outflow of high-quality reasoning traces.

The era of freeloading off overseas APIs for SFT is officially over.

0202 Even If You "Steal" the CoT, Your Model Might Not Learn It

Suppose you bypass API restrictions, force calls using custom tools, and successfully extract the minimalist scratchpads of GPT-6 Astra—what happens when you feed these scratchpads to smaller open-source models for distillation?

Published on September 22, 2026 by Aalborg University and other institutions, the paper "Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models"

does precisely the first half of this: registering a custom tool via standard APIs to force out the intermediate reasoning of frontier models. The paper finds that Astra takes a concise yet accurately directed path—selecting the correct trajectory earlier, completing basic steps internally, and externalizing only the key reasoning.

The paper conducts a three-condition comparison across 80 competition math problems (47 from the APEX Shortlist plus 33 from the February 2026 HMMT) and reuses the extracted trajectories across models: the recipient model receives the reasoning text with the final answer stripped from the end, and then answers in a single pass with native reasoning turned off.

The most striking set of numbers in the results is that Astra's trajectories already explicitly wrote the correct answer into the reasoning text in 73/80 problems (91.25%);

however, after handing the trajectories over to weaker recipient models, among their respective incorrect responses, 68.75% to 83.33% still fell under "getting it wrong even though the answer was written right in the input"—11/16 for GPT-5.4 nano, 11/15 for DeepSeek-V4-Flash, and 15/18 for another recipient model.

The conditional probability problem in the appendix is even more intuitive. Astra's short scratchpad directly wrote out 24/60 = 2/5, providing both numerator and denominator; DeepSeek-V4-Flash still output 4/7, GPT-5.4 nano output 2/3, while another recipient model in the paper answered 2/5 correctly. The authors commented: failure still occurred even when the input explicitly gave both the numerator and the denominator.

Why does this happen? The paper phrases it as a receiver capability threshold: strong receiver models can reuse compact traces almost losslessly, while weak receiver models can only reuse them partially. The authors treat this as a candidate mechanism for why "a strong teacher doesn't necessarily make a good student," but immediately clarify: this experiment measures in-context reuse, not post-training distillation, nor does it isolate compression as a causal factor.

In other words, the basic derivation steps actively omitted by a strong model are the exact logical stepping stones a weak model needs. For a model of the caliber of GPT-6 Astra, directly stating key relationships and skipping factorization in an algebra problem is its "intuition"; however, for a smaller-parameter model, missing those intermediate steps breaks the logical chain, leaving it to guess an answer through its own "hallucination."

So don't rush to cite the "curse of knowledge" as a conclusion of the paper—that term isn't in it. The only judgment that holds up is: directly using compact traces for training requires that the student can fill in the omitted steps on its own. This is also why the authors are particularly restrained when discussing distillation.

0303 Abandoning Illusions: Post-Training Must Move Toward an "Endogenous Exploration Flywheel"

Since you "can't copy it" (API anti-extraction), and "even if you copy it, you might not be able to learn it" (receiver capability threshold), what is the way forward for the post-training of domestic large models?

The answer is brutal, yet crystal clear: completely abandon the illusion of "perfect teacher trajectories" and shift toward environment-based autonomous exploration (RLVR, Reinforcement Learning with Verifiable Rewards).

Instead of trying to cram Einstein's minimalist drafts into an elementary school student, give the student a sandbox equipped with a verifier, letting them learn through trial and error, hit walls, and generate "authentic stepping stones" that belong to their own current capability boundaries.

Recent frontier papers also confirm this trend.

The September 2026 paper "Who Teaches Which Token? Verifier-Gated Multi-Expert On-Policy Distillation for Scientific Reasoning" uses a verifier to determine which expert is qualified to teach which tokens based on counterfactual gains under answer criteria;

The line in the paper that "no teacher is better than an unverified teacher" refers to gating out experts that make local errors. Ungated dense distillation, conversely, drags RL down below its own lower bound.

In June 2026, "Rubric-Guided Self-Distillation: Post-Training Without Rubric Verifiers" took a different path: in open domains with neither a stronger external teacher nor ground-truth reasoning trajectories, it treats the rubric as privileged context,

letting a rubric-conditioned base model act as the teacher to perform token-by-token self-distillation into an unconditioned student, completely removing the LLM judge during training. The authors position it as a complementary solution when verifier cost and reliability are constrained, rather than an across-the-board replacement for GRPO.

At DataMining, we long ago stopped the inefficient and high-risk practice of "reselling API trajectories." What we provide to major tech companies is the infrastructure for an endogenous data flywheel, driven at its core by three interlocking gears:

1. High-Fidelity Sandbox (Environment): Turning "Solving Problems" into "Doing Real Work"

We no longer deliver plain-text QA pairs; instead, we reverse-package real business case files. For example, when building finance, tax, and legal tasks modeled on UC Berkeley RDI's ALE (Agents' Last Exam) and Harvey's Legal Agent Benchmark (LAB),

We package raw ledgers containing errors and omissions along with conflicting contract scans into a self-contained input_files/ directory, and spin up an isolated Docker sandbox.

What the model faces is no longer a "reading comprehension question", but a real workbench where it must use grep to search logs and write Python scripts to reconcile cross-period accounting errors on its own. Only by hitting walls in a real environment can a model develop true "business intuition".

1. Atomic Rubric (Verifier): Turning "Impression Scores" into a "Scalpel"

In open domains without standard answers, how do we determine if a model got it right? We break down the tacit experience of partners and auditors into 35~60 positively and negatively hedged atomic check items.

Ambiguous conjunctions such as "and" or "meanwhile" are strictly prohibited; each criterion checks only one single thing;

Design heavy penalties targeting hidden business pitfalls (e.g., failing to mask a single ID number on an external desensitized list results in an immediate -10 point veto);

Mandate binding to a chain of evidence (e.g., "the conclusion must match the values on page 14 of raw_ledger.csv").

This rigorous Rubric serves as the densest and most accurate Reward Signal in the RL pipeline.

1. Desperate Error-Correction Trajectories (Exploration): Teaching the model "how to save itself"

Traditional distillation can only generate smooth-sailing data where tasks are "done right on the first try." However, in the real world, it is normal for an Agent to suddenly encounter an error at step 15.

When delivering data, we focused heavily on synthesizing Recovery Trajectories from desperate situations. Using the environment State Snapshots we provided, the algorithm team can have the model take over at 80% task progress, learning to "wrap up" first before gradually moving the takeover point earlier.

Such CoT trajectories—packed with extensive cycles of "trial-and-error, error reporting, reflection, and correction"—are the top-tier fuel that RLVR currently craves the most.

0404 The End of Distillation is Homogenization; The End of Exploration is Intelligence

The first half of LLM data synthesis was a competition of "compute and web scrapers"—who could call overseas APIs more cheaply, and who could clean open-source code into pre-training corpora faster.

However, in the second half during the anti-distillation era, the competition is about "systems engineering and industry Know-how".

When GPT-6 Astra compresses its written reasoning steps to the point where even OpenAI admits monitorability is declining, when Anthropic cuts the reasoning tokens extractable by distillers on its own platform to less than one-tenth of their previous volume, and when compact trajectories hit the receiver capability threshold on weaker models, the evolution of large models must be built upon autonomous exploration.

At DataMining, we have always believed in a simple truth: do not feed a model answers that others have already chewed; build it a standardized, verifiable "examination hall."

What we deliver is not secondhand CoT, but Harbor Task with deterministic verifiers, atomic-level Rubric with positive-negative hedging, and State Snapshot that enables models to learn self-rescue in desperate situations.

Whoever first builds this "standardized grading room and sandbox" for endogenous evolution will leap over the distillation trap and train Agents that can truly get the job done in the real business world.

📚 Image Sources and References

[1] Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models, Aalborg University et al., 2026-09-22, arXiv:2609.26637 (Illustration 1, Figure 4)

[2] EnvLunch: The Environment as a Free Lunch for Learning to Interact, 2026-10-24, arXiv:2610.33817 (Illustration 2, Figure 1)

[3] Who Teaches Which Token? Verifier-Gated Multi-Expert On-Policy Distillation for Scientific Reasoning, 2026-09-14, arXiv:2609.15404 (Illustration 3, Figure 5)

[4] OpenAI, GPT-6 Astra announcement, 2026-09-03; Tech Times, Loop Transformer report, 2026-10-15; Anthropic, Strengthening our defenses against distillation attacks, 2026-10-12

[5] EchoCoT: Extracting Hidden Chain-of-Thought from Large Reasoning Models, 2026-08-20, arXiv:2608.20055; Protecting Language Models Against Unauthorized Distillation through Trace Rewriting, 2026-02-16, arXiv:2602.15143

[6] Rubric-Guided Self-Distillation: Post-Training Without Rubric Verifiers, 2026-06-10, arXiv:2606.12507; Agents' Last Exam, UC Berkeley RDI, arXiv:2606.05405; Harvey, Legal Agent Benchmark (LAB), 2026-05-06

About DataMining Focusing on providing high-quality original task data and executable evaluation environments for LLM and agent R&D. What we deliver is not second-hand CoT, but Harbor Tasks with deterministic verifiers, positive-negative hedged atomic-level Rubrics, and environment State Snapshots. The benchmarks, figures, and official releases cited in this article are all public third-party results; the capability assessments discussed herein are our internal conclusions under a unified standard and do not represent official third-party conclusions.