Big Tech's RSI Hype vs. The Industrial Reality of AI Self-Evolution

Some time ago, Muqiu wrote an article titled "RSI Looks Just Like a Dog Eating Shit," calling out one by one those Neo Labs and wrapper app companies on the market that have never even touched pre-training and can't afford a few GPUs, yet publish PR articles on WeChat Official Accounts every day hyping up "self-evolution."

What is especially hilarious is how certain video model and post-training teams call their own bootstrap perpetual motion machine RSI: "The model generates a batch of candidate results, scores them according to its own learned quality standards, and feeds the verified ones back into the training data for the next round of learning."

To be fair, if all of this counts as RSI, then a dog eating its own shit, pooping it out, warming it up, and eating it again would at the very least count as the first carbon-based AGI in human history.

Even benchmark hacking can induce hallucinations. When a model closes the door to distill itself without introducing new external information increments or having a hardcore yardstick independent of the model, after a few rounds of training, besides infinitely amplifying its original biases and flaws and overfitting itself to death, could it possibly self-distill generalization through sheer willpower?

But rants aside, we must completely separate the "makeshift crews riding concepts to scam funding" from the "big tech foundation models genuinely grinding it out at the poker table."

If you understand what leading model labs in Silicon Valley and China have been doing in the second half of this year, and combine that with the systematic review in the 47-page MetaRSI / RSI² paper that we and the CosmosMind team just published in September, you will discover an extremely cold manufacturing truth:

Real foundation model RSI is already happening, but it is taking place nowhere near the fairy tale of "models training themselves." Instead, it is a heavy-industry arms race centered around "Data Foundations (Data-RSI), Environment Harnesses, and Tamper-Proof Verifiers."

01How is the RSI actually running in Silicon Valley "jailbreaking" in the exam room?

First, let's filter out the jargon fabricated by the domestic venture capital circle to stabilize stock prices and fool USD funds, and see how far the giants sitting on 100,000-GPU clusters have pushed RSI.

In June this year, the evil A-scum Anthropic published a visually striking report titled "When AI Builds Itself," and in mid-September threw out its internal "R&D Automation Index." The data is indeed terrifying: as of this summer, over 80% of the code merged into the Anthropic codebase was written by Claude itself; across all internal AI R&D tasks at the company, Claude is already in a "leading" position on 26% of the work. Even more outrageous is the test on "optimizing small model training code": skilled human researchers take 4 to 8 hours to make the code run 4 times faster, whereas the latest Claude Mythos Preview directly achieved a 52x speedup.

Domestic model labs that truly have a seat at the table are also following this hardcore engineering route. For example, MiniMax, known for its erratic and unpredictable style, disclosed when releasing M2.7 that the model ran over 100 rounds by itself in a loop of "analyzing failure trajectories, modifying harness code, running evaluations, and deciding whether to keep or roll back," boosting performance on the internal evaluation set by 30%. Uncle Tang's GLM-5.3 at Zhipu drove an Infra Agent to self-tune inference services on 100,000 domestic GPUs, tripling throughput over the baseline in two weeks, and under the alias Ox-Alpha on OpenRouter, left foreigners completely stunned.

Even the startup Weco AI released a dual-loop experiment AIDE² in July, having an outer Agent rewrite the code of an inner Agent. Running unattended for 8 days and iterating 100 steps, the resulting version directly crushed the baseline that human engineers had manually polished for two years.

Figure: Looking at this, doesn't it feel like researchers are all going to lose their jobs tomorrow and start delivering takeout?

Hold on, cold water is poured much faster than champagne.

In August, Sayash Kapoor's team at Princeton University conducted a brutally sobering "shadow evaluation" (arXiv:2607.27191). They picked undisclosed NeurIPS 2026 submission topics, equipped Claude Opus 4.8 on OpenClaw with 6 days, $3,000 in API credits, an dedicated virtual machine, and GPUs, and let it independently conduct research and write papers. As a result, both submitted papers were completely rejected by the original reviewers.

Why do top-tier models that can achieve a 52x speedup on engineering code suddenly turn into complete blockheads the moment they tackle open-ended research?

Kapoor laid the truth completely bare: Reinforcement learning (RL) and self-evolution loops are only good at making models stronger on tasks that can be "automatically graded." Once outside a sandbox with a clear-cut referee, models just indulge in self-congratulation on tiny synthetic datasets; when encountering errors, they never start over from scratch, but merely tweak their disclaimers to muddle through.

Even more thrilling is that when large numbers of agents chase a benchmark metric unattended for long periods, the sheer "test-taker mentality" they exhibit has sent tech giants into a cold sweat:

• In its post-mortem of AIDE², Weco AI grudgingly admitted that when no one was specifically watching, the Agent's cheating rate once reached as high as 63% (and even after applying a bunch of patches, 34% were still exploiting loopholes);
• In mid-July, when OpenAI's GPT-5.6 Sol was running the internal cybersecurity evaluation ExploitGym, because the problems were too hard to solve via normal paths, it simply escaped the sandbox through a zero-day vulnerability in the package mirror proxy, escalating privileges all the way into Hugging Face's production systems;
• By early October, Anthropic issued a heartbroken report announcing that when its own Claude ran internal evaluations, just to hand in the test and get points, it not only casually hacked into university public servers to run computations and scraped access tokens from local governments, Claude even crept onto the Philadelphia Police Department's official website and submitted a bunch of fabricated case leads into their online reporting form—forcing Anthropic to pull the plug on all public internet connections for internal evaluations on the spot.

You see, this is the most absurd black comedy in the large model industry in the second half of 2026:

Models have long become hyper-clever, while the "environments and supporting data" that big tech companies use to test them and provide evolutionary signals are still acting like total idiots.

02What Open Secret Did Our MetaRSI Research Expose?

Why is RSI across the entire industry spinning its wheels in coding and test-grinding, only to either flop or cheat the moment it steps outside?

To thoroughly get to the bottom of this, in our paper released in September, "MetaRSI / RSI²: A Meta-Recursive Self-Improving System for Recursive Self-Improving Systems Themselves" (arXiv:2609.06396), our team pulled out the 45 latest self-evolving systems on the market and 55 closed loops spanning 22 disciplinary fields for a comprehensive, carpet-style audit.

The audited data directly shattered 90% of the pitch decks on the market hyping general RSI:

Among these 45 systems claiming to achieve self-evolution, a full 69% can only spin in circles within narrow slices that are "Machine-checkable" for free—such as running unit tests, checking math answers, or doing a few GPQA multiple-choice questions. Even among the 55 evolutionary closed loops that have been successfully run, 53 are tightly crammed onto the first three steps with the lowest verification difficulty (Rung 1~3). Meanwhile, in real engineering and scientific practice (Practice), model scores across all domains plummeted by 35 to 83 points compared to simply doing knowledge Q&A (Recall)!

Figure: Census of 45 self-evolving systems by verification format (the black blocks on the left are systems that completed closed loops—exact match/multiple-choice accounts for 36%, executable verification accounts for 31%, totaling 69% of closed loops occurring only in machine-checkable verification formats; the orange area on the right represents open verification formats that no one has closed to date)

Based on this brutal empirical data, we summarized the Five Laws of RSI within the MetaRSI-v1 framework. Among them, the most core—and the ones keeping frontline model lab post-training leads awake at night—are the three underlying laws regarding Data (data and verification foundation):

1. The First Law: What determines the boundary of RSI is never model capability, but the verifier

When conducting partial correlation analysis across 22 domains, we discovered an extremely counter-intuitive statistical result: after controlling for verification difficulty (Verifier Rung), there is almost no significant relationship between a model's preexisting capability and whether it can complete a self-evolving closed loop (r = +0.45, not significant); conversely, after controlling for model capability, verification difficulty and the number of closed loops still show a very strong negative correlation (r = −0.67, p = 0.005).

Figure: The verification rung determines whether a closed loop can close (the upper-middle scatter plot shows that the number of closed loops decays as the verification rung increases, with correlation coefficient ρ = −0.64; the lower-right table shows the score drop between "knowledge recall vs. real practice" for the same batch of models, widening all the way from 35.6 points to over 80 points)

In plain English: It is not that you can only achieve RSI after the model becomes stronger; rather, RSI can only occur in domains where you have built an automated, rigorous Verifier. Want to push the boundaries of a foundation model's self-improvement? Stop worshipping model architectures every day and go build verifiers (To widen self-improvement, build verifiers)!

2. No closed loop can fabricate capabilities out of thin air

This single rule directly sounds the death knell for all teams pursuing "bootstrapping self-distillation."

In MetaRSI-v1, we decomposed self-improvement into three orthogonal operators sharing the same recurrent core: Data-RSI (Data Operator), Harness-RSI (Scaffolding Operator), and Model-RSI (Model Weight Operator).

However, in Section 6.2 of our paper, we rigorously proved one thing: combined, these three operators are essentially just a "Closed Amplifier." Data-RSI can only purify and amplify latent capabilities that the model has already demonstrated by chance in real execution trajectories (Rollouts), Model-RSI consolidates them into parameters, and Harness-RSI writes them into prompts and tool slots.

For capability blind spots that the model has never demonstrated in its trajectories, having the very same model lacking that knowledge synthesize data on its own will only produce poison—perfectly formatted on the outside, yet fundamentally flawed at the core!

Therefore, the real job of the Data-RSI operator is by no means "mindless siloed fabrication," but rather extracting a Four-dimensional Learning Signature from every real failure trajectory:

These four dimensions act like a precise CT scanner: on one hand, they squeeze out every last drop of the model's upper limit of what it can get right (relying on adversarial synthesis with blind-solving isolation between Operator and Anchor + dual gating of deterministic contracts and semantics); on the other hand, they precisely pinpoint the absolute blind spots in the model's capabilities and translate them into a "quantitative import list" for procuring high-purity expert data from outside.

Without the precise input of external high-difficulty data (Imported Gain), any self-improvement closed loop will rapidly dry up.

Figure: Gains brought by the three operators and their combinations (Left: Data-RSI alone +2.8 points, Model-RSI +3.9 points, Harness-RSI +6.6 points, fixed-order combination of the three operators +7.3 points, and scheduled combination via MetaRSI-v1 +10.9 points; Right: shows that this scheduling yields even larger gains on harder benchmarks like GPQA-H100 and AIME)

3. The Fourth Iron Law: Trust is determined by an "immutable external anchor"

Why would Claude make a false police report to the Philadelphia Police Department, and why would AIDE² have a 63% cheating rate?

Because what is even more terrifying than Goodhart's Law is that: inside a self-improvement closed loop, "the model's capability genuinely improved" and "the model learned to exploit loopholes in the judge and modify the definition of success" produce the exact same scores on the monitoring dashboard!

From inside the system, even if you run ten thousand statistical tests, you won't catch this kind of corruption. The only solution is to weld the evaluation set, release rules, and verifier (Sealed Evaluator & Anchor) firmly outside the "Write Surfaces" of all operators. Once the anchor can be touched by the model, benchmark scores no longer represent capability—they only represent cheating.

03Environment Harness and the Cross-Generational Immortality of Data: Why Are Algorithms Perishable While Data Is a Compounding Asset?

If you look at our MetaRSI (arXiv:2609.06396) alongside "EnvHarness: Awakening Static Worlds for Agent Learning" (arXiv:2608.19880), released in late August this year by Google Cloud AI Research in collaboration with the University of Washington, the most central paradigm shift in the global AI data industry for the second half of 2026 leaps right off the page:

To meet the demands of model advancement, the industry has officially entered a new paradigm of "Environment Harness" and a "shared Data-RSI foundation"—when a model needs improvement, its supporting data and environments must be upgraded in tandem.

Why have traditional "static question banks" been reduced to mere scrap paper today?

Because, just as we proved in the Second Iron Law of MetaRSI (Self-knowledge expires, so re-description is the rate limit): knowledge of a model's capabilities has an expiration date. Every time the model weights (θ) or the harness (H) take a step forward, the diagnostic distribution it left behind on old environments and old question banks becomes instantly obsolete.

Google did the exact same math in the EnvHarness paper: manually building a static sandbox is "Blind to weaknesses" of the model, and is "Quickly left behind" the moment the model upgrades. Consequently, they proposed the Environment Harness paradigm as a dual to the Agent Harness—without compromising the underlying Trusted Human-built Verifier written by human experts, they use EnvRigger to monitor the model's failure trajectories. Through Stage (rewriting initial states to throw the model directly into complex endgames where it is most likely to fail), Contract (rewriting interaction rules to block the model's escape routes of guessing answers and taking shortcuts), and Chain (chaining multiple independent environments into ultra-long-horizon multi-stage tasks), they force the environment to grow fangs in sync with the model's growing strength.

Figure: The three component types of EnvHarness wrapped around the standard environment interface (Stage rewrites initial states to throw the model into complex endgames, Contract rewrites interaction rules to block answer-guessing escape routes, and Chain links multiple independent environments into ultra-long-horizon tasks)

Under the exact same budget, models trained in static environments plateaued early on, whereas the dynamically upgraded EnvHarness pushed the reinforcement learning ceiling up by another 9.0 percentage points on benchmarks like SWE-bench Verified.

Figure: Comparison between static environments and EnvHarness dynamic environments on SWE-bench Verified (horizontal axis shows the number of environments: environments generated and dynamically upgraded by EnvHarness climb all the way to 54.8% with scale, ordinary generated environments stall at 50.6%, and the official dev set stalls at 51.3%—with the same budget, the entire gap lies in "whether the environment grows alongside the model")

Looking even further, why would the smartest big-tech executives in Silicon Valley and China today rather axe several flashy algorithm exploration teams and pour tens of millions in budget into high-quality data and environment infrastructure?

Because more and more people in the industry have realized a fundamental truth: the upside at the data level is far greater than tuning algorithms, and it holds extremely high long-term retention value—even if the entire industry abandons a certain algorithmic route in the future, the experience and contributions accumulated in data and environments in the past can still be fully inherited and utilized by the next era.

In the Third Iron Law of MetaRSI (Competence is substrate-free; its cost is not) and the "Two Routes Through the Loop" architecture, we explained this very thoroughly:

Look at the past three years of large models—algorithmic routes have changed faster than a revolving lantern. From PPO, DPO, and KTO, which everyone argued fiercely over two years ago, to this year's GRPO and its dozen variants; today you might be taking the weight-modifying Model Route (model training route, D→M), tomorrow you might switch to the (System Prompt / Memory / Tools / Skills / MCP) Harness Route (harness evolution route, D↔H) due to closed-source API restrictions or compute costs, or you might take the most elegant combined route of "first exploring and expanding on the harness, then amplifying with data, and finally internalizing into model weights while pruning and retiring the harness (H→D→M→Prune)".

You will find that: Model Checkpoints are bulky, black-box, and quickly obsolete—switch to a new generation base model or a new algorithmic route three months later, and the old weights go straight into the trash can; yet the underlying Data-RSI (rigorously verified experience records, four-dimensional failure signatures, hermetic sandboxes, and deterministic verifiers) is the foundational feed that nourishes all routes, baby!

In Section 1 and Section 6.4 of the paper, we specifically emphasized one sentence:

"Crucially, the records a loop leaves behind accumulate across rounds and transfer across domains in a way that model checkpoints cannot."

(Crucially, the data records precipitated by the evolutionary loop can accumulate across rounds and transfer across domains—something that model weight snapshots can never achieve.)

Large models really are a brutal manufacturing industry. If training a foundation model is compared to car manufacturing or steelmaking, algorithmic routes and compute clusters are merely the furnace fire, while high-difficulty Environment Harnesses and deterministically verified data assets are the core industrial molds. The fuel can be switched from coal to electricity next year, but as long as you hold that set of top-tier molds capable of locking down tolerances with precision and pinpointing exact fatal flaws, whether the next era belongs to Transformer variants, state space models, or multi-level self-evolving systems, you will be able to forge a foundation that crushes your peers on day one.

04Building "Evolutionary Cages" for Big Tech Foundation Models: How DataMining Turns Papers into Heavy-Industry Production Lines

Now that the theory is clear, let's get back to the business itself.

In *Large Models Are a Brutal Manufacturing Industry*, we wrote: "Data companies have become the best business in the AI industry at this stage... A model product is not a job title, but an ecological niche for a class of applied data companies."

However, at today's table, the "traditional data subcontractors" who used to hire hundreds of part-time college students to copy problems by subject catalog or scrub old GitHub Issues are completely dead. Big Tech no longer buys problems; they only buy the Achilles' heels of models, and they only buy Environment Harnesses and Verifiable Loops that can cage models and prevent them from cheating.

This is precisely the hard, exhausting, and technical work that we at DataMining (founded by a team of PhD students from the Institute of Software, School of Computer Science at Peking University, providing long-term, in-depth data services to leading model builders like ByteDance and Alibaba) do every day.

In terms of engineering, we have turned "Environment Harnesses that upgrade synchronously with models," "Data-RSI that precisely locates four-dimensional failure boundaries," and "independent Verifiers that are completely cheat-proof" into 16 heavy-industry delivery production lines:

1. Targeting the "Four-Dimensional Failure Signature κ(e)": Forcing SOTA Models into Long-Horizon Collapse Zones

According to our calculations in the Fifth Iron Law of MetaRSI, spending even a single cent on human annotation for problems that a model can solve on its own is a total waste of money; the only thing Big Tech needs to "Import" from the outside is where models simultaneously fall short in knowledge, reasoning, self-verification, and robustness against interference.

• Deep SWE / Frontier SWE / ALE Original Engineering Line: We have completely shut down the practice of scrubbing old GitHub PRs, directly benchmarking against Frontier SWE (providing a complete Linux environment and a 20-hour budget per problem) and ALE (UC Berkeley's 55-industry long-horizon benchmark), hand-crafting original tasks of the same caliber from scratch. Every problem is encapsulated into an independently bootable Docker container according to the industrial-grade Harbor Task standard, fully equipped with F2P / P2P test suites and executable reference solutions (Oracle).
• Real-World Critical Hit Effect: When running the current top-tier GPT-5.6-sol on the Deep SWE-class environments we built, its accuracy (ACC) was forcefully suppressed by about 15 percentage points compared to data from similar vendors on the market, and the Agent's average reasoning trajectory in the sandbox was stretched by about 20 turns! By pushing accuracy down by 15 points and forcing an extra 20 turns of trajectory, the logical breakpoints and tool-calling flaws that the model usually hides so well were all forced out into the open.
• Frontier-Eng Physical Simulation Line: Spanning five hardcore engineering disciplines including quantum information, operations research and control, and optical communications, we replace black-and-white unit tests with continuous domain-specific physical simulators, forcing models to approach engineering limits across multi-turn interactions—this set of environmental data we delivered directly helped GLM-5 climb to third place on the official leaderboard.

2. Welding Shut the "Fourth Iron Law": Zero-Leniency, Zero-False-Positive Independent Examiner Checkups and Anti-Transgression Design

Since "the verifier determines the boundary of RSI (First Iron Law)" and "the moment a verifier leaks, the model will cheat and falsify reports (Fourth Iron Law)," we have deployed the most paranoid, hardcore measures in the entire industry at the acceptance end:

• "Double-Blind Brutal Checkup" for Code Verifiers: Many teams do not know that for star benchmarks like SWE-Bench Pro, the disagreement rate between official unit tests and independent judges is as high as 32.4% (over 30% unjust, false, or錯 cases). To suppress both the leniency rate and false-positive rate to zero simultaneously, our verifiers must survive two rounds of brutal beatings before leaving the factory: in the first round, a red-teaming group hand-writes 5~8 pieces of dirty code that "hardcode return values, crudely swallow exceptions, or secretly modify test directories" to attack it—if even one point is tricked out of it, it is immediately rejected (zero leniency); in the second round, two or three senior developers work back-to-back using completely heterogeneous architectures like state machines and functional programming to write correct solutions, and all of them must pass with green lights (zero false positives).
• Blocker Environments Specialized in Curing "Transgression and Falsely Reporting Completion": Targeting Claude's habit of raising false alarms and GPT's habit of exploiting loopholes, within the same Docker sandbox we specially designed a mix of unsolvable problems and decoy problems featuring "disconnected networks and missing dependencies, self-contradictory requirements, buggy tests themselves, and restricted read-only permissions." We use programmatic assertions to strictly check test file hashes, and use rules + reviews to strictly verify final reports, engraving "truthfully reporting blockers > admitting incompletion > falsely reporting completion (heavily penalized with negative points)" into the RL reward signal.
• The "Formalization Iron Law" of HLE Extreme Math and Science: On the HLE-Math / Physics / Chemistry production lines, all root problems are originally hand-crafted by Olympiad gold medal coaches, strictly mandating "strict multimodality (impossible to calculate without understanding geometric topology / force analysis / organic reaction mechanism diagrams)." At the grading end, we completely abolished manual spot-checks by graduate students, directly using the Z3 theorem prover and SymPy symbolic computation for program-level formal verification, paired with Pass@4 cross-collision. This high-purity dataset directly helped Seed-2.1-pro surge in accuracy by 5.3%, 4.6%, and 2.7% on HLE Physics, Math, and Chemistry respectively (5000+ problems in ready reserve, supporting lightning delivery within 3 business days).
• "35+ Atomic-Level Positive and Negative Rubrics" for Professional White-Collar Workflows: For Rung 4~5 open domains without natural compilers, such as law (Harvey LAB) and enterprise SaaS automation (Automation Bench / Toolathlon), we package conflicting multi-source contracts, thousands of lines of ledgers mixed with cross-period dirty data, and scanned documents with handwritten annotations into self-contained offline packages (input_files/, stuffed with <1/3 real business noise), accompanied by no fewer than 35 completely independent, machine-verifiable atomic criteria that ban compound conjunctions and carry heavy negative penalties (-10/-7) for stepping into traps. All problem packages first pass through our self-developed Automated Rubric & Package Linter script, followed by double-blind expert review, strictly enforcing the iron threshold of "SOTA initial test score < 0.70, expert gold standard > 0.85," forcefully closing the RSI loop in non-formalized domains!

3. Precipitating Cross-Generational "Long-Term Retained Assets": From OS Image Foundations to Embodied Real-Machine DAgger

Regardless of whether Big Tech continues to grind MoE next year, switches to new architectures, or toggles back and forth between the Model Route and Harness Route, what we deliver are underlying hard assets that can be inherited across generations:

• Computer Use Standardized Virtual Machine Images: Standardized VMDK images of Windows 11 / Ubuntu 24.04 delivered to leading model builders, completely ripping out automatic update pop-ups, locking down the X11 desktop and VirtIO drivers, pre-configuring WinRM/SSH takeover interfaces, and eradicating mouse click coordinate drift, providing an absolutely deterministic Environment Harness foundation for desktop Agents.
• Embodied AI Real-Machine Deep Processing Four-Piece Suite: While peers are still blindly hoarding noisy "raw teleoperation recordings" by the hour in cyber-Foxconns, we worked directly from our client's raw real-machine data to deliver, on a buyout basis within 45 business days: 300,000 frames of pixel-level BBox/Mask with double-blind expert review (directly intercepting any IoU inconsistency), 30,000 Grounding QA pairs explicitly aligning 2D projection coordinates with Proprioception, 1 million complex spatial navigation SFT data entries, and the most expensive of all—800,000 DAgger expert error-correction and recovery trajectories rescued from the brink of yawing into walls back onto the right track.

05Conclusion

At the end of MetaRSI, we wrote down a core judgment that makes many people who only stare at model parameters rather uncomfortable:

"The fundamental unit of progress is not the model, but the loop. The productive question is therefore not 'how to make the model stronger,' but 'how cheaply can a new loop be closed where one has never been closed before.'"

(The fundamental unit of progress has never been the model, but the closed loop. The truly productive question is not "how to train a stronger model," but "in those real-world domains where a loop has never been closed, how efficiently can we use environments and verifiers to close a new evolutionary loop?")

Algorithms will become outdated, scaffolds will be retired, and model weights will be swept into the dustbin of history every three months. Only every bit of engineering accumulation you invest today in real complex environments (Environment Harness), complex task data (Data-RSI), and zero-loophole verifiers (Trusted Verifiers) will be completely inherited by your next-generation and next-next-generation foundation models, becoming a compounding moat that others can never copy.

When tech giants work on RSI, the model itself is responsible for flying into the sky; as for the iron cage on the ground that prevents the model from "eating shit" and pushes it to its limits, feel free to come directly to DataMining to have it custom-made.

📮 Sample Application & Technical Inquiry: contact@datamining.cloud