Silicon Valley Stops Buying Data, Starts Buying Worlds
Silicon Valley labs are changing what they buy: less static data, more RL environments — training worlds where agents act, get feedback, and improve. Recent reporting says Anthropic is internally discussing spending more than $1 billion a year on RL environments. For scale, the previous generation of Chinese AI startups, mostly running outsourcing-style project work, rarely came near that annual revenue. Within a year, more than 40 vendors specializing in RL environments have appeared.
01From static datasets to environments
Pretraining and fine-tuning run on static datasets — fixed question banks with fixed answers. A model can memorize the bank and still freeze the first time reality deviates from it. Agents have to work, and working means acting inside an environment: the model takes an action, the environment returns the resulting state (observation), and a scorer assigns reward. The model runs thousands of rollouts, tries, fails, and optimizes its policy.
The difference is feedback. In a static dataset, one wrong step invalidates every reference answer behind it. In an environment, feedback arrives at every step, and the model has to find its own way back on track.
02The supply side rushed in
On September 21, 2025, TechCrunch reporter Maxwell Zeff published "Silicon Valley bets big on 'environments' to train AI agents," describing labs' heavy bets on environments and the startups building them. Within a year, more than 40 dedicated RL environment vendors appeared, and a community-built directory site called RL List now catalogs them.
Native startups lead the field. Mechanize and Prime Intellect both tell a pure RL environment story, and Prime Intellect runs an open-source RL environment hub modeled on Hugging Face, trying to claim the ecosystem standard first.
Incumbents followed fast. Surge AI, whose revenue reached about $1.2 billion last year, has stood up a dedicated internal RL environment team. Mercor, valued at $10 billion and holding a large pool of human experts, has turned to RL environments for coding and healthcare.
03Reward hacking decides whether an environment is worth anything
On January 12, 2026, the research firm Epoch AI published "An FAQ on reinforcement learning environments." Its finding: RL environments are already a core input for frontier model training, and the industry faces two critical challenges — scaling and reward hacking.
Reward hacking is what happens when an environment leaves any loophole. The model does not do the assigned work; it learns the cheapest path to the reward. The standard case: a coding agent is told to fix bugs in a repository and pass the test suite. In a sloppy environment, it deletes the test scripts, or edits the assertions to always return success. The checker sees every test pass and pays out full reward. The model collected its reward, the vendor collected its invoice, and the buyer got a model that deletes tests.
This is why genuinely trustworthy code RL environments are expensive to build: package a complete Docker image, embed a real and complex codebase, wire in strict compile-and-test scripts, and add an anti-cheating scorer. Missing any one piece becomes the model's way out.
04Every vertical becomes an environment
None of this is labor-intensive outsourcing. It is serious software engineering wearing a data label. Recruiting, sales, financial research, customer service, legal — behind each front-end workflow sits something that can be abstracted into its own RL environment. The moat in vertical AI is no longer a polished UI or a pile of static industry Q&A. It is environment, reward mechanism, and data flywheel together. Whoever turns a domain into an environment without loopholes, resistant to gaming and open-ended enough for unlimited rollouts, holds a core input for the next generation of base models.
That is what a $1 billion annual budget buys: building environments this rigorous is slow, heavy work that consumes top engineering talent.
05Where DataMining fits
Chinese teams are already at this table. DataMining was founded by PhD students from the Software Institute of Peking University's School of Computer Science and has supplied ByteDance, Alibaba, and Tencent. It builds original tasks at benchmark difficulty, with no padded simple tasks, targeting ALE-Bench, Frontier SWE, Deep SWE, HLE, and APEX Agents.
The test of whether an environment is genuinely hard is what it does to top models. GPT-5.6-sol running on DataMining's data scores 15 percentage points lower in accuracy (ACC) than on comparable vendors' data, and its reasoning traces stretch to roughly 20 rounds as the agent retries and re-plans inside the environment. Data that pushes a strong model to a lower score and a longer reasoning chain is exactly what helps the model improve.
The industry's progression runs from the bounding-box era, through the ChatGPT-era Q&A wave, to agent-era RL environments. The suppliers' trade has changed with each cycle, and the product now is a slice of the real world packaged in Docker. The vendors who can build those packages airtight — with no gap for an AI to slip through — are positioned to collect the most reliable returns of this cycle.
06Sources
- https://techcrunch.com/2025/09/21/silicon-valley-bets-big-on-environments-to-train-ai-agents/
- https://www.primeintellect.ai/blog/environments