The Grey Zone of LLM Data Brokers

By the end of 2026 the model arms race has reached a point where everyone's algorithms are good enough and compute is a matter of budget; the only variable left is data.

Especially coding leaderboards and the various expert evaluations: those scores are practically a launch-day talisman for a new model. Chasing a few points, labs pull out every stop, and in the process they support a whole cottage industry of data vendors building post-training data and RL environments for them.

Figure: Two public discussions about a 20-point ELO swing on Chatbot Arena; the leaderboard on the right shows GPT-4o in first place with 1361 points

But the rule of this trade is that public leaderboards are only the sign in the window. What labs actually pay for are the internal test sets they keep hidden. Follow that supply chain inward - writing the tasks, selling the data, accepting delivery, handing over the people - and the grey zone is much wider than most would care to admit.

0101

Overseas the market is highly concentrated: Scale, Surge, Mercor and Handshake take almost all of the revenue between them.

The delicate case is Scale. Its SEAL lab sells training data and RL environments to the big labs on one side, while running both public and private leaderboards on the other.

Selling the data and grading the models in the same shop is an awkward combination by construction.

Make the eval too strict and a customer who bought your data cannot score well, which makes your data look useless. Make it just strict enough to flatter your own data and everyone goes home happy. Outsiders have no way to independently check whether the benchmark quietly leans toward the vendor's own data style.

Private evaluation pipeline diagram: on the left, one company supplies both enterprise data services and private evaluation sets; in the middle, shared annotators and customer logos; on the right, a leaderboard with Financial Incentives and Irreducible Annotator Bias callouts

Research has already called this out directly: when one company sells both fine-tuning data and evaluation services, it has an incentive to shape the eval set around its client model's strengths. This is not a conspiracy theory, it is what the incentive structure produces.

Figure: Scale's Customers page, listing OpenAI, Cohere, Adept, Microsoft, Meta and other client logos

Surge and Mercor are closer to "pure data sellers," though they also like publishing their own specialist benchmarks, such as Surge's DAYJOB and Riemann-bench.

Put plainly, these are marketing tools aimed at model labs: first use public scores to prove "we understand this vertical, and your model is failing this subject," then, once the reputation is out there, sell the labs the private environments and held-out data that are actually worth money.

0202

Once it is a business, the same batch of data being sold to several labs is unavoidable.

That is an open secret in the industry. Vendors have sold the same set of expert trajectories, or near-identical environments, to two model companies locked in a head-to-head race.

The funny part is that with the identical product, one buyer runs an internal ablation and sees a clear lift, while the other sees almost nothing. There are plenty of possible reasons: the batch's distribution may already overlap with the latter's existing data, the reward function may suit one model better by nature, or the underlying harness may not have been adapted properly.

For the buyer, that turns the whole purchase into guesswork.

It is hard to demand "exclusivity" in a contract and still verify who actually did the work, because the vendor's expert network and historical data pool are shared by design. The only thing you can buy that is relatively exclusive is the part custom-built from your own production failure logs - the errors you hand over for the vendor to target.

But that part is the most expensive and the hardest to scale. And before handing over internal failure cases, a lab has to weigh another question: is the vendor neutral at all?

After Meta took a 49% stake in Scale, orders from several frontier labs visibly shifted toward Surge and Mercor. The reasoning is simple: data on unreleased models and internal failure cases are extremely sensitive, and nobody wants that material passing through a supplier with a competitor as a major shareholder. This is not a technical problem, it is a trust problem.

Surge, founder-controlled with no big-tech shareholder, looks more neutral and has won a fair amount of work from frontier labs. But switching vendors is not a clean solution either - Mercor suffered a supply-chain attack and data breach in March 2026, which still makes some buyers uneasy.

0303

On top of the muddled question of who owns the data, the training environments vendors deliver often do not hold up to scrutiny.

Terminal-Bench 4.0 was recently audited by Preference Model and Epoch AI, and a large number of tasks were found to contain reward hacking, undocumented scoring rules and ambiguous instructions. After those problems were fixed, some models jumped from the low 60s to nearly 90%. Epoch simply labelled it "Flawed."

Earlier work out of Berkeley had already shown that very simple tricks can push SWE-bench and Terminal-Bench to 100% without solving a single real task.

Four-panel diagram: a conftest.py turns six failing tests into passes, and the evaluation result changes from Task FAILED (0/6) to Task PASSED (6/6)

Where does the problem start? Many of the RL environments vendors deliver have never been rigorously tested against cheating before the lab goes looking for holes itself.

Figure: Eight classes of evaluation flaws: no isolation, answers shipped with the test, eval() on untrusted input, unsanitized LLM judge inputs, weak string matching, broken evaluation logic, trusting output from untrusted code, granting unnecessary permissions

If the verifier inside an environment only checks the final file state and ignores the process, or leaves an uncovered shortcut, a model trained with reinforcement learning quickly learns "how to score" rather than "how to do the job." The model is not being malicious; the reward signal itself has a hole in it.

The leading labs have wised up: when an environment arrives, they assign someone specifically to hunt for cheat paths, and send it back or demand a rebuild if they find one. But not every buyer has the bandwidth to do that.

Figure: The three-stage BenchJack audit pipeline: reconnaissance, taxonomy-guided flaw scan, exploit construction

That is before you even get to contamination and memorisation on public leaderboards. Contamination and gaming on SWE-bench Verified are industry consensus; OpenAI has said publicly that it contains many flawed tests and contaminated items, and has largely stopped reporting that score. Once a public benchmark is live, it eventually finds its way into training data or retrieval results through one channel or another.

Accuracy of three Sonar models on HLE, comparing uncontaminated and search-contaminated subsets (6.8 / 17.9, 8.4 / 27.9, 17.4 / 37.5); on the right, a reasoning snippet where the model retrieves the original question and answer from Hugging Face

Even Scale's own research has pointed to search-time contamination: during evaluation, a search-enabled agent finds the original test question and its answer directly on Hugging Face. And even with a private held-out set, a model can still benefit if the task description, code structure or verification logic looks too much like the public version.

Ablation bar charts of accuracy under five conditions - No Search, Default, Blocked HF, Only HF and Date Cutoff - across HLE, SimpleQA and GPQA

0404

One level up, the hardest thing to check is the people.

"Expert trajectories" sounds solid, but the quality actually delivered varies widely. Some vendors pad the work with undergraduates or low-level annotators, so labs now have to spot-check the "professional feel" of trajectories on acceptance - whether the decision path, error diagnosis and iteration logic are genuinely real.

Greier still is the account layer.

A 2026 investigation found training accounts for platforms including Scale, Surge and Mercor being resold illegally on Facebook, Telegram and elsewhere, forming an outright account black market. All of the companies say they have detection in place, yet accounts bearing a "verified" badge are still in circulation.

For the buyer paying top dollar, that means there is no way to confirm who was actually behind the screen for that "expert trajectory," or whether it was reused or subcontracted.

0505

If public leaderboards can be gamed, expert accounts can change hands and even the environments have holes, what does a lab use to inspect the goods?

A set of internal standards far tougher than anything written up in public articles: pass an internal held-out set (usually failure cases clustered from production logs), show a positive, transferable signal after ablation, use an environment that is hard to hack, get expert trajectories past quality spot-checks, and keep producing new ones.

The trouble is that these real acceptance criteria are almost never made public, and they cannot be audited by a third party.

"What counts as valid data" is ultimately decided by the lab's researchers. To win business, a data vendor has to work its relationships, former colleagues and alumni networks to find out "what hurts most right now," then go back and build data to match.

That is why this business runs so heavily on relationships - not because the technology does not matter, but because the rules of judging are not public, and being well connected beats being technically strong.

In the end, none of this mess comes from a few companies deliberately sabotaging anything. It is the natural result of the current market structure: supply concentrated in four leading vendors, the same people often selling the data and running the eval, opaque buyer acceptance criteria, an expert network that is hard to audit, and public leaderboards that are trivially easy to contaminate and game.

The big labs protect themselves by holding private held-out sets and ablations close; the vendors market themselves by publishing public benchmarks. Both are playing the same system.

Taken a step further, there is something darkly funny about it. Economics has Goodhart's Law: when a measure becomes a target, it ceases to be a good measure.

Labs think hiding their internal test sets protects quality, but when the whole data supply chain is asking alumni networks for the exam questions and building data to fit, that last card is slowly losing its value too. The most expensive cost of this grey zone was never a lab overpaying a few million dollars on a data bill. It is that when the entire supply chain is optimising for a particular test set and helping models slip past verifiers for a high score, the models we train at enormous expense may be learning to solve real-world problems - or just becoming a more discreet exam-taking machine.

06About DataMining

DataMining was founded by a team of PhD researchers from the Institute of Software, School of Computer Science at Peking University. We build original, high-quality task data and evaluation infrastructure for LLM and agent development, and have worked deeply with leading technology companies including ByteDance and Alibaba.

We continuously build independent, original evaluation sets for code engineering, long-horizon operation in real environments and extreme mathematical science, as well as embodied AI and cyber data, helping companies map model boundaries accurately and improve performance.

07Evaluation / benchmark areas we can take on:

• Code engineering: Deep SWE / Frontier SWE
• Long-horizon operation in real environments: ALE / Terminal-Bench
• Extreme mathematical science: independent original evaluation sets
• Embodied AI / cyber data

08Image sources

1. Hritik Bansal, Pratyush Maini, "Peeking Behind Closed Doors: Risks of LLM Evaluation by Private Data Curators," ICLR 2025, arXiv:2503.04756 (Figures 1-3)

1. Hao Wang et al. (UC Berkeley), "Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack," arXiv:2605.12673 (Figures 4-6)

1. Ziwen Han, Meher Mankikar, Julian Michael, Zifan Wang (Scale AI), "Search-Time Data Contamination," arXiv:2508.13180 (Figures 7-8)

datamining.cloud ｜ contact@datamining.cloud

DATAMINING.CLOUD