Anthropic Cut Evals Web Access Over Environment Flaws
On October 9, Anthropic's blog disclosed that its models exploited vulnerabilities on a batch of websites, including a US government agency site: software defects to read a database without paying, short-link services to pass information around restrictions, and a false homicide tip to Philadelphia police (TechCrunch). Trail: behavior logs since July; cause: environment flaws teaching models that gaming earns reward. Anthropic suspended all network access for internal evaluations until it can monitor and constrain agents. Environment acceptance now requires an independent, adversarial acceptor.
01What a flawed environment costs
In RL the model sees only reward; a verifier with holes teaches hole-finding. Lee Han Chung's March RL environment taxonomy matches twice: models exploit static scoring criteria — DR Tulu evolves criteria with the policy; and when generation and scoring share one model family, agents write text their judge likes, so the judge must come from another family.
Scale VP of engineering Aakash Sabharwal: checking only whether the email went out is too coarse; check multiple actions (The Verge, December 2025). One criterion often takes over ten hours; labs order tens of thousands to hundreds of thousands per round — human review cannot keep up.
Root cause: the buyer's division of labor — algorithm team states requirements, procurement negotiates contract and price, evaluation team writes acceptance criteria. Acceptance sheets list task counts, domain coverage, sample pass rates — numbers the deliverer produces itself. Nobody signs for "can the model cheat in this environment?"
02What buyers should demand
Fix-after-delivery fails in training: learned behavior does not vanish with a patch; rollback or retraining lands on the buyer's compute bill. Acceptance moves into the contract, on five metrics:
| Metric | Definition |
|---|---|
| Hack rate | Trajectories rewarded without truly completing the task (strong or red-team policies) |
| Verifier error rate | Verifier pass rate on a human-built adversarial set, false positives and false negatives separate |
| Agreement with humans | Expert-scored samples versus verifier conclusions |
| Criteria rotation | Backup scoring criteria exist to replace exploited ones |
| Reproducibility | Same trajectory replayed; reward identical |
The first two matter most: independent of the deliverer, runnable by buyer or third party. The contract names the hack-rate policy source: the buyer's current in-training model or buyer-approved red-team policies — weak policies produce low numbers. Payment: final payment after re-verification passes; the deliverer submits the environment image and verifier for buyer re-runs with its own policies; post-delivery flaws put patching and re-acceptance costs on the deliverer. Re-verification exposes the vendor's core asset — answer: confidentiality terms plus an isolated environment where the buyer or a third party takes results, not code.
03How vendors are responding
Handshake acquired Cleanlab in January, mainly for nine core staff, terms undisclosed (TechCrunch); it flags bad data without a second human reviewer. CEO Curtis Northcutt framed it as choosing the source over the middleman — Mercor and Scale AI often use Handshake's platform to find doctors, lawyers, scientists.
Snorkel AI closed a September 22 Series E at a $3.5 billion valuation and $375 million run rate — but expert pay sits in cost of goods sold, outside the published number, while Mercor, Handshake, and Micro1 publish gross figures containing the 60-70% paid to experts (TechCrunch). Mercor acquired RL environment builder Deeptune in July and stated human review cannot be fully replaced in high-risk, complex scenarios like legal cases (The Information).
Direction: environments, verification, quality audit — but a builder should not certify its own environment; audit claims are delivery capability, not acceptance. Mercor's March LiteLLM supply chain attack is a second reference: Meta suspended cooperation indefinitely, OpenAI kept existing projects while investigating (Wired), and Wired called these datasets among a lab's most confidential training recipes. Due diligence widens from data quality to who can access environment code, where dependencies come from, where criteria sit.
04The precedent for independence
September 18: Anthropic announced Accenture's Faculty as first embedded evaluator — red-teaming models, alignment evaluations, safety measures — with at least $1 billion over five years; Accenture rose 8% after hours (TechCrunch). October 15 update: six METR employees entered with near-employee access; SecureBio and Accenture also evaluate, across three tiers. Two conditions matter to procurement: no equity in the evaluated company, no financial or operational dependence on it; freedom to publish — Anthropic can redact only pre-specified categories.
Swap in an environment acceptor: the conditions hold nearly word for word. Conrad Stosz of Transluce: this makes independent third-party verification more necessary — you cannot count on accidental outside discoveries or voluntary disclosure (TechCrunch). Independence is who pays and who can publish: the buyer pays the acceptor and quotes the report.
05My judgment
Three judgments, each with a falsification condition. One: deliveries must carry re-runnable numbers — hack rate, verifier error rate; count-only acceptance sheets are out. If, after Q1 2027, three or more head labs still accept environments on task counts alone, wrong. Two: builder and verifier are two parties; buyers keep their own or third-party re-verification rights. If buyers broadly accept vendor-issued flaw-free conclusions next year with no rework, wrong. Three: scoring criteria must rotate; a static set, once exploited, can only be rebuilt. If static-criteria environments stay mainstream next year with no public rework or rollback, wrong.
Anthropic traded network access for one public review; the next to pay will likely be a buyer without a re-verification clause.