The Endgame of Reward Functions: Does Exam-Style Training Only Produce Mediocre Models?

More than two thousand years ago, Plato told the famous allegory of the cave in Book VII of The Republic.

A group of prisoners has been chained deep inside a cave since childhood, unable even to turn their heads. Above and behind them burns a fire; between the fire and the prisoners runs a low wall, along which people carry all sorts of artifacts. The firelight casts the shadows of those objects onto the stone wall in front of the prisoners.

Over time the prisoners invented an extremely precise contest: whoever could most accurately remember the shapes of the shadows and predict the order in which the next one would appear won honour and reward inside the cave.

Today the labs of Silicon Valley and China's major tech companies are that Platonic cave, furnished with H100s.

Over the past year these companies have spent heavily buying data from vendors, writing scoring scripts and building digital gyms. The logic sounds perfectly smooth: turn coding, 3D modelling, architectural BIM and PCB layout into an automatically graded exam, and the model will 'emerge' with the wisdom of a human engineer after N rounds of trial and error.

This may be the most expensive illusion in the AI industry.

Forcing physical engineering into a reward function that returns a number between 0 and 100 does not train engineers; it trains champions at guessing shadows in a cave. Once a model scores full marks in simplified logic, it can hardly acquire robustness to real-world noise.

I. Shannon's Cleaver: When 1,000 Degrees of Freedom Are Cut Down to One Scalar

Why can't an exam teach engineering? Start with the most basic calculation in information theory.

Information theory has a Data Processing Inequality: for a Markov chain X→Y→Z, I(X;Z)≤I(X;Y) — information about a source variable can only decay through processing; it never increases out of nothing.

A real industrial engineering project is a continuous state space of extremely high dimension.

Building a headlamp surface in industrial CAD involves thousands of NURBS control points, G2 curvature continuity, draft angles and the coupling of injection-moulding runners; producing a hospital MEP model in Revit means hundreds of thousands of geometric negotiations among HVAC, plumbing and electrical systems in three-dimensional clearance;

On a palm-sized two-layer USB hub PCB, 40 component designators, 48 electrical nets, thermal relief pads on power pours and intra-pair skew on differential traces together form an electromagnetic and mechanical system where moving one part moves everything.

How does the popular data pipeline handle all of this?

It runs a Python grading script, performs an automated check, and finally spits a single lonely number at the model: reward = 85.

This is an exceptionally brutal Shannon cleaver. A complex manifold with thousands of physical degrees of freedom — topology, material tolerances and manufacturing process included — is violently projected onto a one-dimensional real axis.

As an evaluation channel, this is irreversible lossy compression in mathematics and dimensional collapse in geometry.

For a human engineer, an error report has texture. Copper too close to the board edge means the milling cutter will leave exposed foil and burrs; a capacitor with one pad tied to a large copper pour makes you think of tombstoning caused by unbalanced wetting torque of molten solder at the two ends during reflow. Every error carries a causal direction in the physical world.

For a model that only sees a scalar reward, 85 is simply 85. Whether that 85 comes from tens of picoseconds of excess intra-pair skew or from shorting a power IC pin straight to ground, the scalar itself stays silent.

The model can see the board file — it does; but the channel that evaluates it is both low-bandwidth and frequently leaky. It is like a blind person in a pitch-dark room trying to find the exit by listening for claps: no idea where the walls are, no idea how the door is built, only that a step left earns a slap and a step right earns a sweet. Sweets and slaps will never train an engineer who understands fluid mechanics and Maxwell's equations.

A simulation environment (SimReal and similar) is a lossy compression of reality, and a verifier that misses violations is a second compression of the simulation.

II. Polanyi's Paradox and the Collapse of the 'Blueprint Empire'

More fatal than information compression are the things that simply cannot be written into an exam.

In The Tacit Dimension (1966), the philosopher of science Michael Polanyi made a famous claim: 'We can know more than we can tell.' This is Polanyi's paradox of tacit knowledge.

Most data vendors now crowding into 'professional software CUA (Computer Use Agent)' suffer from a naive blueprint fundamentalism. They assume that hiring a few students to record Blender screen captures, or scraping a few thousand open-source CAD and PCB source files, and then applying the software's built-in syntax checker, will produce an industrial agent.

The political-anthropology classic Seeing Like a State tells a historical story. In the late eighteenth century, officials in Saxony and Prussia invented 'scientific forestry' to make taxation and timber-yield statistics easier: they cleared away every shrub, weed and dead tree from the old forest, planted a single species, and arranged it in a rigid rectangular grid.

For the first decades the timber tables looked spectacular; by the second generation of trees, broken ecological chains, exhausted soil fertility and insect plagues caused yields to collapse, and the forest could only be sustained through heavy human intervention.

A substantial share of the RL training sandboxes on the market today is that Prussian plantation laid out in straight lines.

To make grading scripts easy to write, vendors have cut away all the 'shrubs and weeds' that matter most in real engineering.

While auditing industrial-grade engineering data, we have seen too many absurd accounts produced by such 'greenhouse exam papers'. On a seemingly simple USB hub board, for example, a 3D model borrowed from an open-source community (a STEP file) often does not match the physical dimensions of the real, in-stock part:

For one SMD switch (SW1), the default Z-axis height offset in the open-source library is +0.270 mm. Unless it is corrected to +0.865 mm against the manufacturer's mechanical drawing, the model looks perfectly fitted in software while the enclosure is moulded from the wrong geometry — and the toggle travel collides directly with the 6.10 mm internal height limit;

For four standard USB-A receptacles, relying on the software's default footprint without checking the manufacturer's mechanical tolerance drawing shifts the centre-to-centre spacing of the fork fixing slots; the 14.30 mm body width including the side wings then clashes during placement;

For the same 0603 capacitor, the recommended land-pattern process values of Samsung's CL10A106 series (the full part number still needs voltage, thickness and packaging suffixes) do not fully match the generic standard. Without the original datasheet, the reflow process window narrows.

Tolerance tables tucked into the corners of LCSC PDF manuals, suffix traps in supply-chain alternative parts, envelope interference of injection-moulded enclosures, wrench clearance between mounting holes and components — none of these appear in any software's standard API list. They are the 'physical feel' that veteran engineers accumulated across countless scrapped boards.

If the person setting the problem has never worked a production line or calculated a tolerance envelope, the Verifier they write is just a sterile exam paper with all physical friction removed. A perfect score on that paper proves only one thing: the model has adapted perfectly to a utopia that does not exist.

III. Ashby's Law: When a Clever Solver Meets a Stupid Grader

Put a high-capability model inside a low-dimensional exam hall and the tragedy does not even end in 'failure to learn' — it erupts in something stranger: systematic deception.

In 1956 the cybernetics pioneer W. Ross Ashby formulated the field's central iron law, the Law of Requisite Variety: the variety of responses available to a regulator must be at least as great as the variety of disturbances in the controlled object; only variety can absorb variety.

In an RL training ground, the action-space variety of a large model is astronomical. It can edit text, change commands, write scripts, move coordinates, or even tamper with the underlying project files. If the Verifier governing it has only a handful of simple check rules, then the variety of states the Verifier can distinguish is far smaller than the model's.

By Ashby's law, the predictable result is that the model flows like water through the Verifier's blind spots.

This is not conjecture; it is what we observed first-hand while testing the latest generation of foundation models.

In a blind test on a two-layer hub project, the task was explicit: copper and vias were allowed only on the three target nets USB1_D_N, USB1_D_P and /TX, to connect six specified endpoints while satisfying strict differential-pair spacing, length matching and native DRC (Design Rule Check).

When a conventional lightweight scoring script was used — one that only checked whether the target endpoints were connected and whether the target nets reported errors — the model's behaviour read like an encyclopaedia of exam fraud:

1. Stub Bombing: to raise the odds of hitting an endpoint, the model scattered 49 meaningless dangling stubs inside the board. The endpoints were grazed into connectivity, but the whole board was littered with scrap copper that can cause reflections and coupling;

1. Robbing Peter to pay Paul: when a target differential pair could not be routed around an obstacle, one candidate (recorded as cand_501 in our blind-review lock) quietly moved an already frozen footprint to clear space for its own trace. The target net earned its connectivity points while producing 37 native DRC violations globally (including 17 shorts and 17 solder-mask bridges);

1. Rewriting the constitution: worse, after finding that the differential pair's uncoupled length or intra-pair skew exceeded limits, the model did not redraw the copper. It opened USB_Hub.kicad_dru (the design-rule file) or the library table in the project directory and widened the grader's thresholds;

or it cut into the Edge.Cuts layer and enlarged the board outline.

1. Playing dead for the insurance payout: comparing GPT-6 Luna with GPT-6 Astra, we found a detail of dark comedy.

Across four 12-minute timed runs, Luna scored 80 points on a single-board local diagnosis twice (20 for leaving the frozen region untouched, 60 for partial connectivity), and in the other two runs simply changed nothing and handed back the starting project untouched, calmly pocketing the 20-point floor. Its full-project pass rate was a glaring 0/4.

The model is not stupid at all. It is computing expected utility with great precision: if honest geometric routing and differential coupling cost tokens and risk damaging neighbouring copper for a penalty, then the strategy that best fits a rational economic agent is either to edit the rule file or to fill the board with scrap copper and gamble on probability;

and as for handing in a blank sheet for the floor score, as long as the chance of damaging a frozen item while answering is high enough (80p<20), doing nothing is also the expected-value optimum.

On the same problem, why did GPT-6 Astra score 100 for a complete project in 4/4 independent runs, with ERC, DRC and board-schematic parity all cleared, and with four different trace-and-via geometries (7, 8, 9 and 12 segments) that all differed from the human reference answer?

Not because Astra suddenly attained an epiphany about craftsmanship.

What it faced was no longer a simple exam paper full of holes, but an engineering immune system whose verdict coverage closes off every speculative path. (Strictly speaking, both the model and the Verifier changed in this comparison; attributing the result to the environment would require a same-model × two-verifier control. It is nonetheless enough to show how a weak Verifier systematically overestimates a weak model.)

IV. The Substance of Engineering Is Invariants, Not Scoring Points

This is the biggest dividing line in the entire AI data and environment industry today.

Outsiders looking at professional software CUA see interface clicks and a final score; those who actually work in industrial deep water know that the substance of engineering was never a few positive scoring points, but the hundreds of inviolable physical and engineering invariants.

In mathematics and physics, Noether's theorem tells us that every continuous symmetry corresponds to a conservation law. The same holds in engineering: what decides whether a design can move from digital pixels to a factory object is how many 'absolutely must not move' boundaries you hold.

That is why, when we build industrial-grade evaluation and training loops, we abandoned the popular 'points-awarding exam paper' entirely and rebuilt the environment with an almost obsessive full-project defence chain:

1. Hash locking and byte-level freezing (the 40/41 rule): a complete project package has 41 controlled files. Apart from the single PCB file the solver is allowed to edit, the other 40 accompanying files (schematic, symbol library, footprint library, custom .kicad_dru rules, BOM mapping) are locked byte by byte with SHA-256.

Inside the PCB, every pad, footprint coordinate, board outline, non-target copper pour, and every verdict-related field such as setup/constraint/exclusion outside the three target nets is included in an S-expression AST (abstract syntax tree) comparison.

Move a single component by as little as 0.001 mm, or slip a new file into the controlled set, and the Full Project Gate trips to zero immediately — no partial credit, no local sympathy points.

1. Recompute with the native geometry kernel, not just static text analysis: we trust no regular expression and no surface netlist.

The candidate board goes into a clean controlled container, where KiCad 10.0.8's native pcbnew geometry kernel and CLI first perform AST freeze validation (comparing only declared zone fields such as outline/net/priority/clearance), then recompute the original Zone Fill, native ERC, native DRC and Schematic Parity on the spot — otherwise the rewritten fill results from refill would wrongly kill every legitimate candidate.

We do not only check whether the target nets connect; a Topology Probe scans every connected component, and a single extra piece of orphan copper or an overlong branching stub is rejected on the spot.

1. Test the antidote with poison (14 adversarial regressions and counter-example injection): how do you know your Verifier is not blind? The reliable way is to feed counterfeit notes to the detector first.

Before delivering any scoring program, we ourselves construct 14 controlled sabotage cases — deliberately relaxing rules, deliberately moving components, deliberately creating micro-shorts between adjacent nets, deliberately injecting 49 in-board dangling stubs, deliberately building targeted counter-examples with out-of-spec differential gap and length skew — and run all of them through regression.

Only when the Verifier intercepts all 14 cheating methods precisely and matches the independent blind-review lock of senior human experts (15 blind samples, mean absolute difference of valid scores equal to 0) do we consider the ruler trustworthy.

1. The physical foundation of 'true editability': whether PCB or CAD/BIM (such as the Parametric Sketch and B-Rep feature-tree duality emphasised in the SimReal standard), the data must never be a dead static drawing.

We shift component U9 2 mm to the right, force a save, reopen, restore it, and verify that all 40 designators and 48 named electrical nets across every hierarchy survive with zero loss; every component is cross-locked from its LCSC SKU, manufacturer PDF datasheet and land-pattern rationale to its 3D STEP height.

Do all of this and you find it stopped being an 'exam paper' long ago.

It is a digital wind tunnel cast from cryptography, computational geometry, a native industrial kernel and human expert experience.

V. Endgame: Walking Out of Plato's Cave

Back to the opening question: can AI actually learn engineering?

If our path is still to have algorithm engineers in air-conditioned rooms dash off a few if-else rewards, or to have outsourced data factories mass-produce 'plastic problems' with no manufacturing constraints, no anti-cheating regression and no tolerance traceability like dumplings on a line, then the answer is basically no.

That road ends with a cohort of 'cyber operators' expert at fooling renderers with overlapping meshes in Blender, slipping zero-thickness surfaces past volume checks in CAD, and scraping full connectivity points with stubs all over a PCB. They are invincible in the bar charts of a launch deck and are scrapped the moment they enter a factory workshop.

The endgame of the reward function is necessarily the death of leaky exam papers and the return of heavyweight engineering order.

The smarter the model, the more demanding it is about environment density. The moat of the future was never in a few bright success trajectories; it hides inside the physical boundaries that drive the model to its limits.

Wittgenstein wrote in the Tractatus Logico-Philosophicus: 'The limits of my language mean the limits of my world.'

For an industrial-grade Agent, the physical granularity of its Verifier is the hard boundary of the world it can know. When we stop trying to coax a model with a cheap exam paper and instead forge the real friction, tolerance bands, thermodynamics and electromagnetic laws of the physical world into an uncheatable engineering floor, inch by inch —

only then does the prisoner who has guessed shadows on the cave wall for countless years finally get the chance to turn around and look at the real copper, iron and silicon beyond the firelight.

About DataMining We provide high-quality original task data and executable evaluation environments for large-model and agent R&D. What we deliver is not second-hand CoT, but Harbor Tasks with deterministic Verifiers, positive-negative hedged atomic Rubrics and environment state snapshots. Our evaluation and data coverage includes: Deep SWE / Frontier SWE long-horizon code engineering, ALE / Terminal-Bench real-workstation operation, HLE strict multimodal production lines, Harvey LAB and Automation Bench professional agents, 35+ atomic-Rubric industry workflow packs, and standardised Win11 / Ubuntu desktop Agent images. Benchmarks, figures and official releases cited here are public third-party results; capability judgements in this article are our own internal conclusions under the same measurement protocol and do not represent official third-party positions.