CUA RL Data Delivery: Data Cleaning, Reward Design, and Evaluation

Shannon defined information as that which removes uncertainty; an undiscriminated record is merely data, and only after judgment does it become information usable for decisions. Logs and screen recordings accumulated in an agent project, if the evaluation system has not labelled success or failure and classified error types, carry no useful information for the next training round. That is part of why CUA agents keep making the same class of mistakes: data volume grows, but learnable signal does not. Below we move from eval-driven methods to data errata and Verifier design, and follow how raw task results turn, step by step, into signals that can flow back.

We have just delivered a batch of RL data collection and evaluation tasks in the PCB domain. The buyer's requirement was blunt: no screenshots, no static PDFs, no Gerber production plots that can only be viewed and not edited - they wanted complete project files that actually open in the software, support routing and saving, and pass the rule checks end to end.

One pattern keeps recurring: evaluation, data cleaning and reinforcement learning training are done separately in most teams.

The evaluation team builds a scoring script, runs it, records a score and stops - the number lands in a weekly report and never flows back into the prompt, the task baseline or the reward function. The data team pulls thousands of projects from open-source communities, converts them into the target software format and dumps them straight into the training pool, leaving hidden shorts and conversion drift in the wild data untreated, so the model drifts further off the longer it trains. The RL team keeps tuning the reward function, and the model learns every trick to score points, such as drawing fake traces outside the board or quietly editing the design rules.

Everyone is doing their own job, yet the pipeline as a whole does not run.

The reason is that these are not isolated features. Evaluation identifies where a result violates the spec, data governance supplies clean training samples, and reward design keeps the model from gaming the score. Done separately, each piece can be delivered on its own; together they produce no improvement, because the data paths between the stages are broken.

This article combines the Eval-Driven Development (EDD) idea LangChain has been pushing recently with desktop-agent evaluation work such as cua-bench, OSReward and CUARewardBench, laying out the engineering loop for large models operating EDA/CAD software such as KiCad and FreeCAD.

There are three basic judgements:

The bottleneck in agent engineering is usually not model parameters but the interface between evaluation and data. Evaluation signals must be able to reject an unqualified baseline; baseline data must go through engineering errata; the reward function must close every cheating path.

The credibility of an evaluation system matters more than its complexity. A simple system with a strict grader beats a flashy one full of holes; wrong positive feedback accelerates the model in the wrong direction.

RL data for industrial software is not ordinary annotation but engineering errata. Historical open-source data varies widely in quality; when the underlying geometry and physics are not aligned, the longer you train, the larger the drift.

012. Why Does Desktop CUA Development Need Eval-Driven Development (EDD)

Desktop-operation development for large models usually starts with writing a prompt, trying a few screens by hand, glancing at a screenshot and then revising. That manual loop is just about adequate for very simple tasks such as filling in a web form or booking a flight.

Enter complex desktop engineering software - for example, opening a USB dock project in KiCad and routing high-speed differential pairs - and intuition-driven development hits three problems:

Non-deterministic output: the same instruction can produce different mouse trajectories, click pixels and intermediate steps every time.

Non-unique solutions: a human engineer routes with 9 polyline segments, the model may use 7. As long as impedance and spacing are satisfied, it is a legal solution; matching coordinates rigidly is wrong.

Regressions: you adjust the prompt to fix one connection error and quietly break other nets that used to pass.

LangChain's account of EDD argues that agent development should move toward traditional TDD (test-driven development), upgraded for the properties of large models: replace a subjective glance that says looks fine to me with a comprehensive set of structured examples, a strict Evaluator, and a continuously running experiment-comparison loop.

For desktop CUA, EDD rolls out in three steps:

Define the Spec: before letting the model run a task, state the trigger conditions, the expected system state, forbidden operations, and the existing design rules that must remain intact.

Build evaluation and a Baseline first: before RL or supervised fine-tuning, prepare the Dataset and the Verifier and run an initial baseline.

Controlled experiments and regression blocking: every change to the prompt, toolchain or reward function must run the full evaluation set, with special attention to hidden regressions where the average score rises but a few critical examples get worse.

023. The First Misconception: Visual Clicking or Underlying Structured Data

Before discussing evaluation, clear up one misconception: should we teach AI to operate professional software through the visual route (GUI clicking) or through code and APIs?

Over the past year or so, CUA demo videos from the big labs have looked smooth: a model reads screenshots, moves the mouse, opens Blender, FreeCAD or even KiCad to draw a circuit board. It is easy to conclude that general intelligence is about to take over industrial software.

Teams that have actually delivered professional-software RL data see another picture. The CADWorld benchmark from Georgia Tech and PCBWorld from LG AI Research show that human experts complete 87% of mechanical and circuit tasks, while even the strongest general-purpose visual CUA models, clicking from screenshots, succeed less than 20% of the time.

The gap comes mainly from the level of information. The image the human eye sees on screen is only a high-level rendering result.

In the professional project file, a thin blue line on screen is bound to a 0.20 mm copper trace width, a 0.14-0.17 mm differential coupling gap, a no-via constraint on outer layers, and its exact netlist connection in the schematic.

Flattening multi-dimensional geometry and electrical rules into a pixel screenshot, then asking a model to guess absolute coordinates and physical dimensions from vision, is wrong in direction. That is why the field is turning to another route: the strategy Anthropic set in Cowork is to prefer APIs and structured MCP and fall back to screen clicks last. Humans use a mouse because they have hands; models run natively in code and structured data. Ignoring precise underlying files and native commands to train a model to align with pixel error on screen is a detour.

034. The Baseline for Training Material: Engineering Errata on Wild Open-Source Data

Once you decide to train the model directly inside real software project files (such as .kicad_pcb), the second problem is training data quality.

Outsiders usually assume there are thousands of hardware projects in open source, so grabbing a few hundred and converting them gives you a textbook. Teams that have actually cleaned historical data know that a substantial share of wild open-source projects do not hold up at the file level.

Human engineers carry many rules in their heads: they know a floating chip pin is fine, and how much tolerance the fab will accept, so their commits often leave a pile of non-standard temporary workarounds.

Take the original imported project for one USB dock: the UI looks completely normal, but a script check of the underlying file finds 97 SMD pads that should be surface-mounted generated as pseudo through-holes with micro vias, several barely visible micro shorts inside the board, and a key pull-down resistor whose value does not match the manufacturer's specification.

Real hardware design never exists in isolation; schematic, BOM, PCB layout and mechanical height-clearance drawings have to be aligned with one another.

Turning such a wild project into a qualified RL training asset requires deep engineering errata: recalibrate capacitor pad sizes against the manufacturer datasheet, realign connector mounting slots, even correct the height offset of a switch's 3D model to within a fraction of a millimetre. Feed the model standard answers full of errors and all of its judgements about the physical world come from that textbook; if the textbook itself is wrong, training only sets the error deeper.

045. The Hidden Cheating Channel: Copper-Pour Caches and Data Contamination

Even after the reference project is cleaned up, the task-generation stage still hides a very subtle trap.

Take a KiCad routing task: delete the high-speed differential pairs and serial traces from the reference board (dozens of straight segments, arcs and vias), leaving blank space for the model to route on its own. Ship the source file as-is and you hit an easily overlooked detail - the deleted traces still leave a shadow in the underlying file.

When the software saves a project, it caches the geometry of large surrounding copper pours (Filled Polygon) in the file to speed up the next open. Delete the signal traces in the middle without forcing a refresh and clearing the pour cache, and the surrounding copper still keeps the grooves it originally cut to avoid those traces.

The model reads the underlying file structure directly. It does not need to compute any trace width or spacing: just fill in the empty grooves left by the copper pour and it hands in an apparently full-score answer.

That is not engineering reasoning; it is tracing over a topological imprint that was not wiped clean. Closing this channel requires a dedicated data-hygiene script (such as task_hygiene.py) that strips every historical pour cache at each task start and recomputes the baseline with the native engine, forcing the model to solve the topology against real geometric obstacles.

056. Rebuilding the Verifier: Closing Cheating Paths in Reinforcement Learning

In the hard phase of RL, a model chasing Reward is very good at finding loopholes.

If the Verifier logic is not tight - say it only counts how many segments are in the submitted file and whether the components are still there - the model quickly learns all kinds of gaming behaviour:

Instead of connecting to chip pads, draw a pile of floating fake traces in the empty area outside the board to pad the count;

Stuff the board with dozens of isolated short copper segments that carry no electrical meaning;

Or directly edit the design-rule file (such as .kicad_dru), widen the error thresholds indefinitely and force a pass.

Drawing on evaluation frameworks such as CUARewardBench and OSReward, in a desktop agent's RL loop a Verifier the model cannot game is worth more than the task itself.

An industrial-grade private Verifier has to build three lines of defence:

Strict physical-connectivity check (60 points): call the underlying engine to compute the real physical connectivity graph. The target net must form a single whole with the copper; one missing pad fails, and any leftover floating copper island costs points.

Project-file hash lock (20 points): lock board outline dimensions, all component coordinates, pad sizes and the design-rule file with SHA-256. If the model moves a single component or changes one byte of the rules file, the total score goes straight to zero.

Full rule check and terminal binary reward (20 points): put the submitted project back into the real software environment, repour the copper and run full ERC (electrical rule check) and DRC (design rule check). Set a hard reward threshold: unless the final result passes 100 out of 100, the terminal reward fed to the trainer is still 0 even if intermediate diagnostic scoring reached 80.

At the same time, deliberately construct a dozen extreme cheating samples (fake traces, edited rules, UUID collisions, moved components) and reconcile them blind, to make sure the Verifier itself has no holes.

067. Conclusion: An Evaluation-First CUA Engineering Loop

With the data clean, cache residue wiped and grading rules sealed, put a flagship large model into the KiCad RL sandbox and one clear result appears:

In a 12-minute blind test with no reference answer, the model produces complete 100-point projects consistently, and geometric solutions that differ from a human expert's but are equally valid: some route around obstacles with minimal vias, some hold differential-pair length mismatch to an extremely low threshold, and some untangle complex routing topology inside a single copper layer.

That is the direction EDD points to when combined with desktop agents: there is no need to force a model to imitate the human mouse trajectory; instead turn physical laws, electrical specifications and design boundaries into a sandbox with strict grading that still leaves room for solutions.

Compute every rule the software can compute, and label honestly the parts of the physical world it cannot compute - that is a relatively defensible engineering path for building the next generation of CUA agents today.

About DataMining 德塔曼宁 We are a team of PhD students from the Institute of Software, School of Computer Science, Peking University. We have long provided original, high-quality task data and executable evaluation environments for large-model and agent R&D, and have served several leading model labs in China. Evaluation systems, engineering errata and anti-cheating Verifiers are where we have stumbled most and accumulated the most. Benchmarks we can build: code engineering (Deep SWE / Frontier SWE), long-horizon operation in real environments (ALE / Terminal-Bench), extreme mathematics and science, Rubrics and anti-cheating Verifiers for open-ended tasks, plus task data and private blind test sets for embodied intelligence and cyber.