The Reckoning for Embodied-AI Data Factories: Why Hundreds of People in VR Headsets Grabbing Cups Still Cannot Train a Useful Robot

In the second half of 2026, the busiest places in the embodied AI industry are not roaring manufacturing workshops, but newly inaugurated "data collection centers" across the country.

In factory buildings spanning thousands of square meters, dozens or even hundreds of dual-arm robots are lined up in formations. Wearing VR headsets and holding motion-capture controllers, data collectors repeat the same actions eight hours a day: picking up a cup from the table, folding a towel, and stuffing parts into a plastic crate.

At the World Internet Conference in Wuzhen in early November, Cao Peng, Chair of the JD.com Technology Committee, gave a rather honest assessment: if a perfect score for general embodied AI is 10, the entire industry today is probably only at the level of 3 to 4. At the same time, he announced plans to build a 10-million-hour-scale dataset over the next two years.

The hardware shipment numbers look impressive, but their destinations don't hold up to close scrutiny. Data from market research firm Counterpoint shows that although global humanoid robot shipments surged by nearly 300% in the first half of 2026, those actually entering smart manufacturing and logistics scenarios accounted for less than 20% combined. The remaining 60% or more were entirely absorbed by entertainment and commercial performances, scientific research and education, and data collection.

To put it bluntly, a large number of expensive real robots were bought only to be sent into data collection farms to serve as "video recorders" for the next generation of models.

Yet algorithm teams quickly stumbled upon a frustrating reality: even after accumulating tens of thousands of hours of teleoperation video in data collection farms, the moment the model leaves that specific lab table with its colored tablecloth—if the lighting changes, if a softer paper cup is used, or if someone accidentally nudges a storage box nearby—its grasping success rate plummets off a cliff.

With hundreds of people wearing VR headsets day and night just to grab cups, why can't they feed and train a general-purpose model that actually works?

Let's do the math.

0101 The Physical and Economic Ledger: A Shell Game of Numbers

Pre-training large language models relies on scraping existing web pages from the internet at near-zero marginal cost. In the physical world, however, there are no ready-made joint torques or spatial trajectories to harvest for free. Every single second of real-robot data must be ground out in real time through human labor and hardware.

This grinding process suffers from a staggering attrition rate. According to frontline surveys, under the mainstream one-to-one physical teleoperation paradigm—factoring in equipment calibration, motor overheating protection, mechanical fault recovery, and the time spent manually resetting objects on the table after each grasp—an hour of work by a skilled operator typically yields less than 30 minutes of valid trajectory data saved to disk.

Even more troublesome is the "embodiment gap." Google hit this wall when leading the Open X-Embodiment dataset: 22 robots with different configurations varied completely in number of axes, link lengths, camera angles, and control frequencies. Joint trajectories collected at a cost of tens of millions on Manufacturer A's dual-arm platform cannot even align their coordinate systems when directly applied to Manufacturer B's 7-DoF robotic arm.

Domestic motion capture equipment provider Noitom also stated bluntly in its 2026 review: the biggest drain on the current industry lies in "one-machine, one-collection," which produces a massive number of non-interoperable data silos.

Since real-robot teleoperation is so expensive, causes such high wear and tear, and is so difficult to transfer across embodiments, why have regions across the country been frantically launching data collection bases at the scale of hundreds of units over the past year?

Because behind this lies a "financial ledger" detached from model performance.

Some enterprises partner with local platforms to establish so-called data centers, first selling their own robots at high prices to the data center to recognize hardware revenue; the data center then sells the collected trajectory data back to the robotics company or related parties.

The prospectus of a leading humanoid robotics company once disclosed that of its full-size robot sales revenue, about 45% came from data collection operations, while the share of revenue genuinely derived from industrial manufacturing scenarios was only 3.88%.

When "data collection" itself becomes a means to absorb hardware capacity and inflate revenue streams, the KPIs naturally turn into: How many hours were the machines running today? How many tens of thousands of trajectories were saved to disk? As for whether those tens of thousands of cup-grasping trajectories actually contain the physical information the model truly lacks, nobody bothers to look into it.

It was not until the second half of 2026, when the open-source community released a massive amount of free, high-quality human manipulation datasets, that the unit price of homogenized, hourly-billed motion trajectories on the market rapidly collapsed, and this business model of "piling up headcount to rack up hours" finally stopped making financial sense.

0202 Technical Flaw 1: Losing the Soul of Physical Interaction Turns Models into "Spatially Blind"

Setting aside the commercial bubble and looking solely at the technology itself: why can't data recorded purely with VR headsets and visual cameras train models with generalization capabilities?

The most direct reason is that this data loses the core dimensions of physical interaction right from the source: force sensing and precise spatial alignment.

When a human wears VR controllers to teleoperate a robotic arm to grab a plastic cup, their eyes stare at the screen while their wrist stops at the edge of the cup based on experience. At this point, only two types of data are saved to disk: the RGB frames from the camera and the position commands of each joint of the robotic arm. What is conspicuously missing is the force distribution, micro-slippage, and deformation feedback at the exact moment the fingers touch the cup wall.

In a lab, grasping rigid wooden blocks might only require pure visual alignment. However, real-world commercial environments are filled with high-density contact tasks.

In August 2026, a unicorn startup named Sharpa deployed the world's first zero-modification commercial robotic Blizzard ice cream station in Shanghai. To make a qualified Blizzard, the robot must execute 55 consecutive action steps.

An on-site breakdown shows that 98% of these 55 steps directly rely on tactile feedback: from pulling a single soft paper cup out of a tightly nested stack without crushing it, to fine-tuning grip force in real time based on machine vibrations, all the way to the final 180-degree flip for the "upside-down no-spill" test.

Sharpa succeeded only because its dexterous hand packs thousands of tactile sensing units into each fingertip, firmly embedding contact force signals into the control loop.

In contrast, to cut costs, most outsourced data factories uniformly use two-finger parallel grippers without tactile arrays, or simply record joint positions without feedback.

The longer a model is trained on such datasets, the more it resembles a carpentry apprentice who learns solely by watching videotapes—it memorizes at which pixel coordinate the hand should close, but once given a cup with a different surface friction coefficient or 50 grams of extra weight, it has no idea how much force to apply to maintain static friction.

Beyond lacking a force dimension, current mainstream VLA (Vision-Language-Action) models are also widely bottlenecked by the precision of "Spatial Grounding."

Large multimodal models are naturally "spatially blind." They can recognize a mug and a gripper in the frame, yet cannot accurately calculate how many millimeters separate the gripper fingertips from the mug handle; even less can they figure out where the target object's boundaries go when the robotic arm's wrist rotates 30 degrees and blocks half the view.

In practical engineering delivery, we found that feeding raw teleoperation videos directly into the model often causes it to learn spurious correlations from background textures. To solve this problem, leading domestic embodied AI companies partnering with DataMining no longer purchase crude raw footage, but instead custom-order Grounding QA and pixel-level target annotations in real-robot scenarios.

Specifically, on real robotic arm operation frames, the target object's 2D coordinates, bounding boxes (BBox), and scene semantic descriptions are explicitly bound to the robotic arm's own joint states and next movement trends, turning them into question-answer pairs cross-validated by multiple models.

On keyframes where the end-effector gripper is about to contact the object and severe self-occlusion occurs, pixel-level Mask annotations must be applied to the target object, accompanied by double-blind expert review.

Only by first bridging the gap between "understanding pixels" and "accurately calculating 3D physical space" will the downstream action policy network avoid grasping at thin air the moment the wrist turns.

0303 Technical Flaw 2: The "Greenhouse Effect" of Imitation Learning and Compounding Error

Even if visual spatial localization and sensor dimensions are filled in, the collection logic of traditional data factories has an even more fatal algorithmic flaw: the Compounding Error of imitation learning.

This is a theorem rigorously proven in the machine learning field as early as 2011. Under the Behavior Cloning framework, if the training data consists entirely of "flawless walkthrough videos" where collectors smoothly pick up a cup, then tiny single-step prediction errors from the model will accumulate exponentially over time during continuous execution.

The reasoning is quite simple. Collectors are all skilled workers; when recording data, their hands always move steadily along the optimal path, almost never knocking over a cup or bumping into a corner when turning in a hallway. As a result, the trained model has only ever seen frames on the "correct track" throughout its life, becoming a veritable "fair-weather warrior".

Once in the real-robot closed-loop testing phase, if the floor is slightly slippery, or a lighting change causes the action at frame 10 to deviate by 2 millimeters, by frame 20, the robotic arm's position has already drifted from the perfect trajectory. At this point, the image captured by the camera becomes an "Out-of-Distribution" state that has never appeared in the training set.

Because human collectors never demonstrated "how to recover" from this off-track position, the model instantly panics and immediately makes even more outrageous moves until it sweeps everything on the table onto the floor.

Thousands of hours of perfect trajectories actually have an astonishingly low information density during real-robot testing. The smooth cup-grasping action collected in the 500th hour and the action collected in the 1st hour provide almost completely overlapping gradient information.

Back then, the solution was called DAgger (Dataset Aggregation): first let the immature model run on its own; when it drifts off course or is about to crash, a human expert intervenes to specifically record the corrective process of "pulling it back from a drifted state to the normal track," and feeds this into the training set for iteration.

This logic is now becoming a core watershed in post-training data procurement for embodied AI. Taking the embodied navigation data delivered by DataMining to a domestic university's embodied AI lab as an example, the team did not pile up massive redundant samples of walking in straight lines, but broke the work into two steps: the first step uses multimodal spatial perception tasks for Supervised Fine-Tuning (SFT) to build foundational perception;

The second step directly cuts into building DAgger post-training data for embodied navigation, specifically targeting edge states where the robot drifts off course in complex environments or gets stuck by dynamic obstacles, collecting intervention trajectories of "recovering back to the correct path from a deviated pose."

A model only needs a few hundred demonstrations to learn how to pick up a cup, but learning how to adjust its pose to save the situation in that half-second when the cup slips or the wrist gets knocked askew relies on specially designed "recovery data", rather than having collectors repeat the grasp ten thousand more times in a greenhouse.

0404 Doing the Math on Alternatives: Handheld Grippers and World Models—Shortcuts Have Pitfalls Too

By the end of 2026, the industry had already recognized the dead end of traditional VR real-robot data collection factories. To bypass the expensive, slow, and cross-embodiment limitations of real robots, academia and industry are aggressively pursuing two alternative routes: one is "handheld gripper collection" completely detached from robots, and the other relies on "World Action Models" to generate the future in the mind.

However, each of these two new approaches has its own tough bill yet to be settled.

Let's look at the handheld gripper route first.

In the second half of 2026, solutions based on handheld dexterous manipulation collection systems (such as YUBI and the subsequent AIRoA-0) gained massive popularity. AIRoA-0 did something extreme: it pre-trained on 16,000 hours of bimanual demonstration data recorded purely with handheld grippers, without using even a single second of real-robot data throughout the entire process.

Across 10 high-difficulty long-horizon tasks, it achieved an average success rate of 78%, whereas the four mainstream VLA models tested alongside it peaked at only 27%. More importantly, the exact same set of model weights can be deployed directly onto robotic arms of different configurations with zero modification.

The trick that allows the handheld route to achieve cross-embodiment transfer lies in tightly anchoring the state and action spaces to that specific gripper and its two wrist cameras. Yet its physical boundary is also locked right here: every robot to be deployed must have the exact same gripper installed at its end-effector as the one used during data collection.

Once downstream clients use five-finger high-DoF dexterous hands, or the task requires using the robotic arm's elbow and whole-body coordinated force to move heavy objects, the dynamics calibration debt skipped by handheld grippers immediately has to be paid back with interest.

Now look at another highly anticipated route: "world models."

Since real-robot collection is slow, can we let the model predict actions while mentally simulating upcoming frames like a video generation model? In late September 2026, an in-depth study on World Action Models (WAM) exposed their core pain point during real-robot execution: the future frames imagined by the model often carry severe "optimistic hallucinations."

During the free-movement phase before the robotic arm touches the object, the frames predicted by the world model are highly consistent with reality; however, once it enters the fine-contact phase (such as snapping buckles or when an object slips), if friction and collision in the real physical world deviate even slightly, the model—blindly continuing to execute the imagined action sequence along a fixed horizon—will immediately grasp at air or even damage the hardware.

The research team had to specially train a lightweight "future-reality consistency verifier" to compare the model's imagined frames with the real frames captured by the camera in real time on the physical robot; as soon as it detects that the physical contact does not match the mental simulation, it immediately cuts off the original plan and replans, which finally boosted the real-robot success rate by 35%.

As you can see, whether it is handheld grippers or world models, they ultimately cannot bypass the same conclusion: "dirty data" and blind trajectories that lack strict spatial verification, contact physics alignment, and failure recovery annotations—even if scaled up to tens of millions of hours—cannot automatically give rise to emergent generalization capabilities in the real physical world.

0505 Conclusion: Do Not Mistake "Manual Labor Recordings" for AGI Fuel

Looking back, the reason the embodied intelligence industry rushed to build thousand-person-scale VR data collection factories between 2025 and 2026 is not because practitioners did not understand compounding errors, nor because they were unaware of the consequences of lacking tactile feedback and spatial alignment.

This is an inevitable product of the collision between capital cycles, local investment attraction demands, and early technical routes. For hardware manufacturers in urgent need of commercialization, selling robots to data collection centers is the fastest way to realize current revenue; for local industrial parks, hundreds of robots grasping cups in unison is the most intuitive and easiest-to-inspect model project for the digital economy;

For early algorithm teams, before figuring out how to design a corrective data distribution, piling up collection duration from 1,000 hours to 100,000 hours is the most effortless narrative to prove to investors that they possess a "data moat."

However, hidden within this is a classic Goodhart's Law trap: when "collection hours" turns from an auxiliary reference for measuring model capabilities into a rigid assessment metric for financing, subsidies, and hardware sales, the entire production system automatically tilts toward simple, repetitive actions that are easiest to pad hours with and least prone to errors.

Many teams like to use the data flywheel of Tesla FSD to defend their own data collection factories, while deliberately ignoring the fundamental difference in economic structure between the two. Tesla's millions of mass-produced cars are bought by consumers with their own money and driven on the road.

Routine normal driving is never uploaded at all; only at the exact moment a human driver slams on the brakes or jerks the steering wheel to take over the system—that high-value out-of-distribution moment when model predictions clash sharply with human decisions—does Shadow Mode trigger data transmission.

Its marginal hardware cost is zero, and by nature it filters out only the "corrective samples" with the highest information density.

Meanwhile, the vast majority of embodied AI data collection factories today are doing the exact opposite: spending huge sums of their own money to buy robots without a closed commercial loop, hiring hundreds of full-time workers, and recording uneventful "peaceful days" from morning till night in highly controlled fake rooms.

The second half of the embodied AI data race is long past comparing who has the largest factory floor or hires the most VR collectors. Leading embodied AI teams are shifting their budgets from extensive teleoperation outsourcing to high-precision real-robot spatial Grounding and DAgger corrective post-training because everyone has finally figured out the math.

When the open-source community can already provide tens of thousands of hours of basic grasping videos for free, if the millions of trajectories held by a data factory still lack both sub-millimeter spatial mask alignment and a single corrective record pulled back from the brink of failure, what is sitting on those hard drives—is it fuel toward artificial general intelligence, or merely a video recording of human physical labor captured at the expense of costly compute and electricity?

If your embodied AI team is advancing R&D in VLA model spatial localization, complex contact-rich manipulation, or indoor/outdoor navigation algorithms, and is facing engineering bottlenecks such as poor generalization of real-robot teleoperation data, grasping failures in self-occluded scenarios, or large compounding errors in long-horizon tasks, feel free to connect with DataMining.

Founded by a PhD team from the Institute of Software at Peking University's School of Computer Science, DataMining's core members specialize in multimodal and large model data engineering, deeply serving leading tech enterprises such as ByteDance, Alibaba, and Tencent, as well as top-tier domestic embodied AI companies and university laboratories.

In embodied AI, rather than producing homogenized, ineffective hours, we focus on delivering customized algorithmic data with high information density:

• Customized Grounding QA for real-robot scenarios: explicitly integrating 2D coordinates, BBox, scene semantic descriptions, proprioception, and motion awareness, cross-validated by multiple models.
• Pixel-level BBox/Mask annotation for real-robot targets: introducing double-blind expert review for hand-object interaction occlusion scenarios to strictly control millimeter-level spatial alignment accuracy.
• Embodied navigation SFT and DAgger post-training corrective dataset construction: helping model teams break through generalization bottlenecks in real physical scenarios with less, more accurate closed-loop data.

📚 Image Sources

[1] Open X-Embodiment: Robotic Learning Datasets and RT-X Models, arXiv:2310.08864, Figure 1

[2] SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model, arXiv:2501.15830, Figure 2

[3] A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning, Ross et al., AISTATS 2011, arXiv:1011.0686, Figure 2

[4] AIRoA-0: Long-Horizon, Precise, Cross-Embodiment Manipulation from Handheld Data, arXiv:2611.03815, Figure 2

[5] When to Trust Imagination: Adaptive Action Execution for World Action Models, arXiv:2605.06222, Figure 1

All 5 images above are original public figures from the papers, inserted in the order of the text; the captions only describe the visible content in the images.