Physical AI and embodied RSI: can robots learn, and also improve the way they learn?

01Does a robot becoming proficient count as self-evolution?

A robot arm repeatedly fails to grip the same object: is the policy missing a recovery action, is the world model reading a light touch as a firm grasp, was last time's successful fix never saved, or does it choose the wrong kind of repair every time?

These questions point to one distinction:A robot that is getting better is not the same as a robot that is getting better at getting better.The former may just mean a few more rounds of practice with the same training procedure. The latter also asks whether the training procedure, and the improver in charge of modifying it, have learned anything from experience.

we present this survey Toward Recursive Self-Improvement in Physical AI. Organized around one main thread, where physical experience leaves its changes, it maps the relationship between robot learning, world models, and self-improving agents. Figure 1 draws the path: experience first changes behavior, prediction, or external knowledge; it may then change the learning procedure; and only at the end does it reach the improver itself.

The cup-grasping example is worth carrying through the whole read. Grasping the cup steadily this time is a task outcome; saving a regrasp action is a retained change; starting to prioritize collecting cups that slip easily is a learning-procedure change. Only if it can also learn better ways to modify the procedure from these fixes does the question move one step further, into recursion.

02Four types of change: where does the experience actually get stored?

🎯 Policy | improving behavior.Experience is stored in the policy, value function, or residual control, and directly shapes the next action. An expert correcting the grasp direction, or adding a recovery action after failure, both land here. The question is whether the next execution actually uses it.

🌍 World Model | improving prediction and practice.Contact and interaction data correct the predictive model, letting the robot rehearse inside the model. It can judge whether the object will slip after a grasp, and it can also supply training experience. But more accurate prediction does not automatically mean practice that is more effective for the policy.

🧠 Memory · Skills · Code | improving reuse and recovery.Save the memory that this cup slips easily, a callable regrasp skill, or code that monitors for slipping. They act through retrieval, invocation, or execution, and do not necessarily require updating policy weights.

🔧 Learning Procedure | improving how it learns.Change what data is collected, how success is judged, how skills are edited, and how the policy is updated. The direct product here is a better learning procedure, and the question to watch is whether it teaches better subsequent learners.

Figure 4 links grasping experience, retention location, and use. Pay particular attention to the dashed lines: the same piece of knowledge can have a second use. A policy can execute tasks and can also become a data collector; a world model can predict and can also take part in evaluation.The taxonomy looks at where a change is retained and what it is used for next, not just at what the system is called.

03How can a policy learn new actions from failure?

Failure is not one uniform signal. Figure 5 separates four ways of handling it: relabeling, reward, correction, and replay. Each changes different learning material.

Relabeling reinterprets an existing trajectory. For example, the cup was not picked up but was pushed to another position; that experience can serve a different goal. Reward expresses progress, and expert correction supplies recovery actions the model does not yet have. Replay uses old experience together with new experience, which helps retain existing behavior.

The four signals are not freely interchangeable. A scorer knowing that an action failed does not mean it knows what the correct action is; a person demonstrating a recovery action does not mean the model can already execute it through its own action interface.Feedback has value only when there is an optimizer, a selector, and an execution interface able to absorb it.

04Is the world model changing, or is the policy practicing inside a fixed model?

This group of studies is most easily blurred together by one phrase, robots self-evolving in imagination. Figure 6 marks which component in each system actually gets updated: read the arrows first, the names second.

SILVR：Data is first collected on a real robot, then the task video planner is adjusted round by round; a general video prior provides fixed guidance, and an inverse dynamics model is trained separately. Successful experience returned from real execution feeds into later planner adjustments.

RISE：The dynamics model and value model stay fixed during policy training, and the policy improves under the conditions those models provide. The evidence supports policy improvement; one cannot infer from the use of a world model that the world model is also continually self-correcting.

PlayWorld：Autonomous play data is collected first to fit the world model and the reward, then a noise policy is trained. The figure explicitly distinguishes the frozen base policy from the trainable noise policy. Learning the model first and then training the policy is not the same process as updating model and policy together in a loop.

VLAW：The world model is updated with new and retained real trajectories; the vision-language-action policy is updated with successful real and imagined trajectories; then real data is collected again. The figure shows two rounds of the loop and marks the stages where the reward filter is fit.

So when comparing papers, one practical question to ask is:Which component did this round actually write, and which component does the next round read?Only when both the update location and the use location are stated clearly can you tell behavior learning, downstream learning benefit, and procedure improvement apart.

05Memory, skills, and code: once fixed, is it remembered the next time?

The robot has finally found a way to grasp the cup steadily again. If that is thrown away when the round ends, the next attempt starts from scratch. Keeping experience is the foundation of cumulative improvement, but retention also takes different forms.

Memory is information read out at use time, such as this cup slips easily; a skill is a callable behavior with inputs, outputs, and preconditions, for example regrasp(); running code can monitor state, interrupt an action, execute a recovery, and return to the original task.

Figure 7 separates these three forms and adds a cross-robot reuse check. Observation fields, action semantics, timing, hardware capabilities, and safety constraints all have to line up. A fix that works on robot A does not start working just because it is copied into robot B's folder.

There is one more place where misreading is easy: the monitor keeps being modified while the internal policy may stay frozen the whole time. Overall system performance really did improve, but the source of the change may be external recovery logic. This is exactly why the survey tracks where the change is retained.

06The ladder of claims: 'getting better' needs evidence to match

This survey asks not only whether the robot got better, but also which level of claim the experiments can support. It splits conclusions into behavior improvement, downstream learning benefit, learning-procedure improvement, and recursive self-improvement.Evidence for a lower level does not automatically establish the level above it.

Behavior improvement asks whether the same robot executes the task better. Downstream learning benefit asks whether the data, models, or knowledge left behind help learners trained later. Procedure improvement goes one level up: can the modified collector, evaluator, or updater teach better learners?

Figure 8 gives the comparison for procedure improvement. Two groups of learners share the same starting point, training recipe, and budget, with only the component under test swapped. Final results are judged by an independent, fixed check; an evaluator that was just modified must not grade its own work.

Beyond the current success rate, also look at transfer to new tasks, retention of old capabilities, and total cost. If a new procedure picks out all the easy tasks and scores better for itself, yet never teaches the robot to handle harder states, that is not the learning improvement we wanted to verify.

07R1 and R2: which two tests does recursion have to pass?

🔧 R1: can repair knowledge make the improver better?Give the improver carrying old knowledge and the improver carrying new repair knowledge the same new failure history, tools, goals, and budget. Each proposes a learning procedure, matched robot learners train on them, and the shared physical outcomes are compared.

Here you cannot just count how many suggestions the improver wrote. Whether the suggestions ended up teaching a better robot is the outcome test for this relation. Taking repair knowledge from saved to helping the modifier do better is the first step.

R2: can an improved improver drive its own next improvement?Starting from the updated improver, let it modify itself once with frozen knowledge and once with newly acquired knowledge, producing a control improver and a next-generation improver.

The next-generation improver has to beat two references: a control version with its knowledge frozen, and an updated version that has not yet performed this self-modification. Otherwise the gain may just come from one more round of editing and cannot be attributed specifically to recursive use of the new knowledge.

Figure 9 is a proposed validation design, not a scoreboard of completed physical experiments. Under the assumption that only one candidate per improver passes screening, each random seed requires four hardware-trained learners, and the improver's update cost is reported separately.

Existing physical studies already provide evidence for the lower levels of claim, but the survey has not yet found one that truly isolates the recursive gain. Some systems modify the improver's knowledge or tools, and some systems show improved results; what is missing is evidence that still attributes the gain to recursion after comparison against a knowledge-frozen version.

08Four conditions for gains to keep accumulating

💡 Feedback is informative:Can a grasp-failure record tell apart a missing recovery action, a contact-prediction error, and a scoring misjudgment? A label that only says failed again may not be enough to decide which component to fix.

Changes are usable:Saving a skill is not enough; it also needs the right interface, the optimization capability, and a moment to be invoked. An excellent human correction still cannot be executed if the robot's action space cannot express it.

Changes persist across updates:After tasks, hardware, or policies update, do past repairs still work? The system's own improvement can also shift the data distribution and invalidate the existing evaluator and world model.

Practice can continue:Can the robot safely and affordably try again? The real world carries time, resetting, equipment wear, and human supervision costs, and a dangerous operation can even be irreversible.

Together, the four conditions explain why data volume and component accuracy are not reliable proxies for progress. Collectors with similar success rates may collect data of different training value, and a model that predicts more accurately may also produce practice that is less useful to the policy.

09Putting the framework back on real robots: six case figures

The six figures below come from the paper's appendix. They show what hardware and settings the research happened in; a photo of the setup alone cannot prove recursion. Viewed with the framework above, it becomes easier to tell which kind of change each system actually left behind.

🤖 Figure 10 | practice on real hardware.From pouring milk, threading shoelaces, and USB insertion to sandwich making and walking on a yoga ball. Contact, timing, and stability all bear directly on the result. Beyond watching the demo, ask how failures are fed back and whether fixes are stored in the policy or in external code.

🌐 Figure 11 | robot fleets.ENPIRE, Learning While Deploying, RoboCat, AutoRT, Open X-Embodiment, and RoboArena show robot farms, multi-embodiment data, and distributed evaluation. More robots widen the sources of experience, and also bring differences in interfaces, calibration, and evaluation conditions. Whether shared experience genuinely improves downstream learning still needs controlled comparison.

🌍 Figure 12 | simulation and reality.HARBOR, EmbodiRSI, DreamGen, PlayWorld, World-Gymnast, and EmbodiedSWE connect simulators, video models, and real workspaces. What is most worth tracking here is where the loop closes: does real-world feedback update the model? Does the updated model then guide real practice? Or does training just continue inside a fixed model?

💻 Figure 13 | robot agents that write code.EmbodiedEvo, Show-Harness, HIL-SERL, and BEE involve code control, expert intervention, and online learning. Editing code until the robot succeeds does not mean the code editor has also learned a better way to edit. Modifying the runtime logic and modifying the improver need to be tested separately.

🧪 Figure 14 | autonomous laboratories.A-Lab, Coscientist, and the mobile robotic chemist turn actions into experiments. Feedback is not only grasp success; it can also be material properties and experimental measurements. Real consumables, experiment duration, and safety constraints make the cost of a few more tries unusually concrete.

🚁 Figure 15 | more embodiments.Quadrupeds, drones, robot arms, mobile manipulation, and human-demonstration collection appear in one figure. What they share is learning from physical experience; what differs is how they reset, their action interfaces, and their risks. Whether repair experience from a cup-grasping task transfers to a new failure type cannot be answered by task count alone.

10What experiments are worth doing next?

The directions the survey gives include evaluator alignment, reuse after system changes, physical resource budgets, bottleneck diagnosis, next-experiment selection, and repair-knowledge transfer. A reasonable place to start is components whose modifications are easy to observe, such as reward construction and skill-library editing.

First put the old evaluator and the new evaluator into experiments with matched learners and check the independent results; then have improvers carrying old knowledge and new knowledge handle the same batch of new failures; finally run the knowledge-frozen comparison and track the next-generation improver's gains. This order separates behavioral, procedural, and recursive conclusions.

What evidence is still missing between a robot that can learn and a robot that can improve the way it learns? We hope this survey gives robot learning, world models, self-improving agents, and related directions a shared frame of discussion. Comments and discussion are welcome, and a Star on GitHub or an Upvote on Hugging Face is appreciated.