Robotic manipulation is moving past rigid, single-purpose automation toward adaptive, real-time physical interaction. Previously, we demonstrated Spot uprighting and rolling a 15 kg tire. Because this weight exceeds the payload limits of Spot’s manipulator arm alone (11 kg peak, 5 kg continuous), the control architecture coordinated ground reaction forces across its legs and torso in tandem with arm trajectories, synthesizing reinforcement learning with sampling-based trajectory optimization. However, this early setup remained tethered to the lab, reliant on an external motion-capture array, off-board desktop computation, and explicit state-machine scripting to sequence behaviors.

Our latest milestone transitions this system to full autonomy and untethered outdoor execution. By replacing external motion capture with onboard depth-camera perception and running all policy evaluation locally, the robot can now perceive, locate, and manipulate objects in unmapped, real-world environments. The core of this advance lies in visual policy learning that directly processes noisy sensor streams. As a result, the platform dynamically sequences and adjusts behaviors like rolling, uprighting, dragging, and recovering from physical interferences, allowing the robot to adapt to whatever is around it.