Robotic manipulation has come a long way since the 1990s. We’ve gone from the two-ball paddle juggling robot to AthenaZero, who can juggle barehanded using onboard vision feedback. By moving away from task-specific passive end-effectors such as cups or paddles and using multi-fingered hands, it can transition between a wide range of patterns including cascade, half-shower, tennis, shower, and box.

As expected, increased flexibility comes at a cost. Contact interactions between articulated fingers and objects are complex and difficult to predict and control.

We address this by learning directly on hardware in real time. Key skills such as the cascade are learned in under 10 minutes of real interaction. Different patterns share common building blocks, allowing skills to be composed and reused.

This is a step toward robots that learn and adapt in the real world.