In This Story
From September 27th to October 1st, Pittsburgh, PA will become the epicenter of the robotics world as it hosts the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). As one of the premier global robotics conferences, IROS 2026 brings together researchers and industry leaders to present groundbreaking work, showcase robotic innovations and discuss the future of intelligent systems through technical sessions, workshops, and demos.
Research teams at the RAI Institute had several papers accepted, covering topics from closed- loop visuomotor policies and frameworks for self-supervised learning on outdoor LiDAR point clouds to whole-body model predictive control for quadruped table tennis.
RAI will also have a booth in the exhibition hall (#1102) where you can check out some of our custom built robotic research platforms including the Ultra Mobility Vehicle (UMV), Roadrunner, and our Koala grippers for robotic data collection.
Jointly Learning Predicates and Actions Enables Zero-Shot Skill Composition
Learning from Demonstration (LfD) enables robots to learn complex behaviors from expert examples, yet existing approaches often fail to generalize to new compositions of known skills without retraining. Modern generative policies model distributions over action trajectories alone, thus are unable to reason about the symbolic outcomes required for robust composition.
We propose that skills should jointly model action trajectories and the symbolic outcomes they induce. To address this gap, we introduce Predicate-Action Skills (PACTS), a class of closed- loop visuomotor policies that model skills as a joint generative process over action and predicate belief trajectories, producing coherent action–outcome rollouts within a single model. Jointly generating actions and predicates enables PACTS to learn internal representations that improve both action generation and predicate classification. Furthermore, we demonstrate zero-shot composition of learned skills via planning by leveraging online predicate predictions from PACTS as a symbolic interface for sequencing and monitoring execution.
Vernata: Self-Supervised Learning of LiDAR Point Representations
LiDAR serves as a primary sensing modality for robots operating in outdoor environments. However, the performance of deep learning models in this domain is severely limited by the scarcity of labeled data, a direct result of the high cost of 3D annotation. Self-supervised learning addresses this scarcity by learning general-purpose features from unlabeled data. In this work, we present a multi-modal, multi-teacher distillation framework for self-supervised learning on outdoor LiDAR point clouds.
Building upon the Sonata architecture, we introduce Vernata, consisting of three extensions: sparse view augmentation to improve robustness against varying point densities, a memory bank mechanism to stabilize resource-constrained training, and cross-modal distillation utilizing dense, high-resolution 2D image features to enable fine-grained semantic guidance. We evaluate our method on the GrandTour, TartanGround, and Waymo datasets, as well as data collected from our own robotic platforms.
Our experiments demonstrate a significant performance improvement over Sonata baselines, yielding mIoU scores of 54.7 on TartanGround (+5.9 points, +12.1%) and 57.1 on Waymo (+7.3 points, +14.7%). Finally, we show that the self-supervised approach maintains strong performance even in reduced-modality settings (lacking color or normals), achieving competitive mIoU scores of 49.4 and 50.2 on the respective datasets.
Whole-Body Model Predictive Control for Spin-Aware Quadrupedal Table Tennis
Developing table tennis robots that mirror human speed, accuracy, and ability to predict and respond to the full range of ball spins remains a significant challenge for legged robots. To demonstrate these capabilities, we present a novel continuous-time model predictive controller (MPC) for agile full-body control of a quadrupedal robot equipped with an arm.
This formulation enables the flexibility to place time-critical constraints, such as those required to strike a ball, anywhere along the trajectory. We further develop a hybrid model-based and learned spin estimator that can accurately predict ball spin from its observed trajectory and an aiming planner that dictates how the ball must be struck. Notably, a continuous set of stroke strategies emerge automatically from different ball return objectives after combining the aiming planner and whole-body MPC. We demonstrate the system on hardware with a Spot quadruped, evaluate the accuracy of each system component, and exhibit coordination through the ability to aim and return balls with varying spin types. As a further demonstration, the system is able to rally with human players.
