Abstract
Legged robots, particularly humanoids, represent an emerging technology whose widespread acceptance depends on their ability to perform meaningful tasks at the human cadence in the real world. These systems are inherently complex, and unlocking their full potential requires control strategies that coordinate whole-body motions. These complexities pose significant challenges for current motion control methods, underscoring the need for innovative approaches that extend beyond existing frameworks. Similar to how text data catalyzed progress in natural language processing, motion data from humans and animals holds the potential to usher in a new era of physical intelligence. Yet, unlike abundant and well-structured text corpora, human motion data is sparse, often incomplete, and typically lacks action labels, which limits the direct application of supervised learning techniques. To bridge this gap, recent years have seen the integration of imitation learning with structured simulation-based methods like reinforcement learning and trajectory optimization, which shows promise in generalizing natural behaviors to robotic systems. At the same time, transferring behaviors from simulation to physical hardware has taken significant strides, particularly for specialized policies. These advancements have been primarily fueled by improvements in robotic hardware, physics simulations, and techniques for sim-to-real transfer. As a result, the field of robotics has evolved considerably. Today, robotics is shifting its focus from specialized policies toward broader challenges such as generalization, high-level control, and behavior sequencing, all while continuing efforts to bridge the gap between simulation and reality. This workshop aims to gather experts in 3D human/animal modeling, motion generation algorithms, and robotics to review recent progress and chart the next steps for advancing the field. It will serve as a unique forum where the latest advances in computer graphics, particularly in controlling simulated characters, intersect with cutting-edge developments in robotics, from hardware innovations to sim-to-real transfer. The workshop aims to drive innovation in these fields by encouraging deeper collaboration across disciplines and utilizing their complementary strengths.
Content
This workshop will bring together leaders in human/animal simulation, control, animation, and robotics to discuss the state-of-the-art techniques for natural motion generation of physics-based characters and robots. The workshop has three main aims that motivate the breadth of participation:
- Expose algorithm developers to emerging technologies that create accurate 3D human models from various input data sources, such as a single image or video. (Data to Model)
- Discuss the benefits, drawbacks, and long-term potential of ongoing developments in physics-based character motion generation to guide their adoption in robotics applications. (Model to Physics-based Characters)
- Create a forum for roboticists to address the challenges of transferring simulation results to physical robots. (Physics-based Characters to Robots)
The 3/4 day workshop will begin with a 15-minute introduction that outlines the open research questions related to extending natural behavior to robotic forms. Throughout the day, the presentations will be linked to these questions through Q&A sessions, connecting the various topics of this workshop. The workshop brings together presentations from industry leaders and researchers from top institutions and academia.
The presentations are divided into two sessions of 2 and 3 speakers. Each presentation is scheduled within a 35-minute time slot, followed by a 10-minute Q&A. This format will allow for rich interaction with the audience. Furthermore, inviting speakers from both industry and academia will promote the exchange of ideas and enhance the influence of scientific advances across various technical domains. Finally, the last 45 minutes of the workshop will be dedicated to a panel discussion summarizing the key points that emerged during the day. The tentative schedule is as follows:
| Time | Content |
| 10:30 – 10:45 | Welcome & Introduction |
| 10:45 – 11:30 | Talk 1: Naureen Mahmood |
| 11:30 – 12:15 | Talk 2: Xue Bin (Jason) Peng |
| 12:15 – 14:00 | Lunch Break |
| 14:00 – 14:45 | Talk 3: Yuting Ye |
| 14:45 – 15:30 | Talk 4: Moritz Bächer |
| 15:30 – 16:15 | Talk 5: Michiel Van de Panne |
| 16:15 – 16:30 | Buffer Time |
| 16:30 – 17:15 | Panel Discussion |
Speakers
Naureen Mahmood
Naureen Mahmood is the CEO and co-founder of Meshcapade, an award-winning tech start-up based in Europe’s largest AI ecosystem: Cyber Valley. Mahmood received her B.Sc. from Lahore University of Management Sciences, Pakistan in 2006. She received a Fullbright Scholarship for her graduate studies, she completed her M.Sc. at Texas A&M’s Visualization Department in 2011. From 2012 to 2018 she worked at the Max Planck Institute for Intelligent Systems in Tübingen, where she has been a key author of several academic publications in computer graphics, machine learning and computer vision.
Xue Bin (Jason) Peng
Jason is an Assistant Professor in the School of Computing Science at SFU, and a part-time Research Scientist at NVIDIA. He received a PhD at UC Berkeley, and an MSc and BSc at UBC. His work leverages machine learning to develop motor control models that enable simulated agents and real-world robots to reproduce the agile and versatile motor skills exhibited by humans and other animals.
Yuting Ye
Yuting Ye is a research director at Meta Reality Labs Research, with the goal of creating intuitive tools that empower everyday users to express themselves through 3D animation and storytelling. Her current work focuses on character animation and control for embodied avatars in the Metaverse, including full body tracking in VR and motion retargeting to stylized and fantastical avatars. Before joining Meta, Yuting worked at Industrial Light & Magic (ILM), where she contributed to several award-winning software systems and blockbuster movies, such as Star Wars, Transformers, and the Avengers series. Before ILM, she earned a PhD in computer science from Georgia Tech.
Moritz Bächer
Moritz Bächer is the Lab Director of Disney’s Zurich-based robotics team, where he leads a strategic program focusing on the development of novel model- and learning-based tools for the design and control of believable robotic characters. His core expertise is the optimal design and control of both soft and rigid systems, using a combination of differentiable simulation and reinforcement learning. Prior to joining Disney, Moritz received his Ph.D. from the Harvard School of Engineering and Applied Sciences and his master’s degree from ETH Zurich.
Michiel van de Panne
Michiel van de Panne is a Professor in Computer Science at UBC, with interests in deep reinforcement learning, physics-based models of human and animal movement, robotics, and control. Together with his students and collaborators, his work has seen use in film, games, and humanoid robotics. He helped co-found the ACM/Eurographics Symposium on Computer Animation and currently serves as Deputy Director of CAIDA, which is UBC’s main AI organization.
