Rapid Locomotion via Reinforcement Learning

Gabriel Margolis* ¹, Ge Yang* ¹ ², Kartik Paigwar ¹, Tao Chen ¹, Pulkit Agrawal¹ ²

¹ MIT Improbable AI Lab ² NSF AI Institute for Artificial Intelligence and Fundamental Interactions

Presented at Robotics: Science and Systems 2022 (Talk)

Abstract: Agile maneuvers such as sprinting and high-speed turning in the wild are challenging for legged robots. We present an end-to-end learned controller that achieves record agility for the MIT Mini Cheetah, sustaining speeds up to 3.9 m/s. This system runs and turns fast on natural terrains like grass, ice, and gravel and responds robustly to disturbances. Our controller is a neural network trained in simulation via reinforcement learning and transferred to the real world. The two key components are (i) an adaptive curriculum on velocity commands and (ii) an online system identification strategy for sim-to-real transfer leveraged from prior work.

Indoor Sprint (3.9 m/s)

Outdoor 10-Meter Dash (3.4 m/s)

Indoor Spin Test (5.7 rad/s)

Spinning on Icy Patch

Climbing a Gravel Hill

Recovery from Tripping

High-Speed Controlled Failure

Operating with Stuck Joint

Media Coverage

Bibtex

@inproceedings{margolisyang2022rapid, title={Rapid Locomotion via Reinforcement Learning}, author={Margolis, Gabriel and Yang, Ge and Paigwar, Kartik and Chen, Tao and Agrawal, Pulkit}, booktitle={Robotics: Science and Systems}, year={2022}}

Our Related Projects

Learning to Jump from Pixels

Gabriel Margolis, Tao Chen, Kartik Paigwar, Xiang Fu, Donghyun Kim, Sangbae Kim, Pulkit Agrawal

A System for General In-Hand Object Re-Orientation

Tao Chen, Jie Xu, Pulkit Agrawal

External Related Projects

Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion

Gwanghyeon Ji, Juhyeok Mun, Hyeongjun Kim, Jemin Hwangbo

Learning to Walk in Minutes using Massively Parallel Deep Reinforcement Learning

Nikita Rudin, David Hoeller, Philipp Reist, Marco Hutter

RMA: Rapid Motor Adaptation for Legged Robots

Ashish Kumar, Zipeng Fu, Deepak Pathak, Jitendra Malik

Acknowledgment

The authors thank the members of the Improbable AI Lab and the Biomimetic Robotics Laboratory for providing valuable feedback on the project direction and the manuscript. We are grateful to MIT Supercloud and the Lincoln Laboratory Supercomputing Center for providing HPC resources. The Mini Cheetah robot used in this work was donated by the MIT Biomimetic Robotics Laboratory and NAVER. The Biomimetic Robotics Laboratory also provided hardware support for the robot. This research was supported by the DARPA Machine Common Sense Program and in part by the MIT-IBM Watson AI Lab, and the National Science Foundation under Cooperative Agreement PHY-2019786 (The NSF AI Institute for Artificial Intelligence and Fundamental Interactions, http://iaifi.org/). This research was also sponsored by the United States Air Force Research Laboratory and the United States Air Force Artificial Intelligence Accelerator and was accomplished under Cooperative Agreement Number FA8750-19-2-1000.