Project: Transfer Learning for Robot-to-Robot Adaptation

Description

A popular paradigm in robotic learning is to train a policy from scratch for every new robot. This is not only inefficient but also often impractical for complex robots. The project revolves around the exploration and advancement of techniques for transferring policies between different robotic systems, particularly in environments built in the MuJoCo physics-based simulation engine. The primary objective is to enable efficient policy transfer, allowing one robot to adapt and leverage knowledge learned on another robot with significantly different kinematics and morphology. This endeavor aims to address the inefficiency and impracticality of training policies from scratch for each new robot, especially in the context of complex robotic systems.

Objectives:

Continuous Evolutionary Models for Policy Transfer: Building upon the foundation of the REvolveR [1] method, which uses continuous evolutionary models for robotic policy transfer, the project aims to extend and refine this approach. Explore ways to interpolate between the source robot and the target robot by identifying a continuous evolutionary change in robot parameters.

Pre-Training Methods: Investigate the effectiveness of pretraining techniques in multi-robot learning. Assess how pretraining on one robot can accelerate learning on a different but related robot, improving sample efficiency and adaptation to new environments.

Robot-Robot Transfer with Prompting: Explore the concept of robot-robot transfer with prompting, where robots communicate or prompt each other to facilitate knowledge transfer and adaptation. Investigate the role of communication in transferring policies across robots.

Curriculum Learning and World Models: Experiment with curriculum learning and world models [2] to enhance the policy transfer process. Evaluate the benefits of structured learning and model-based approaches in multi-robot adaptation.

Outcomes:

This project aims to contribute to the field of multi-robot learning and adaptation by advancing the capabilities of robots to efficiently transfer policies and adapt to varying kinematics and morphologies. The research findings have the potential to revolutionize multi-robot systems, improving their ability to adapt and learn in diverse environments.

Reading:

[1] X Liu, D Pathak, KM Kitani. REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy Transfer. ICML 2022

Details

Supervisor

Meng Fang

Secondary supervisor

Tristan Tomilin