Scalable Dexterous Robot Learning with AR-based Remote Human-Robot Interactions

Abstract：Robot learning is studied in the dexterous robot arm-hand systems, where the remote human-robot interactions via augmented reality (AR) are constructed to collect the expert demonstration data for enhancing the system robustness. In such a system, an imitation and contrastive learning empowered RL algorithm is proposed to address the general manipulation task problem. To validate the efficacy of the proposed algorithm, both the physics simulations via PyBullet and real-world trials are carried out. The results demonstrate that the proposed algorithm not only significantly reduces the training time, but also achieves high success rate for completing the manipulation tasks. By conducting the ablation study, it is confirmed that the proposed algorithm can effectively overcome policy collapse.