The ability to flexibly leverage limbs for loco-manipulation is essential for enabling autonomous robots to operate in unstructured environments. Yet, prior work on loco-manipulation is often constrained to specific tasks or predetermined limb configurations. In this work, we present ReLIC, an approach that enables versatile loco-manipulation through flexible interlimb coordination. The key to our approach is an adaptive controller that seamlessly bridges the execution of manipulation motions and the generation of stable gaits based on task demands. Through the interplay between two controller modules, ReLIC dynamically assigns each limb for manipulation or locomotion and robustly coordinates them to achieve the task success. Using efficient reinforcement learning in simulation, ReLIC learns to perform stable gaits in accordance with the manipulation goals in the real world. To solve diverse and complex tasks, we further propose to interface the learned controller with different types of task specifications, including target trajectories, contact points, and natural language instructions. Evaluated on 12 real-world tasks that require diverse and complex coordination patterns, ReLIC demonstrates its versatility and robustness by achieving a success rate of 78.9% on average.
Mobile Interlimb Coordination: Test the robot's ability to manipulate large objects using its arm and one leg while navigating with the remaining three legs.
Stationary Interlimb Coordination: The robot coordinates its arm with one designated leg for object manipulation while maintaining balance through static support from the remaining three legs.
Foot-assisted Manipulation: Tasks can be completed using only the arm, incorporating an additional leg as a manipulator demonstrates measurable performance improvements in stability and task execution.
Reinforcement Learning for Interlimb Coordination (ReLIC): Based on various types of task specifications, ReLIC enables the robot to perform versatile loco-manipulation. Conditioned on the dynamic assignment of limb functions, the ReLIC controller generates the actions through the interplay between a model-based module that prioritizes task success, and an RL policy that robustly maintain stable gaits in accordance with the manipulation behaviors. This design decouples the two challenging subproblems without relying on rigid heuristics or sacrificing inter-module coordination.
Multi-limb Co-Tracking: ReLIC can simultaneously guide the arm and a leg along separate trajectories, using the remaining legs for navigation, while maintaining low tracking error on both—enabling precise interlimb coordination.
Gait Transition: ReLIC supports seamless transitions across gait patterns. The robot can flexibly lift or place any leg to assist manipulation, without pausing or disrupting the overall motion.
Stability: ReLIC can robustly maintain a stationary pose for extended periods, even under external perturbations.
Failures: Failures are categorized into SLAM errors, tracking errors, balance loss, and inaccurate contact.
Contact Points: Tasks can be described through contact points and associated motion directions, from which target trajectories are computed via motion planning.
Language Instructions: ReLIC also supports free-form language instructions by leveraging vision-language models.