Multi-Contact Task and Motion Planning Guided by Video Demonstration - ENS - École normale supérieure Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Multi-Contact Task and Motion Planning Guided by Video Demonstration

Résumé

This work aims at leveraging instructional video to guide the solving of complex multi-contact task-and-motion planning tasks in robotics. Towards this goal, we propose an extension of the well-established Rapidly-Exploring Random Tree (RRT) planner, which simultaneously grows multiple trees around grasp and release states extracted from the guiding video. Our key novelty lies in combining contact states, and 3D object poses extracted from the guiding video with a traditional planning algorithm that allows us to solve tasks with sequential dependencies, for example, if an object needs to be placed at a specific location to be grasped later. To demonstrate the benefits of the proposed video-guided planning approach, we design a new benchmark with three challenging tasks: (i) 3D rearrangement of multiple objects between a table and a shelf, (ii) multi-contact transfer of an object through a tunnel, and (iii) transferring objects using a tray in a similar way a waiter transfers dishes. We demonstrate the effectiveness of our planning algorithm on several robots, including the Franka Emika Panda and the KUKA KMR iiwa.
Fichier principal
Vignette du fichier
paper.pdf (638.89 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03945110 , version 1 (18-01-2023)
hal-03945110 , version 2 (24-03-2023)

Identifiants

  • HAL Id : hal-03945110 , version 1

Citer

Kateryna Zorina, David Kovar, Florent Lamiraux, Nicolas Mansard, Justin Carpentier, et al.. Multi-Contact Task and Motion Planning Guided by Video Demonstration. ICRA 2023 - International Conference on Robotics and Automation, IEEE, May 2023, Londres, United Kingdom. ⟨hal-03945110v1⟩
255 Consultations
218 Téléchargements

Partager

Gmail Facebook X LinkedIn More