The current paradigm for motion planning generates solutions from scratch for every new problem, which consumes significant amounts of time and computational resources. For complex, cluttered scenes, motion planning approaches can often take minutes to produce a solution, while humans are able to accurately and safely reach any goal in seconds by leveraging their prior experience. We seek to do the same by applying data-driven learning at scale to the problem of motion planning. Our approach builds a large number of complex scenes in simulation, collects expert data from a motion planner, then distills it into a reactive generalist policy. We then combine this with lightweight optimization to obtain a safe path for real world deployment. We perform a thorough evaluation of our method on 64 real-world motion planning tasks across four diverse environments with randomized poses, scenes and obstacles, in the real world, demonstrating an improvement of 23, 17 and 79 motion planning success rate over state of the art sampling, optimization and learning based planning methods.
We present Neural Motion Planner, which consists of 3 main components. Left: Large Scale data generation in simulation using expert planners Middle: Training deep network models to perform fast reactive motion planning Right: Test-time optimization to ensure safe and reliable deployment
Bins: 100%
Shelf: 100%
Articulated: 87.5%
In Hand: 81.25%
Minecraft Sword
Shelf to Bookcase
Bookcase from Side
GPU into Cabinet Drawer
Long Pasta in Microwave
Bookcase from Behind
Tidying Cabinet
Making Coffee
Drying Plates
Rearranging Ketchup
Cleaning Dishes
Defrosting Meat
Sword Dodging
Multi-stage Obstacle Avoidance
Cubby
Dishwasher
Microwaves
Boxes
Shelves
Cabinets
@article{dalal2024neuralmp,
title={Neural MP: A Generalist Neural Motion Planner},
author={Murtaza Dalal and Jiahui Yang and Russell Mendonca and Youssef Khaky and Ruslan Salakhutdinov and Deepak Pathak},
journal = {arXiv preprint arXiv:2409.05864},
year={2024},
}