IEEE IPAS 2025

End-to-end Sketch-Guided Path Planning through Imitation Learning for Autonomous Mobile Robots

SKIPP — Sketch-guided Imitation for Path Planning

Anthony Rizk1,2 Charbel Abi Hana2 Youssef Bakouny1 Flavia Khatounian1
1Saint Joseph University of Beirut 2idealworks GmbH, Munich
Paper (IEEE) Code 🤗 Dataset 🧠 Models

Abstract

Path planning is crucial for Autonomous Mobile Robots applications. Traditionally, path planning based on human input and preferences has relied on hard-to-define reward-based learning or costly techniques requiring additional hardware. This work introduces a more accessible and flexible approach through sketch-guided imitation learning, where nontechnical users can simply draw the desired navigational path on a provided 2D map, which is then used to teach U-net models path planning behaviors. Additionally, the work draws on metrics from the fields of image generation and robotics to provide a novel evaluation framework. The approach is integrated into an end-to-end robotics stack to demonstrate its usability.

Image Generation Local Path Planning Deep Learning U-Net Imitation Learning

Proposed Method

SKIPP frames path planning as an imitation learning problem. Instead of relying on complex reward engineering, users draw the desired navigation behavior directly on a 2D map. These sketches are post-processed and used to train a lightweight U-Net model through Behavioral Cloning. The trained model is integrated within NVIDIA's ISAAC SDK for real-time end-to-end navigation.

Input Encoding

Each input instance is a multi-channel image encoding:

  • Occupancy grid — obstacles in white, navigable areas in grey, unobserved in black
  • Goal position — overlayed as a blue rectangle
  • Robot position — encoded as a red rectangle
  • Expert path — hand-drawn sketch processed and overlayed in green
Input Encoding

Input encoding: occupancy grid with start, goal, and expert path overlayed.

Model Architecture

The network uses a U-Net architecture with an encoder-decoder structure and skip connections. The encoder progressively downsamples the input while capturing contextual features. The decoder restores fine-grained spatial details. A Sigmoid activation produces a binary mask representing the planned path. The model is trained with Binary Cross Entropy loss.

SKIPP Model Architecture

SKIPP Model Architecture — encoder-decoder with skip connections.

Expert Behaviors

The dataset is divided into two behaviors commonly demanded in industrial AMR use cases. The annotation tool allows experts to intuitively draw desired navigation paths which are then post-processed (thinning, DBSCAN clustering, spline smoothing) into actionable trajectories.

L-shape behavior

L-shape behavior

U-shape behavior

U-shape behavior

Results

Monte Carlo Cross-Validation with k=50 random seeds was used to evaluate stability. The model was trained for 160 epochs with batch size 32 using Adam optimizer on an NVIDIA Quadro RTX 8000.

Behavior FID ↓ Mean-APE (m) ↓ RMSE-ATE (m) ↓ Inference (ms) ↓
mean std mean std mean std mean std
L-shape 50.25 7.11 0.050 0.014 0.055 0.014 5.28 0.09
U-shape 44.00 7.00 0.053 0.015 0.059 0.016 5.32 0.10
47.13
Avg. FID Score
High similarity to expert paths
0.051m
Mean APE
Positional accuracy
0.055m
RMSE-ATE
Trajectory deviation
5.3ms
Inference Time
Real-time capable

FID Quality Comparison

High FID (poor quality)
FID 254 — Poor
Low FID (good quality)
FID 47 — Good

Trajectory Accuracy (APE Analysis)

Color-coded trajectory error along the navigation path compared to expert data (dashed lines). Both examples show very low mean APE values with minimal standard deviation, indicating robust and accurate path generation.

L-shape APE

L-shape trajectory — APE 0.017m

U-shape APE

U-shape trajectory — APE 0.007m

Generation Examples

Generation example 1
Generation example 2

Resources

GitHub Repository

Source code, training scripts, and NVIDIA ISAAC integration

 🤗

Expert Dataset

2000 instances of L-shape and U-shape expert demonstrations

 🧠

SKIPP U-Shape Model

Pretrained weights for U-shape behavior

 🧠

SKIPP L-Shape Model

Pretrained weights for L-shape behavior

Citation

@INPROCEEDINGS{10924509,
  author={Rizk, Anthony and Abi Hana, Charbel and Bakouny, Youssef and Khatounian, Flavia},
  booktitle={2024 IEEE International Conference on Mechatronics and Automation (ICMA)}, 
  title={End-to-end Sketch-Guided Path Planning through Imitation Learning for Autonomous Mobile Robots}, 
  year={2024},
  pages={777-782},
  doi={10.1109/ICMA61710.2024.10924509}
}