Abstract

Teaser image

Self-supervised online traversability estimation enables robots to continuously learn from unlabeled openworld experiences and adapt their navigation behavior toward safe and efficient trajectories. Existing approaches either rely on handcrafted proprioceptive traversability scores, limiting robot-agnosticism, or cluster prior data, preventing online learning. Moreover, many continual learning methods incur substantial memory and computational costs, hindering onboard deployment. We introduce COTRATE, an online learning framework for continuous traversability estimation from multimodal, unlabeled robot experience. Our method first infers robust traversability scores using a robot-agnostic, learning-based online terrain assessment module operating on proprioceptive and inertial signals. These scores then supervise a visual traversability network through a novel alignment loss that associates visual embeddings with online terrain assessments. To mitigate forgetting during continual learning, we propose a diversity-aware feature selection strategy that preserves performance using a compact replay memory. We further show that the learned traversability representation supports knowledge transfer across different robot platforms with different locomotion kinematics. We evaluate COTRATE on a dataset of ≈ 50,000 images collected with two robotic platforms across 11 outdoor terrains, and benchmark it on navigation tasks in three representative outdoor environments.

Technical Approach

Overview of our approach
Figure: COTRATE learns traversability scores from inertial and proprioceptive data and successively uses them to incrementally, train a visual network. The visual network uses selective feature replay to mitigate forgetting of prior terrain knowledge during online learning. At inference, per-image traversability scores are accumulated into a dense 2.5D elevation map for downstream path planning and evaluation.

COTRATE (Continual Online Transferable Robot-Agnostic Traversability Estimation), is a unified framework for visual traversability estimation in outdoor environments. COTRATE learns contin- uous traversability scores from multimodal robot experience and leverages them to supervise a monocular visual traversability network during online operation. At the core of our method is a novel online traversability score generation module that predicts numerical traversability scores using an incrementally trained variational autoencoder. These scores supervise a visual model that incorporates a pretrained visual foundation model with our novel feature-distance alignment loss to correlate visual feature similarities with our learned continuous traversability scores. To enable continual learning, we introduce a diversity-aware feature-replay strategy that retains a compact set of representative features and mitigates forgetting under a minimal replay budget. Finally, we study cross-platform transfer and show that the learned traversability representation can be transferred across robotic embodiments with different locomotion kinematics when their terrain-dependent energy-efficiency ratings are compatible. We evaluate COTRATE on a novel collected outdoor dataset comprising 11 terrain types across two robotic platforms with distinct embodiments. Navigation experiments in three representative environments demonstrate the method’s effectiveness, with systematic ablations quantifying the impact of terrain scoring, visual alignment, feature replay, and cross-platform transfer. The results show that COTRATE enables continuous self-supervised traversability learning while maintaining low memory overhead and supporting transferable terrain understanding across platforms.

Code

A software implementation of this project based on PyTorch can be found in our GitHub repository for academic usage and is released under the GPLv3 license. For any commercial purpose, please contact the authors.

Publications

If you find our work useful, please consider citing our papers:

Julia Hindel, Simon Bultmann, Houman Masnavi, Daniele Cattaneo, and Abhinav Valada
Self-Supervised Online Robot-Agnostic Traversability Estimation for Open-World Environments
arXiv preprint arXiv:2407.18145

(PDF(ArXiv)) (BibTeX)

Authors

Julia Hindel

Julia Hindel

University of Freiburg

Simon Bultmann

Simon Bultmann

University of Freiburg

Houman Masnavi

Houman Masnavi

University of Freiburg

Daniele Cattaneo

Daniele Cattaneo

University of Freiburg

Abhinav Valada

Abhinav Valada

University of Freiburg

Acknowledgment

This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – SFB 1597 – 499552394.