Auto3DSG

Auto3DSG: Autoencoding for 3D Scene Graph Learning
via Object-Level Scene Reconstructions

ICCV 2023 - SG2RL Workshop

Sebastian Koch1,2,3 *        Pedro Hermosilla4       Narunas Vaskevicius1,2
Mirco Colosi2        Timo Ropinski3
1Bosch Center for Artificial Intelligence   2Robert Bosch Corporate Research   3University of Ulm  
4Vienna University of Technology  
Paper Poster Talk

News 📰

>> We released a full paper version on arxiv here <<



TL;DR: We present Auto3DSG, a novel pre-training approach for 3D scene graphs usning object-level scene reconstructions.

Abstract

3D scene graphs are an emerging representation for 3D scene understanding, combining geometric and semantic information. However, fully supervised learning of 3D semantic scene graphs is challenging due to the need for object-level annotations and especially relationship labels. Self-supervised pre-training methods have improved performance in 3D scene understanding but have received little attention in 3D scene graph prediction. To this end, we propose Auto3DSG, an autoencoder-based pre-training method for 3D semantic scene graph prediction. By reconstructing the 3D input scene from a graph bottleneck, we reduce the need for object relationship labels and can leverage large-scale 3D scene understanding datasets. Our method outperforms baseline models on the main 3D semantic scene graph benchmark and achieves competitive results with only 5% labeled data during fine-tuning.

Video 🎬

Method Overview

3D Scene Graph Predictions

3D Scene Reconstructions