OpenSplatGraph: From Dense Semantic Maps to Structured Scene Graphs for Open-Vocabulary Robot Perception

1Queensland University of Technology, 2CSIRO Robotics

Accepted at ACCV 2026

Comparison of dense semantic maps and scene graphs for open-vocabulary 3D scene understanding

TL;DR: OpenSplatGraph builds persistent 3D scene graphs directly from an online open-vocabulary dense map, enabling high-fidelity perception, language grounding, relational reasoning, and downstream robotic tasks within a single system.

Abstract

Dense 3D mapping with semantic understanding is essential for robotic perception in complex environments. Recent 3D Gaussian Splatting-based mapping approaches enable high-fidelity geometry and efficient open-vocabulary perception, but typically represent semantics as unstructured feature fields that limit object-centric reasoning. In contrast, 3D scene graphs explicitly model objects and their relationships for structured reasoning, but are commonly constructed from sparse geometric representations that do not fully exploit dense semantic maps. In this work, we present OpenSplatGraph, a unified framework that constructs persistent 3D scene graphs directly from an online Gaussian-based open-vocabulary semantic map. The proposed framework augments the dense semantic map with a reliability-aware semantic field that maintains lightweight observation statistics for confidence-aware, query-conditioned object extraction. Extracted object instances are associated with persistent graph nodes, allowing object attributes and relationships to be incrementally updated across observations and queries. By tightly coupling dense semantic mapping with persistent object-centric representations, our framework supports both language-guided object grounding and structured relational reasoning while preserving the geometric fidelity of Gaussian-based mapping. Comprehensive evaluations on standard 3D scene understanding benchmarks and real-world robotic experiments demonstrate that OpenSplatGraph achieves competitive performance for online open-vocabulary perception and downstream robotic tasks.

Method

Overview of OpenSplatGraph

An overview of OpenSplatGraph. Starting from a multi-view RGB-D sequence, the framework incrementally constructs an online open-vocabulary dense semantic map with reliability-aware semantic fusion. Extracted object instances are associated with persistent identities and accumulated into a 3D scene graph with closed-set spatial and open-set VLM relations, supporting semantic segmentation, object grounding, and robotic navigation and manipulation.

Experiments

3D Semantic Segmentation

Qualitative results of 3D semantic segmentation on ScanNet and Replica

Qualitative results of 3D semantic segmentation on the ScanNet and Replica datasets. OpenSplatGraph produces spatially coherent object instances with improved boundary alignment under the online open-vocabulary setting.

Open-vocabulary 3D semantic segmentation demos on ScanNet scenes.

Open-Vocabulary 3D Scene Graph

Qualitative visualization of open-vocabulary 3D scene graph construction on Replica

Qualitative visualization of open-vocabulary 3D scene graph construction on the Replica dataset, including closed-set spatial relations and open-set semantic predicates.

Nodes

Edges

Demo clips of persistent object nodes and relational edges in the constructed 3D scene graph. Top-30 nodes and top-50 edges by confidence are shown.

Real-world Robotic Experiments

Real-world robotic navigation experiments with OpenSplatGraph

Real-world robotic navigation experiments with OpenSplatGraph. The framework constructs an online open-vocabulary dense map and supports language-guided navigation in indoor environments.

Online map reconstruction

Robot scanning the scene

Online map construction

Online reconstruction of the open-vocabulary dense semantic map during exploration.

Language-guided navigation

Find a place to sit and play with toys.

Find a red ball in the meeting room.

Find a place to dispose of fruit waste.

Language-guided navigation to queried targets in the reconstructed environment.

Video

Supplementary video for OpenSplatGraph.

Acknowledgements

This work was supported in part by the Australian Research Council Discovery Project under Grant DP250103634, and in part by the Commonwealth Scientific and Industrial Research Organisation (CSIRO). The authors acknowledge continued support from the CSIRO's Embodied AI Cluster.

BibTeX

@inproceedings{nguyen2026opensplatgraph,
  author    = {Nguyen, Binh Long and Nguyen, Kien and Fookes, Clinton and Moghadam, Peyman},
  title     = {OpenSplatGraph: From Dense Semantic Maps to Structured Scene Graphs for Open-Vocabulary Robot Perception},
  booktitle = {Asian Conference on Computer Vision (ACCV)},
  year      = {2026}
}