Our Research

We investigate robots that can understand their environment semantically and geometrically, in order to perform manipulation and other safety critical tasks in proximity to humans. This encompasses semantic understanding under open-set conditions, map representations of the environment, active perception and planning, as well as adaptation and continual self-supervised learning.

Loop Closure From Two Views: Revisiting PGO for Scalable Trajectory Estimation Through Monocular Priors

T Yi Lim, B Sun, M Pollefeys, H Blum

IEEE Robotics and Automation Letters 2026

doi arxiv PDF

FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning

Z Fu, R Zurbrügg, K Qu, M Pollefeys, M Hutter, H Blum, Z Bauer

CVPR 2026

openaccess.thecvf.com arxiv PDF website

Hoi!-A Multimodal Dataset for Force-Grounded, Cross-View Articulated Manipulation

T Engelbracht, R Zurbrügg, M Wohlrapp, M Büchner, A Valada, M Pollefeys, H Blum, Z Bauer

CVPR 2026

arxiv PDF website openaccess.thecvf.com

OpenFrontier: General Navigation with Visual-Language Grounded Frontiers

E Padilla-Cerdio, B Sun, M Pollefeys, H Blum

2026

arxiv PDF

Articulated 3D Scene Graphs for Open-World Mobile Manipulation

M Büchner, A Röfer, T Engelbracht, T Welschehold, Z Bauer, H Blum, ...

2026

arxiv PDF

osmAG-LLM: Zero-Shot Open-Vocabulary Object Navigation via Semantic Maps and Large Language Models Reasoning

F Xie, S Schwertfeger, H Blum

IEEE Robotics and Automation Letters 2026

doi arxiv PDF code

Open-Vocabulary and Semantic-Aware Reasoning for Search and Retrieval of Objects in Dynamic and Concealed Spaces

R Menon, Y Schmiede, M Bennewitz, H Blum

Perception and Planning for Mobile Manipulation in Changing Environments …, 2025

SNI-SLAM++: Tightly-Coupled Semantic Neural Implicit SLAM

S Zhu, G Wang, H Blum, Z Wang, G Zhang, D Cremers, M Pollefeys, H Wang

T-PAMI 2025

doi website code

3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection

Y Yang, L Piccinelli, M Segu, S Li, R Huang, Y Fu, M Pollefeys, H Blum, Z Bauer

ICCV 2025

arxiv PDF website code

ActLoc: Learning to Localize on the Move via Active Viewpoint Selection

B Sun, L Di Giammarino, H Blum, M Pollefeys

CoRL 2025

arxiv PDF website

FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction

D Rotondi, F Scaparro, H Blum, K O. Arras

IROS 2025

arxiv PDF website

SpotLight: Robotic Scene Understanding through Interaction and Affordance Detection

T Engelbracht, R Zurbrügg, M Pollefeys, H Blum, Z Bauer

HUMANOIDS 2025

arxiv PDF website

FrontierNet: Learning Visual Cues to Explore

B Sun, H Chen, S Leutenegger, C Cadena, M Pollefeys, H Blum

IEEE Robotics and Automation Letters 2025

doi website arxiv PDF code

CroCoDL: Cross-device Collaborative Dataset for Localization

H Blum, A Mercurio, J O’Reilly, T Engelbracht, M Dusmanu, M Pollefeys, Z Bauer

CVPR 2025

doi localizoo.com

ARKit LabelMaker: A New Scale for Indoor 3D Scene Understanding

G Ji, S Weder, F Engelmann, M Pollefeys, H Blum

CVPR 2025

arxiv PDF labelmaker.org code

DepthSplat: Connecting Gaussian Splatting and Depth

H Xu, S Peng, F Wang, H Blum, D Barath, A Geiger, M Pollefeys

CVPR 2025

arxiv PDF website code

Spot-On: A Mixed Reality Interface for Multi-Robot Cooperation

T Engelbracht, P Lukovic, T Behrens, K Lascheit, R Zurbrügg, M Pollefeys, ...

2025

arxiv PDF

Lost & Found: Tracking Changes from Egocentric Observations in 3D Dynamic Scene Graphs

T Behrens, R Zurbrügg, M Pollefeys, Z Bauer, H Blum

Robotics and Automation Letters (RA-L) 2025

arxiv PDF website doi

HoloSpot: Intuitive Object Manipulation via Mixed Reality Drag-and-Drop

P Soler Garcia, P Lukovic, L Reynaud, A Sgobbi, F Bruni, M Brun, M Zünd, R Bollati, M Pollefeys, H Blum, Z Bauer

2024

arxiv PDF website

Learning Where to Look: Self-supervised Viewpoint Selection for Active Localization Using Geometrical Information

L Di Giammarino, B Sun, G Grisetti, M Pollefeys, H Blum, D Barath

ECCV 2024

doi arxiv PDF code

“Where am I?” Scene Retrieval with Language

J Chen, D Barath, I Armeni, M Pollefeys, H Blum

ECCV 2024

doi arxiv PDF website

OpenDAS: Open-Vocabulary Domain Adaptation for Segmentation

G Yilmaz, S Peng, M Pollefeys, F Engelmann, H Blum

2024

arxiv PDF website

SNI-SLAM: Semantic Neural Implicit SLAM

S Zhu, G Wang, H Blum, J Liu, L Song, M Pollefeys, H Wang

CVPR 2024

arxiv PDF code

A 3D Mixed Reality Interface for Human-Robot Teaming

J Chen, B Sun, M Pollefeys, H Blum

ICRA 2024

arxiv PDF code

Active Visual Localization for Multi-Agent Collaboration: A Data-Driven Approach

M Hanlon, B Sun, M Pollefeys, H Blum

ICRA 2024

arxiv PDF

Spot-Compose: A Framework for Open-Vocabulary Object Retrieval and Drawer Manipulation in Point Clouds

O Lemke, Z Bauer, R Zurbrügg, M Pollefeys, F Engelmann, H Blum

ICRA 2024

arxiv PDF website code

LabelMaker: Automatic Semantic Label Generation from RGB-D Trajectories

S Weder, H Blum, F Engelmann, M Pollefeys

3DV 2024

arxiv PDF labelmaker.org code

Our Research

Loop Closure From Two Views: Revisiting PGO for Scalable Trajectory Estimation Through Monocular Priors

FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning

Hoi!-A Multimodal Dataset for Force-Grounded, Cross-View Articulated Manipulation

OpenFrontier: General Navigation with Visual-Language Grounded Frontiers

Articulated 3D Scene Graphs for Open-World Mobile Manipulation

osmAG-LLM: Zero-Shot Open-Vocabulary Object Navigation via Semantic Maps and Large Language Models Reasoning

Open-Vocabulary and Semantic-Aware Reasoning for Search and Retrieval of Objects in Dynamic and Concealed Spaces

SNI-SLAM++: Tightly-Coupled Semantic Neural Implicit SLAM

3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection

ActLoc: Learning to Localize on the Move via Active Viewpoint Selection

FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction

SpotLight: Robotic Scene Understanding through Interaction and Affordance Detection

FrontierNet: Learning Visual Cues to Explore

CroCoDL: Cross-device Collaborative Dataset for Localization

ARKit LabelMaker: A New Scale for Indoor 3D Scene Understanding

DepthSplat: Connecting Gaussian Splatting and Depth

Spot-On: A Mixed Reality Interface for Multi-Robot Cooperation

Lost & Found: Tracking Changes from Egocentric Observations in 3D Dynamic Scene Graphs

HoloSpot: Intuitive Object Manipulation via Mixed Reality Drag-and-Drop

Learning Where to Look: Self-supervised Viewpoint Selection for Active Localization Using Geometrical Information

“Where am I?” Scene Retrieval with Language

OpenDAS: Open-Vocabulary Domain Adaptation for Segmentation

SNI-SLAM: Semantic Neural Implicit SLAM

A 3D Mixed Reality Interface for Human-Robot Teaming

Active Visual Localization for Multi-Agent Collaboration: A Data-Driven Approach

Spot-Compose: A Framework for Open-Vocabulary Object Retrieval and Drawer Manipulation in Point Clouds

LabelMaker: Automatic Semantic Label Generation from RGB-D Trajectories

Robot Perception and Learning Lab

Lab

Network