Our Research

We investigate robots that can understand their environment semantically and geometrically, in order to perform manipulation and other safety critical tasks in proximity to humans. This encompasses semantic understanding under open-set conditions, map representations of the environment, active perception and planning, as well as adaptation and continual self-supervised learning.

Map-Det3D: Metric Feed-Forward 3D

L Piccinelli¹, SR Bulò, D Rozumny, J Schönberger, Z Bauer¹, H Blum, ...

Computer Vision–ECCV 2026: 19th European Conference, Malmö, Sweden …, 2026

LIME: Learning Intent-aware Camera Motion from Egocentric Video

B Sun, J Li, Y Yang, C Zhang, T Engelbracht, S Hong, C Cadena, M Pollefeys, H Blum

CoRL 2026

arxiv PDF website

Symbolizer: Symbolic Model-Free Task Planning With VLMs

S Azirar, Z Ajanovic, H Blum

IEEE Robotics and Automation Letters 2026

doi arxiv PDF

Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputs

Y Yang, L Piccinelli, S Rota Bulò, S Hong, D Rozumny, J Schönberger, Z Bauer, H Blum, P Kontschieder, M Pollefeys

ECCV 2026

doi arxiv PDF

ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human Interaction

T Engelbracht, R Zurbrügg, M Mittal, M Hutter, M Pollefeys, H Blum, ...

2026

arxiv PDF

DINOcular: Self-Supervised Visuospatial Representations

F Almukhamedov, S Azirar, H Blum

2026

website arxiv PDF

OpenFrontier: General Navigation with Visual-Language Grounded Frontiers

E Padilla-Cerdio, B Sun, M Pollefeys, H Blum

RSS 2026

arxiv PDF website

ContactFlow: A video action conditioning that transfers across embodiments

S Azirar, E Pallotta, J Nogga, J Gall, S Behnke, H Blum

2026

arxiv PDF

DVPSFormer: Efficient Online Depth-aware Video Panoptic Segmentation for Autonomous Driving

YH Yang, L Piccinelli, S Li, M Segu, L Ke, M Danelljan, Y Fu, Z Bauer, ...

2026

arxiv PDF

FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning

Z Fu, R Zurbrügg, K Qu, M Pollefeys, M Hutter, H Blum, Z Bauer

CVPR 2026

openaccess.thecvf.com arxiv PDF website

Hoi!-A Multimodal Dataset for Force-Grounded, Cross-View Articulated Manipulation

T Engelbracht, R Zurbrügg, M Wohlrapp, M Büchner, A Valada, M Pollefeys, H Blum, Z Bauer

CVPR 2026

arxiv PDF website

ArtiTwinSplat: Interactable Digital Twin Reconstruction via Gaussian Splatting from RGB-D videos

P Mishra, R Zurbrügg, M Wilder-Smith, M Hutter, M Pollefeys, Z Bauer, ...

2026

arxiv PDF

Loop Closure From Two Views: Revisiting PGO for Scalable Trajectory Estimation Through Monocular Priors

T Yi Lim, B Sun, M Pollefeys, H Blum

IEEE Robotics and Automation Letters 2026

doi arxiv PDF

Articulated 3d scene graphs for open-world mobile manipulation

M Büchner, A Röfer, T Engelbracht, T Welschehold, Z Bauer, H Blum, ...

2026

arxiv PDF

osmAG-LLM: Zero-Shot Open-Vocabulary Object Navigation via Semantic Maps and Large Language Models Reasoning

F Xie, S Schwertfeger, H Blum

IEEE Robotics and Automation Letters 2026

doi arxiv PDF code

SNI-SLAM++: Tightly-Coupled Semantic Neural Implicit SLAM

S Zhu, G Wang, H Blum, Z Wang, G Zhang, D Cremers, M Pollefeys, H Wang

T-PAMI 2025

doi website code

3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection

Y Yang, L Piccinelli, M Segu, S Li, R Huang, Y Fu, M Pollefeys, H Blum, Z Bauer

ICCV 2025

arxiv PDF website code ieeexplore.ieee.org

ActLoc: Learning to Localize on the Move via Active Viewpoint Selection

B Sun, L Di Giammarino, H Blum, M Pollefeys

CoRL 2025

arxiv PDF website

FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction

D Rotondi, F Scaparro, H Blum, K O. Arras

IROS 2025

arxiv PDF website

SpotLight: Robotic Scene Understanding through Interaction and Affordance Detection

T Engelbracht, R Zurbrügg, M Pollefeys, H Blum, Z Bauer

HUMANOIDS 2025

arxiv PDF website

FrontierNet: Learning Visual Cues to Explore

B Sun, H Chen, S Leutenegger, C Cadena, M Pollefeys, H Blum

IEEE Robotics and Automation Letters 2025

doi website arxiv PDF code

CroCoDL: Cross-device Collaborative Dataset for Localization

H Blum, A Mercurio, J O’Reilly, T Engelbracht, M Dusmanu, M Pollefeys, Z Bauer

CVPR 2025

doi localizoo.com

ARKit LabelMaker: A New Scale for Indoor 3D Scene Understanding

G Ji, S Weder, F Engelmann, M Pollefeys, H Blum

CVPR 2025

arxiv PDF labelmaker.org code

DepthSplat: Connecting Gaussian Splatting and Depth

H Xu, S Peng, F Wang, H Blum, D Barath, A Geiger, M Pollefeys

CVPR 2025

arxiv PDF website code ieeexplore.ieee.org

Lost & Found: Tracking Changes from Egocentric Observations in 3D Dynamic Scene Graphs

T Behrens, R Zurbrügg, M Pollefeys, Z Bauer, H Blum

Robotics and Automation Letters (RA-L) 2025

arxiv PDF website doi

HoloSpot: Intuitive Object Manipulation via Mixed Reality Drag-and-Drop

P Soler Garcia, P Lukovic, L Reynaud, A Sgobbi, F Bruni, M Brun, M Zünd, R Bollati, M Pollefeys, H Blum, Z Bauer

2024

arxiv PDF website

Learning Where to Look: Self-supervised Viewpoint Selection for Active Localization Using Geometrical Information

L Di Giammarino, B Sun, G Grisetti, M Pollefeys, H Blum, D Barath

ECCV 2024

doi arxiv PDF code

“Where am I?” Scene Retrieval with Language

J Chen, D Barath, I Armeni, M Pollefeys, H Blum

ECCV 2024

doi arxiv PDF website

OpenDAS: Open-Vocabulary Domain Adaptation for Segmentation

G Yilmaz, S Peng, M Pollefeys, F Engelmann, H Blum

2024

arxiv PDF website

SNI-SLAM: Semantic Neural Implicit SLAM

S Zhu, G Wang, H Blum, J Liu, L Song, M Pollefeys, H Wang

CVPR 2024

arxiv PDF code ieeexplore.ieee.org

A 3D Mixed Reality Interface for Human-Robot Teaming

J Chen, B Sun, M Pollefeys, H Blum

ICRA 2024

arxiv PDF code

Active Visual Localization for Multi-Agent Collaboration: A Data-Driven Approach

M Hanlon, B Sun, M Pollefeys, H Blum

ICRA 2024

arxiv PDF

Spot-Compose: A Framework for Open-Vocabulary Object Retrieval and Drawer Manipulation in Point Clouds

O Lemke, Z Bauer, R Zurbrügg, M Pollefeys, F Engelmann, H Blum

ICRA 2024

arxiv PDF website code

LabelMaker: Automatic Semantic Label Generation from RGB-D Trajectories

S Weder, H Blum, F Engelmann, M Pollefeys

3DV 2024

arxiv PDF labelmaker.org code

Our Research

Map-Det3D: Metric Feed-Forward 3D

LIME: Learning Intent-aware Camera Motion from Egocentric Video

Symbolizer: Symbolic Model-Free Task Planning With VLMs

Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputs

ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human Interaction

DINOcular: Self-Supervised Visuospatial Representations

OpenFrontier: General Navigation with Visual-Language Grounded Frontiers

ContactFlow: A video action conditioning that transfers across embodiments

DVPSFormer: Efficient Online Depth-aware Video Panoptic Segmentation for Autonomous Driving

FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning

Hoi!-A Multimodal Dataset for Force-Grounded, Cross-View Articulated Manipulation

ArtiTwinSplat: Interactable Digital Twin Reconstruction via Gaussian Splatting from RGB-D videos

Loop Closure From Two Views: Revisiting PGO for Scalable Trajectory Estimation Through Monocular Priors

Articulated 3d scene graphs for open-world mobile manipulation

osmAG-LLM: Zero-Shot Open-Vocabulary Object Navigation via Semantic Maps and Large Language Models Reasoning

SNI-SLAM++: Tightly-Coupled Semantic Neural Implicit SLAM

3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection

ActLoc: Learning to Localize on the Move via Active Viewpoint Selection

FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction

SpotLight: Robotic Scene Understanding through Interaction and Affordance Detection

FrontierNet: Learning Visual Cues to Explore

CroCoDL: Cross-device Collaborative Dataset for Localization

ARKit LabelMaker: A New Scale for Indoor 3D Scene Understanding

DepthSplat: Connecting Gaussian Splatting and Depth

Lost & Found: Tracking Changes from Egocentric Observations in 3D Dynamic Scene Graphs

HoloSpot: Intuitive Object Manipulation via Mixed Reality Drag-and-Drop

Learning Where to Look: Self-supervised Viewpoint Selection for Active Localization Using Geometrical Information

“Where am I?” Scene Retrieval with Language

OpenDAS: Open-Vocabulary Domain Adaptation for Segmentation

SNI-SLAM: Semantic Neural Implicit SLAM

A 3D Mixed Reality Interface for Human-Robot Teaming

Active Visual Localization for Multi-Agent Collaboration: A Data-Driven Approach

Spot-Compose: A Framework for Open-Vocabulary Object Retrieval and Drawer Manipulation in Point Clouds

LabelMaker: Automatic Semantic Label Generation from RGB-D Trajectories

Robot Perception and Learning Lab

Lab

Network