Robotics and Biology Laboratory

Manuel Baum

Office MAR 5-1
Building MAR
Room MAR 5.065

Research Interests

I am interested in interactive perception and task-directed exploration, two related and deeply robotic problems.

Interactive perception is important, as not all information that is relevant to an agent is readily available just from looking at the world. The agent needs to exert forces, interact with the world to reveal what's relevant, e.g. the weight of an object, or the degrees of freedom of a kinematic structure. Furthermore, as the agent knows which actions it performed to generate sensor data, it can make use of that information to interpret its input.

The world is complex, but robots are usually employed to solve a set of tasks for which they just need to know about a subset of the world. This is why it is important not to explore the environment randomly, but to perform task-directed exploration. But how to find out which information is actually relevant to a task? And how can we gather that information? I aim to answer these questions in my research.

Short CV

2015 - present: PhD Student at RBO & SCIoI

2011 - 2015: MSc Intelligent Systems at Bielefeld University

2008 - 2011: BSc Cognitive Informatics at Bielefeld University


Intelligent Kinematic Problem Solving

Robots need to be able to understand and manipulate kinematic structures such as windows, door or drawers. We can draw inspiration from animals such as Goffin's cockatoos to teach robots these skills. Although these cockatoos certainly did not evolve to solve kinematic puzzles, they show remarkable success in such tasks. We want to find out how this is possible and how we can equip robots with similarly robust manipulation skills.

The Physical Exploration Challenge

Actively seeking for information, exploring the environment and thereby acquiring a model of the environment is a crucial aspect of intelligent behavior. Such behavior is also described in terms of curiosity, or the intrinsic motivation to learn about the environment. The goal of this project was to develop methods to realize such behavior concretely in the context of a physical world, where a robot needs to physically explore and interact with its environment so as to uncover its physical and kinematic structure.


The parrobots project was a seed funded project that we used to bootstrap our research and application for the project "Intelligent Kinematic Problem Solving". In this project we started our interdisciplinary cooperation to find out how Goffin's cockatoos can learn to solve mechanical puzzles. To this end we developed a novel experimental setup, the called the Modular Lockbox. It allows to set up new kinematic puzzles in a short time-frame, to quickly perform new experiments.

Supervised Theses

Estimating Objectness From Motion and Appearance

Vito Mengers, July 2021

This thesis presents a method for estimating objectness in a visual scene by fusing information from motion and appearance. Two interconnected recursive estimators estimate objectness in a way tailored to kinematic structure estimation. The method shows improved objectness estimation and improvement in estimated kinematic joints. Further analysis provides insight into the connection between objectness and kinematic joints as well interconnected recursive estimation.

Reducing Uncertainty in Kinematic Model Estimation by Fusing Audition and Vision

Amelie Froessl, June 2021

How can robots reliably estimate the state of mechanical objects around them? While visual estimation offers a way to precisely estimate the state of mechanisms such as drawers or doors, visual estimation also has its shortcomings. Occlusions or bad lighting conditions make it challenging or even impossible to tackle this problem just using vision. In this thesis we explore how audio can be used as a sensor modality that augments or even replaces visual estimation in settings that are challenging to vision.

Modelling and Understanding Cockatoo's Mechanical Problem Solving Behavior

Lukas Schattenhofer, 2020

In collaboration with colleagues from Vienna, researchers are investigating how cockatoos can solve multi-step kinematic puzzles by building models of their behavior, with the goal of improving robots' abilities to explore their environments. The researchers will search for models in the literature and compare them to real bird data to develop a plausible set of hypotheses that could explain the behavior. The thesis developed a taxonomy of models to understand the landscape of potential models in this domain. This work may provide insights into strategies for robots to explore and understand previously unseen kinematic structures in their environment.

Comparing Heuristics and Planners for Solving Simulated Lockboxes

Philipp Braunhart, July 2017

How can a robot explore complex kinematic chains? How can it learn about the kinematic dependencies in such a chain? In this work we developed and tested rule based heuristics and more sophisticated planning methods to explore and manipulate complex kinematic mechanisms in a simulation. In experiments with different, automatically generated lockboxes we evaluated their performance.


Estimating the In-Hand Pose of Objects Using Active Acoustic Sensing and Bayesian Filtering

Aleksander Gloukhman

It is very useful if robots know the pose of objects not only when it sees them lying on a table, but also while these objects are grasped. But while objects are grasped, their pose often cannot easily be perceived visually. Either because the hand itself obstructs the view, or because the task requires visual attention elsewhere. Thus we suggest to estimate the in-hand pose of objects using acoustic sensing. This is a novel sensing technique that enables contact estimation.



Li, Xing; Baum, Manuel; Brock, Oliver
Augmentation Enables One-Shot Generalization In Learning From Demonstration for Contact-Rich Manipulation
Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Mengers, Vito; Battaje, Aravind; Baum, Manuel; Brock, Oliver
Combining Motion and Appearance for Robust Probabilistic Object Segmentation in Real Time
2023 IEEE International Conference on Robotics and Automation (ICRA), Page 683--689
Baum, Manuel; Froessl, Amelie; Battaje, Aravind; Brock, Oliver
Estimating the Motion of Drawers From Sound
2023 International Conference on Robotics and Automation (ICRA)


Baum, Manuel; Brock, Oliver
"The World Is Its Own Best Model": Robust Real-World Manipulation Through Online Behavior Selection
Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)
Baum, Manuel; Schattenhofer, Lukas; R"ossler, Theresa; Osuna-Mascaró, Antonio; Auersperg, Alice; Kacelnik, Alex; Brock, Oliver
Yoking-Based Identification of Learning Behavior in Artificial and Biological Agents
Proceedings of the International Conference on the Simulation of Adaptive Behavior 2022 -From Animals to Animats 16, Page 67–78
Publisher: Springer International Publishing, Cham


Baum, Manuel; Brock, Oliver
Achieving Robustness in a Drawer Manipulation Task by using High-level Feedback instead of Planning
Proceedings of the DGR Days, Page 29-29
DGR Days


Baum, Manuel; Brock, Oliver
Achieving Robustness by Optimizing Failure Behavior
Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Page 5806-5811
Baum, Manuel; Bernstein, Matthew; Martín-Martín, Roberto; Höfer, Sebastian; Kulick, Johannes; Toussaint, Marc; Kacelnik, Alex; Brock, Oliver
Opening a Lockbox through Physical Exploration
Proceedings of the IEEE International Conference on Humanoid Robots (Humanoids)


Baum, Manuel; Meier, Martin; Schilling, Malte
Population based Mean of Multiple Computations networks: A building block for kinematic models
2015 International Joint Conference on Neural Networks (IJCNN), Page 1–8


Baum, Manuel
Modeling kinematics of a redundant manipulator using population coding and the MMC principle
Bielefeld University