Manuel Baum

Robots need to be able to understand and manipulate kinematic structures such as windows, door or drawers. We can draw inspiration from animals such as Goffin's cockatoos to teach robots these skills. Although these cockatoos certainly did not evolve to solve kinematic puzzles, they show remarkable success in such tasks. We want to find out how this is possible and how we can equip robots with similarly robust manipulation skills.

Actively seeking for information, exploring the environment and thereby acquiring a model of the environment is a crucial aspect of intelligent behavior. Such behavior is also described in terms of curiosity, or the intrinsic motivation to learn about the environment. The goal of this project was to develop methods to realize such behavior concretely in the context of a physical world, where a robot needs to physically explore and interact with its environment so as to uncover its physical and kinematic structure.

The parrobots project was a seed funded project that we used to bootstrap our research and application for the project "Intelligent Kinematic Problem Solving". In this project we started our interdisciplinary cooperation to find out how Goffin's cockatoos can learn to solve mechanical puzzles. To this end we developed a novel experimental setup, the called the Modular Lockbox. It allows to set up new kinematic puzzles in a short time-frame, to quickly perform new experiments.

Amelie Froessl, June 2021

How can robots reliably estimate the state of mechanical objects around them? While visual estimation offers a way to precisely estimate the state of mechanisms such as drawers or doors, visual estimation also has its shortcomings. Occlusions or bad lighting conditions make it challenging or even impossible to tackle this problem just using vision. In this thesis we explore how audio can be used as a sensor modality that augments or even replaces visual estimation in settings that are challenging to vision.

Vito Mengers, July 2021

This thesis presents a method for estimating objectness in a visual scene by fusing information from motion and appearance. Two interconnected recursive estimators estimate objectness in a way tailored to kinematic structure estimation. The method shows improved objectness estimation and improvement in estimated kinematic joints. Further analysis provides insight into the connection between objectness and kinematic joints as well interconnected recursive estimation.

Lukas Schattenhofer, 2020

In collaboration with colleagues from Vienna, researchers are investigating how cockatoos can solve multi-step kinematic puzzles by building models of their behavior, with the goal of improving robots' abilities to explore their environments. The researchers will search for models in the literature and compare them to real bird data to develop a plausible set of hypotheses that could explain the behavior. The thesis developed a taxonomy of models to understand the landscape of potential models in this domain. This work may provide insights into strategies for robots to explore and understand previously unseen kinematic structures in their environment.

Philipp Braunhart, July 2017

How can a robot explore complex kinematic chains? How can it learn about the kinematic dependencies in such a chain? In this work we developed and tested rule based heuristics and more sophisticated planning methods to explore and manipulate complex kinematic mechanisms in a simulation. In experiments with different, automatically generated lockboxes we evaluated their performance.

Aleksander Gloukhman

It is very useful if robots know the pose of objects not only when it sees them lying on a table, but also while these objects are grasped. But while objects are grasped, their pose often cannot easily be perceived visually. Either because the hand itself obstructs the view, or because the task requires visual attention elsewhere. Thus we suggest to estimate the in-hand pose of objects using acoustic sensing. This is a novel sensing technique that enables contact estimation.