Robotics and Biology Laboratory

Ph.D Theses

Leveraging Novel Information for Coarse-Grained Prediction of Protein Motion

Ines Putz, 2018

Proteins are involved in almost all functions in our cells due to their ability to combine conformational motion with chemical specificity. Hence, information about the motions of a protein provides insights into its function. Proteins move on a rugged energy landscape with many local minima, which is imposed on their high-dimensional conformational space. Exhaustive sampling of this space exceeds the available computational resources for all but the smallest proteins. Computational approaches thus have to simplify the potential energy function and/or resolution of the model using information about what is relevant and what can be ignored. The accuracy of the approximation depends on the accuracy of the used information. Information that is specific to the problem domain, i.e. protein motion in our case, usually results in better models.

This thesis proposes a novel elastic network model of learned maintained contacts, lmcENM. It expands the range of motions that can be captured by such simplified models by leveraging novel information about a protein’s structure. This improves the general applicability of elastic network models.

© RBO

Robot grasping by exploiting compliance and environmental constraints

Clemens Eppner, 2018

Grasping is a crucial skill for any autonomous system that needs to alter the physical world. The complexity of robot grasping stems from the fact that any solution comprises various components: Hand design, control, perception, and planning all affect the success of a grasp. Apart from picking solutions in well-defined industrial scenarios, general grasping in unstructured environment is still an open problem.

© RBO

Learning Robotic Perception Through Prior Knowledge

Rico Jonschkowski, 2018

Intelligent robots must be able to learn; they must be able to adapt their behavior based on experience. But generalization from past experience is only possible based on assumptions or prior knowledge (priors for short) about how the world works.

Interactive Perception © RBO

Leveraging problem structure in interactive perception for robot manipulation of constrained mechanisms

Roberto Martín-Martín, 2018

In this thesis we study robot perception to support a specific type of manipulation task in unstructured environments, the mechanical manipulation of kinematic degrees of freedom. In these tasks the goal of the robot is to create controlled motion, i.e. to change configuration of the kinematic degrees of freedom (DoF) of the objects in the environment.

© RBO

Soft Hands For Compliant Grasping

Raphael Deimel, 2017

Raphael Deimel's thesis reconsiders hand design from the perspective of providing first and foremost robust and reliable grasping, instead of precise control of posture and simple mechanical modelabilty. This results in a fundamentally different manipulator hardware, so called soft hands, that are made out of rubber and fibers which make them highly adaptable. His thesis covers not only hand designs, but also provides an elaborate collection of methods to design, simulate and rapidly prototype soft robots, referred to as the "PneuFlex toolkit".

© S. Höfer

On Decomposability in Robot Reinforcement Learning

Sebastian Höfer, 2017

Reinforcement learning is a computational framework that enables machines to learn from trial-and-error interaction with the environment. In recent years, reinforcement learning has been successfully applied to a wide variety of problem domains, including robotics. However, the success of the reinforcement learning applications in robotics relies on a variety of assumptions, such as the availability of large amounts of training data, highly accurate models of the robot and the environment as well as prior knowledge about the task.

Leveraging Novel Information Sources for Protein Structure Prediction

Michael Bohlke-Schneider, 2015

Three-dimensional protein structures are an invaluable stepping stone towards the understanding of cellular processes. Not surprisingly, state-of-the-art structure prediction methods heavily rely on information. This thesis aims to leverage new information sources: Physicochemical information encoded in predicted structure models and experimental data from high-density cross-linking / mass spectrometry (CLMS) experiments. We demonstrate that these information sources allow improved structure prediction and the reconstruction of human serum albumin domain structures from experimental data collected in its native environment, human blood serum.

© RBO

Multimodal human computer interaction in virtual realities based on an exoskeleton

Ingo Kossyk, 2012

The key features of this system are a high degree of immersion into the computer generated virtual environment and a large working volume. The high degree of immersion will be achieved by multimodal human-exoskeleton interaction based on haptic effects, audio and three- dimensional visualization. The large working volume will be achieved by a lightweight wearable construction that can be carried on the back of the user.

Interactive Perception of Articulated Objects for Autonomous Manipulation

Dov Katz, 2011

This thesis develops robotic skills for manipulating novel articulated objects. The degrees of freedom of an articulated object describe the relationship among its rigid bodies, and are often relevant to the object's intended function. Examples of everyday articulated objects include scissors, pliers, doors, door handles, books, and drawers. Autonomous manipulation of
articulated objects is therefore a prerequisite for many robotic applications in our everyday environments.

Adaptive Balancing of Exploitation With Exploration to Improve Protein Structure Prediction

TJ Brunette, 2011

The most significant impediment for protein structure prediction is the inadequacy of conformation space search. Conformation space is too large and the energy landscape too rugged for existing search methods to consistently find near-optimal minima. Conformation space search methods thus have to focus exploration on a small fraction of the search space. The ability to choose appropriate regions, i.e. regions that are highly likely to contain the native state, critically impacts the effectiveness of search. To make the choice of where to explore requires information, with higher quality information resulting in better choices. Most current search methods are designed to work in as many domains as possible, which leads to less accurate information because of the need for generality. However, most domains provide unique, and accurate information. To best utilize domain specific information search needs to be customized for each domain. The first contribution of this thesis customizes search for protein structure prediction, resulting in significantly more accurate protein structure predictions.

Exploiting Structure: A Guided Approach To Sampling-Based Robot Motion Planning

Brendan Burns, 2007

This thesis proposes a new utility-guided framework for motion planning that can reliably compute collision-free motions with the efficiency required for real-world planning. The utility-guided approach begins with the observation there is regularity in space of possible motions available to a robot. Further, certain motions are more crucial than others for computing collision free paths. Together these observations
form structure in the robot’s space of possible movements.

Diploma / Master Theses

Vision-Based Teleoperation of the Compliant RBO Hand 3

Using an off-the-shelf computer vision tool, a multi-camera setup tracks the user’s human hand and wrist postures, and estimates the 3D hand pose employing an underlying kinematic model to exploit the knowledge about the human hand. Changing into joint space, the human joint angles are mapped to the RBO Hand 3, first in simulation, then on the robotic hand.

Estimating Objectness From Motion and Appearance

Vito Mengers, July 2021

In this thesis, we propose a method to estimate objectness based on crossmodal fusion of information from motion and appearance. Objectness is a property that describes how we can group parts of a visual scene into objects.

Modelling and Understanding Cockatoo's Mechanical Problem Solving Behavior

Lukas Schattenhofer, 2020

Our overarching goal is to build a model that can solve unknown physical puzzles as good cockatoos. For this we need to know what models exist, which of those are relevant and which should be applied right away in simulation. This thesis creates a taxonomy to provide guidelines for selecting and build models suitable to explain the behavioural data of three cockatoos trying to solve the physical puzzle, a lockbox

Active Acoustic Sensing

This thesis develops and characterizes a contact sensor by playing back a known sound and analyzing how it changes when a PneuFlex actuator touches an object in different ways.

Experimental Validation of Contact Dynamics for Prehensile Pushing

Most models in contact dynamics show some unrealistic behavior due to assumptions that were made for the sake of computational convenience. Unfortunately, there is a lack of experimental work to validate these assumptions and to evaluate how realistic these contact modeling approaches are, which is the purpose of this thesis.

Entropy as an Organizing Principle for Selection in Evolutionary Robotics

In evolutionary computation, a goal-based objective function is typically unable to include the local challenges on the way towards its fulfillment and tends to to cause the search to converge prematurely. Therefore, this work proposes to use objectives that are defined by different aspects of an individuals interaction with the environment and a selection procedure able to reallocate search efforts in order to avoid convergence.

Force-Controlled Action Primitives for Interactive Perception

Interactive Perception exploits the robot capabilities to interact with the environment to reveal hidden properties, like the kinematic structures of articulated objects. However, when the robot faces a new environment, it needs to decide on how to interact to maximize the information gain based on sensor data, and use compliant controllers that allow the articulation to guide the motion.

© RBO

Increasing the Stifness of a Pneumatic Actuator with Granular and Layer Jamming

The flexibility and compliance of Soft Robotic Actuators like the PneuFlex by Raphael Deimel offer several advantages over traditional, rigid mechanisms. They are inherently safe and light, robust to impact and collision, have a high degree of compliance without the need for explicit control, and can be designed and build quickly at low costs.

There are situations, however, in which softness and compliance become a disadvantage. The softness of the PneuFlex actuator (or any other soft actuator) limits the amount of force it can exert onto the environment, for example, when lifting a heavy object or when pressing a switch.

To alleviate this limitation we propose soft actuators capable of changing their stiffness by employing jamming.

Grasping using Visual Feedback (Georg Bartels)

This thesis aims at developing the perceptual strategy of the experimenter, i.e. interactive exploration and perception of the object, decision whether it is graspable, selection of the most promising pregrasp configuration of the robot hand, and positioning of the hand with respect to the object.

3D Perception for Grasping (Stefan Schrandt)

Our grasping approach compared with most current ones does not rely on a perfect model. Additionally, the computational cost for the algorithm is signi cantly lower as planning is no longer necessary. The compliance of the hand is used, hence a coarse model that does not need to be perfect is sufficient to compute grasps successfully.

Bachelor Theses

Convolutional Neural Networks for the prediction of breaking contacts during protein motion

The thesis builds on the work from Ines Putz, applying deep neural networks from the contact prediction field to breaking contact prediction during protein motion. Elastic Network Models are useful in determining the coarse-grained motion of proteins. During structural transitions, certain residue pairs that were spatially close become separated, so called breaking contacts. Incorporating this information improves the prediction of protein motion. In this thesis, we want to apply state of the art machine learning methods to breaking contact prediction.

Identification of Beneficial Morphological Computation on Soft Hands

Marlon Kupfer, March 2019

Compliance in soft hands can be both beneficial and detrimental to functionality. Although recent work has shown the benefits of compliance to object and environment geometry, there is little work in identifying and avoiding the negative aspects of compliance while controlling soft hands. However, a planner or a feedback-controller that exploits compliance should avoid the regions of detrimental morphological computation and guide the interactions to the favorable ones.

Luckily recent work in simulation has shown promising results in differentiating between beneficial/detrimental morphological computations. The challenge ahead is to whether these results can be transferred to real systems. Our lab's work in hand sensorization is a possible tool in this path.

EET-Based Motion Planning Applied to Protein-Ligand Interactions

Friederike Fischer, April 2017

For drug design it is essential to know which ligands can reach the active site of a protein. These ligands are potential candidates that inhibit or activate the given protein, and thereby cure a disease. We will show how to use sampling-based motion planning to solve protein-ligand disassembly problems.

Characterizing PneuFlex Actuator Deformations Using Liquid Metal Strain Sensors

Soft, compliant robot hands passively adapt their shape to excel in grasping tasks. But their shape can also give insight into grasp quality, object properties and their environment. Recovering this shape is difficult because their soft material exhibits complex deformations when subjected to external force.

This thesis aims to enable limited shape sensing of the PneuFlex actuator used for the soft RBO Hand 2. Complex actuator shapes are identified as combinations of simple deformations that are shown to be predictable from strain sensor readings using machine learning. A layout of strain sensors is created from strain estimations during these deformations and optimized to produce a reduced layout of sensors.

This optimized layout is then shown to still accurately predict these deformations, giving insight into the shape of the PneuFlex actuator.

Evaluation of Three Sensor Technologies for Use in Soft Robot Fingers

Stefan Schirmeister, March 2016

Sensing for soft continuum actuators as a necessary technology has emerged recently with the development of so called soft hands, which exploit the high deformability of soft structures and materials. Unfortunately, soft, stretchable sensors capable of withstanding a stretch of 100% are commercially not available. At the same time their tight integration into actuators is required to address the specific challenges of continuously deforming actuators. The thesis evaluates three potential sensor technologies for their suitability in soft hands. The thesis investigates their robustness, ease of use, long term stability and responsiveness with respect to the intended application in soft hands.

Effects of model complexity on speed and accuracy of visual servoing for manipulation

Georg Hieronimus, November 2015

This work approaches the Image Based Visual Servoing (IBVS) from three different technical aspects in order to improve the Speed and Accuracy of robotics grasping. Accuracy increase when tracking kinematic chains through the extension of the particle filter, speed up through GPU processing and strategic utilization of encoders in order to put the tracker on important parts of the input image.

A Practical Guide to Transformed Predictive State Representations

Niklas Gebauer, September 2015

Predictive state representations (PSRs) are gaining a lot of attention in the robotics community lately because, in theory, they promise a powerful model that might be learned directly from data. But the practical application of PSRs remains a difficult procedure. There have only been a few learning algorithms proposed so far and only a small number of successful attempts where PSRs of complex domains were learned are reported. In this practical guide we aim to ease and encourage practical work with PSRs. On the one hand we provide the theoretical background and practical instructions to PSRs and on the other hand we identify possible questions that ought to be investigated to improve their practical applicability. To this end, we have re-implemented an algorithm that learns a PSR of a simulated mobile robot environment. We guide from the theory that is needed to understand the implemented algorithm to practice and provide in-depth information on all parts of our implementation. In a line of experiments we do not only validate former results that the learned PSRs are accurate enough to enable successful reinforcement learning, but further investigate the quality of learned models and the empirical performance of the algorithm itself. Therefore we apply the learning algorithm to environments of different complexity and examine the practical limits of the implemented approach. One of the main challenges we faced was the tuning of parameters. We found that tedious, environment specific fine tuning is needed to reliably learn accurate representations and thus investigate the influence of parameters on the quality of learned representations in several experiments in greater detail. The results are guidance for future work as well as they show possible problems that need to be tackled in order to improve PSR learning and make it applicable to complex real world domains.

Position-Based Servoing via Probabilistic Part-Based Object Models

Manuel T. Wöllhaf, January 2015

To extend the field of application of robots in unstructured environments it is necessary to develop new techniques of environment perception and interpretation. These methods must give machines the capability to generate sufficient information, which enables them to fulfill their tasks with the aid of their sensors.

© RBO

Three dimensional Joint Detection

Andreas Orthey, December 2010

Kinematic structures are currently one of the most important prerequisites for advanced robot technology.
Grasping objects, painting cars or performing household tasks are just a few examples from the manifold space of possible applications.