Proteins are involved in almost all functions in our cells due to their ability to combine conformational motion with chemical specificity. Hence, information about the motions of a protein provides insights into its function. Proteins move on a rugged energy landscape with many local minima, which is imposed on their high-dimensional conformational space. Exhaustive sampling of this space exceeds the available computational resources for all but the smallest proteins. Computational approaches thus have to simplify the potential energy function and/or resolution of the model using information about what is relevant and what can be ignored. The accuracy of the approximation depends on the accuracy of the used information. Information that is specific to the problem domain, i.e. protein motion in our case, usually results in better models.
In this thesis, I propose a novel elastic network model of learned maintained contacts, lmcENM. It expands the range of motions that can be captured by such simplified models by leveraging novel information about a protein’s structure. This improves the general applicability of elastic network models.
This thesis contributes to algorithmic approaches for the motion generation problem for mobile manipulators. This problem is unsolved in unstructured environments, where the robot does not have access to precise models but must infer the state of the world with its sensors.
Grasping is a crucial skill for any autonomous system that needs to alter the physical world. The complexity of robot grasping stems from the fact that any solution comprises various components: Hand design, control, perception, and planning all affect the success of a grasp. Apart from picking solutions in well-defined industrial scenarios, general grasping in unstructured environment is still an open problem.
Intelligent robots must be able to learn; they must be able to adapt their behavior based on experience. But generalization from past experience is only possible based on assumptions or prior knowledge (priors for short) about how the world works.
In this thesis we study robot perception to support a specific type of manipulation task in unstructured environments, the mechanical manipulation of kinematic degrees of freedom. In these tasks the goal of the robot is to create controlled motion, i.e. to change configuration of the kinematic degrees of freedom (DoF) of the objects in the environment.
Raphael Deimel's thesis reconsiders hand design from the perspective of providing first and foremost robust and reliable grasping, instead of precise control of posture and simple mechanical modelabilty. This results in a fundamentally different manipulator hardware, so called soft hands, that are made out of rubber and fibers which make them highly adaptable. His thesis covers not only hand designs, but also provides an elaborate collection of methods to design, simulate and rapidly prototype soft robots, referred to as the "PneuFlex toolkit".
Reinforcement learning is a computational framework that enables machines to learn from trial-and-error interaction with the environment. In recent years, reinforcement learning has been successfully applied to a wide variety of problem domains, including robotics. However, the success of the reinforcement learning applications in robotics relies on a variety of assumptions, such as the availability of large amounts of training data, highly accurate models of the robot and the environment as well as prior knowledge about the task.
Three-dimensional protein structures are an invaluable stepping stone towards the understanding of cellular processes. Not surprisingly, state-of-the-art structure prediction methods heavily rely on information. This thesis aims to leverage new information sources: Physicochemical information encoded in predicted structure models and experimental data from high-density cross-linking / mass spectrometry (CLMS) experiments. We demonstrate that these information sources allow improved structure prediction and the reconstruction of human serum albumin domain structures from experimental data collected in its native environment, human blood serum.
The key features of this system are a high degree of immersion into the computer generated virtual environment and a large working volume. The high degree of immersion will be achieved by multimodal human-exoskeleton interaction based on haptic effects, audio and three- dimensional visualization. The large working volume will be achieved by a lightweight wearable construction that can be carried on the back of the user.
This thesis develops robotic skills for manipulating novel articulated objects. The degrees of freedom of an articulated object describe the relationship among its rigid bodies, and are often relevant to the object's intended function. Examples of everyday articulated objects include scissors, pliers, doors, door handles, books, and drawers. Autonomous manipulation of
articulated objects is therefore a prerequisite for many robotic applications in our everyday environments.
The most significant impediment for protein structure prediction is the inadequacy of conformation space search. Conformation space is too large and the energy landscape too rugged for existing search methods to consistently find near-optimal minima. Conformation space search methods thus have to focus exploration on a small fraction of the search space. The ability to choose appropriate regions, i.e. regions that are highly likely to contain the native state, critically impacts the effectiveness of search. To make the choice of where to explore requires information, with higher quality information resulting in better choices. Most current search methods are designed to work in as many domains as possible, which leads to less accurate information because of the need for generality. However, most domains provide unique, and accurate information. To best utilize domain specific information search needs to be customized for each domain. The first contribution of this thesis customizes search for protein structure prediction, resulting in significantly more accurate protein structure predictions.
Computationally efficient motion planning mus avoid exhaustive exploration of high-dimensional configuration spaces by leveraging the structure present in real-world planning problems. We argue that this can be accomplished most effectively by carefully balancing exploration and exploitation.
This thesis proposes a new utility-guided framework for motion planning that can reliably compute collision-free motions with the efficiency required for real-world planning. The utility-guided approach begins with the observation there is regularity in space of possible motions available to a robot. Further, certain motions are more crucial than others for computing collision free paths. Together these observations
form structure in the robot’s space of possible movements.
Using an off-the-shelf computer vision tool, a multi-camera setup tracks the user’s human hand and wrist postures, and estimates the 3D hand pose employing an underlying kinematic model to exploit the knowledge about the human hand. Changing into joint space, the human joint angles are mapped to the RBO Hand 3, first in simulation, then on the robotic hand.
In this thesis, we propose a method to estimate objectness based on crossmodal fusion of information from motion and appearance. Objectness is a property that describes how we can group parts of a visual scene into objects.
The aim of this thesis is to remotely control a soft pneumatically operated robot hand developed by the department with the help of a data glove in order to be able to carry out in-hand manipulations.
Our overarching goal is to build a model that can solve unknown physical puzzles as good cockatoos. For this we need to know what models exist, which of those are relevant and which should be applied right away in simulation. This thesis creates a taxonomy to provide guidelines for selecting and build models suitable to explain the behavioural data of three cockatoos trying to solve the physical puzzle, a lockbox
This thesis develops and characterizes a contact sensor by playing back a known sound and analyzing how it changes when a PneuFlex actuator touches an object in different ways.
We research how a soft robotic hand manipulates a grasped object more reliably when we consider the position of the object in a closed-loop controller.
We extend the idea of robotic priors to work on non Markovian observation spaces. For this, we train a recurrent neural network on trajectories, such that the network learns to encode past information in its hidden state.
Classical robotic grasping approaches employ static behaviors: First the hand is maneuvered to the object, then the fingers are closed, and finally the hand is retracted from the scene with the grasped object. On the other hand, humans execute wrist movements concurrently with the fingers closure.
In this work we developed and tested rule based heuristics and more sophisticated planning methods to explore and manipulate complex kinematic mechanisms in a simulation.
Most models in contact dynamics show some unrealistic behavior due to assumptions that were made for the sake of computational convenience. Unfortunately, there is a lack of experimental work to validate these assumptions and to evaluate how realistic these contact modeling approaches are, which is the purpose of this thesis.
In evolutionary computation, a goal-based objective function is typically unable to include the local challenges on the way towards its fulfillment and tends to to cause the search to converge prematurely. Therefore, this work proposes to use objectives that are defined by different aspects of an individuals interaction with the environment and a selection procedure able to reallocate search efforts in order to avoid convergence.
I propose a novel method for efficient object search in realistic environments. I formalize object search as a probabilistic inference problem over possible object locations represented by spatial relations (e.g., in and on).
Interactive Perception exploits the robot capabilities to interact with the environment to reveal hidden properties, like the kinematic structures of articulated objects. However, when the robot faces a new environment, it needs to decide on how to interact to maximize the information gain based on sensor data, and use compliant controllers that allow the articulation to guide the motion.
The flexibility and compliance of Soft Robotic Actuators like the PneuFlex by Raphael Deimel offer several advantages over traditional, rigid mechanisms. They are inherently safe and light, robust to impact and collision, have a high degree of compliance without the need for explicit control, and can be designed and build quickly at low costs.
There are situations, however, in which softness and compliance become a disadvantage. The softness of the PneuFlex actuator (or any other soft actuator) limits the amount of force it can exert onto the environment, for example, when lifting a heavy object or when pressing a switch.
To alleviate this limitation we propose soft actuators capable of changing their stiffness by employing jamming.
This thesis aims at developing the perceptual strategy of the experimenter, i.e. interactive exploration and perception of the object, decision whether it is graspable, selection of the most promising pregrasp configuration of the robot hand, and positioning of the hand with respect to the object.
Since we knew that the protein’s sequence encodes its structure, researcher have been trying to predict the structures computationally. Challenged by the vastness of the conformational space, researchers have leveraged the similarities present in the Protein Data Base to guide prediction.
The traditional approach to robotic motion planning is based on a sense, plan, execute cycle.
Based on the current sensor information, a planner determines a complete motion trajectory. This motion trajectory is then executed as precisely as possible by a controller.
Our grasping approach compared with most current ones does not rely on a perfect model. Additionally, the computational cost for the algorithm is signi cantly lower as planning is no longer necessary. The compliance of the hand is used, hence a coarse model that does not need to be perfect is sufficient to compute grasps successfully.
Judging from the performance on CASP9 targets, the proposed setup works well for template based modeling targets. We were able to cover 61 targets (15 more than the control) with 100 % near-native building block matches. Both the proposed setup and the control achieved roughly the same number of residues that were covered with near-native matches.
Haptic devices enhance the range of multi-modal interaction in virtual reality environments. With the wearable haptic device, developed at the RBO Lab, this interaction is not limited to a small workspace any longer. The wider range of motion allows for new application scenarios.
A novel approach for the protein loop closure problem inspired from robot modeling is developed using the kinematic chain representation of the loop chain and a motion planning technique.
The thesis builds on the work from Ines Putz, applying deep neural networks from the contact prediction field to breaking contact prediction during protein motion. Elastic Network Models are useful in determining the coarse-grained motion of proteins. During structural transitions, certain residue pairs that were spatially close become separated, so called breaking contacts. Incorporating this information improves the prediction of protein motion. In this thesis, we want to apply state of the art machine learning methods to breaking contact prediction.
Compliance in soft hands can be both beneficial and detrimental to functionality. Although recent work has shown the benefits of compliance to object and environment geometry, there is little work in identifying and avoiding the negative aspects of compliance while controlling soft hands. However, a planner or a feedback-controller that exploits compliance should avoid the regions of detrimental morphological computation and guide the interactions to the favorable ones.
Luckily recent work in simulation has shown promising results in differentiating between beneficial/detrimental morphological computations. The challenge ahead is to whether these results can be transferred to real systems. Our lab's work in hand sensorization is a possible tool in this path.
For drug design it is essential to know which ligands can reach the active site of a protein. These ligands are potential candidates that inhibit or activate the given protein, and thereby cure a disease. We will show how to use sampling-based motion planning to solve protein-ligand disassembly problems.
Soft, compliant robot hands passively adapt their shape to excel in grasping tasks. But their shape can also give insight into grasp quality, object properties and their environment. Recovering this shape is difficult because their soft material exhibits complex deformations when subjected to external force.
This thesis aims to enable limited shape sensing of the PneuFlex actuator used for the soft RBO Hand 2. Complex actuator shapes are identified as combinations of simple deformations that are shown to be predictable from strain sensor readings using machine learning. A layout of strain sensors is created from strain estimations during these deformations and optimized to produce a reduced layout of sensors.
This optimized layout is then shown to still accurately predict these deformations, giving insight into the shape of the PneuFlex actuator.
Sensing for soft continuum actuators as a necessary technology has emerged recently with the development of so called soft hands, which exploit the high deformability of soft structures and materials. Unfortunately, soft, stretchable sensors capable of withstanding a stretch of 100% are commercially not available. At the same time their tight integration into actuators is required to address the specific challenges of continuously deforming actuators. The thesis evaluates three potential sensor technologies for their suitability in soft hands. The thesis investigates their robustness, ease of use, long term stability and responsiveness with respect to the intended application in soft hands.
This work approaches the Image Based Visual Servoing (IBVS) from three different technical aspects in order to improve the Speed and Accuracy of robotics grasping. Accuracy increase when tracking kinematic chains through the extension of the particle filter, speed up through GPU processing and strategic utilization of encoders in order to put the tracker on important parts of the input image.
Predictive state representations (PSRs) are gaining a lot of attention in the robotics community lately because, in theory, they promise a powerful model that might be learned directly from data. But the practical application of PSRs remains a difficult procedure. There have only been a few learning algorithms proposed so far and only a small number of successful attempts where PSRs of complex domains were learned are reported. In this practical guide we aim to ease and encourage practical work with PSRs. On the one hand we provide the theoretical background and practical instructions to PSRs and on the other hand we identify possible questions that ought to be investigated to improve their practical applicability. To this end, we have re-implemented an algorithm that learns a PSR of a simulated mobile robot environment. We guide from the theory that is needed to understand the implemented algorithm to practice and provide in-depth information on all parts of our implementation. In a line of experiments we do not only validate former results that the learned PSRs are accurate enough to enable successful reinforcement learning, but further investigate the quality of learned models and the empirical performance of the algorithm itself. Therefore we apply the learning algorithm to environments of different complexity and examine the practical limits of the implemented approach. One of the main challenges we faced was the tuning of parameters. We found that tedious, environment specific fine tuning is needed to reliably learn accurate representations and thus investigate the influence of parameters on the quality of learned representations in several experiments in greater detail. The results are guidance for future work as well as they show possible problems that need to be tackled in order to improve PSR learning and make it applicable to complex real world domains.
To extend the field of application of robots in unstructured environments it is necessary to develop new techniques of environment perception and interpretation. These methods must give machines the capability to generate sufficient information, which enables them to fulfill their tasks with the aid of their sensors.
The autonomous execution of mobile manipulation tasks in natural environments requires complex motion capabilities. The motion needs to satisfy various motion constraints, imposed by the task and the environment.
Kinematic structures are currently one of the most important prerequisites for advanced robot technology.
Grasping objects, painting cars or performing household tasks are just a few examples from the manifold space of possible applications.
© RBO
© S. Höfer
© RBO
© RBO
© robotics
© RBO
