Robotics and Biology Laboratory

Learning a Repetitive In-Hand Manipulation Task on the Sensorized RBO Hand 2 Using Policy Search


The RBO Hand 2 is a highly compliant soft robotic hand. Its actuators passively adapt their shape to different objects and the environment. Even though the control of the pneumatic hand is relatively simple, it is capable of complex in-hand manipulation: Ball Rotation (Video)Pen Flip (Video). The recent addition of liquid metal strain sensors has created the opportunity to obtain better feedback about the current state of the hand. The goal of this thesis is to utilize this new sensor information to make the execution of different in-hand manipulation tasks more robust.


We research how a soft robotic hand manipulates a grasped object more reliably when we consider the position of the object in a closed-loop controller. If the open-loop controller ignores the position, uncertainties accumulate and the object drops. We hypothesize that information about the position is sufficiently included in the deformation of the fingers to classify it because they adapt to the shape of the object due to the compliance. We measure the deformation with strain sensors and classify discrete object positions with an accuracy of 100 % and object positions inside of three itself overlapping regions with an accuracy of 64 % to 84 %. We show for an exemplary in-hand manipulation task that a sensorized soft robotic hand is suitable for closed-loop control by training a mapping that estimates an actuation command from the sensor data. Training a mapping at the limits of a specific in-hand manipulation task, allows the closed-loop controller to adapt to an object that slips inside of the limits caused by uncertainties. We train the mapping by executing the desired task multiple times on the real hardware. We design a cost function in task space to be able to extend the closed-loop control to use reinforcement learning.