Robotics and Biology Laboratory

Recurrent State Representation Learning with Robotic Priors

People

Marco Morik
Divyam Rastogi
Rico Jonschkowski
Oliver Brock

Abstract

Marco Morik, 2018

Motivation

In robotics, the sensory inputs are high dimensional, however, only a small sub-
space is important to guide the actions of a robot. Previous work by Jonschkow-
ski and Brock in 2015 presented an unsupervised learning method extracting
a low dimensional representation using robotic priors. These robotic priors en-
code knowledge about the physical world in a loss function. However, their
and that of other methods to learn such a representation rely on an Marko-
vian observation space.

Description of Work

We extend the idea of robotic priors to work on non
Markovian observation spaces. For this, we train a recurrent neural network
on trajectories, such that the network learns to encode past information in its
hidden state. With this we are able to learn a Markovian state space. To train
this network, we combine and modify existing robotic priors to work in non
Markovian environments. We test our method in a 3D maze environment. To
evaluate the quality of the learned state representation, we introduce a valida-
tion network that maps from the learned states to the ground truth.

Results

Using the new validation network, we show that the learned state space contains both
positional and structural information of the environment. Furthermore, we use
reinforcement learning and show that the learned state space is sufficient to
solve a maze navigation task.