Robotics and Biology Laboratory

Integrating Fixation and RL for Adaptable Behavior Generation

Motivation

Humans exhibit remarkable proficiency in navigating and interacting with the 3D world using only 2D sensors like the eyes. Despite the apparent limitations of human retinas compared to advanced sensors like lidars or high-resolution cameras, humans can robustly interpret and respond to their surroundings. One of the keys lies in exploiting regularities in 3D space through purposeful movements, such as gaze fixation. Gaze fixation involves coordinated movements of the body and eyes, allowing humans, in principle, to extract relevant 3D properties by focusing on one object at a time during motion. Taking advantage of such coordinated movements has been demonstrated so far on a robot.

This thesis topic draws inspiration from gaze fixation to structure 3D perception on robots and complements it with Reinforcement Learning to learn optimal policies on top of that structured representation.

Description of Work

By combining gaze fixation with Reinforcement Learning, this thesis aims to quickly create new behaviors on the robot depending on a task description, ultimately paving the way for the development of efficient and adaptable robotic systems.

In this thesis, you will:

  1. Learn Structuring 3D Perception with Gaze Fixation:
    • Investigate the principles behind gaze fixation and its role in structuring 3D perception on robots.
    • Implement gaze fixation movements on the robot in simulation, and evaluate its effectiveness in extracting meaningful 3D information from the environment.
  2. Integrate Reinforcement Learning for Optimal Policies:
    • Develop an RL framework that builds upon the structured representation provided by gaze fixation.
    • Design algorithms that enable the robot to learn optimal policies for various tasks based on the structured 3D perception, starting with the task of pick-and-place.
  3. Compare Structured vs. Unstructured Approach:
    • Compare the performance of the structured gaze fixation-RL approach with traditional unstructured methods, emphasizing the efficiency and adaptability gained through structured 3D perception.
  4. Optional: Real-world Application and Generalization:
    • Apply the developed framework in real-world scenarios to evaluate its effectiveness in practical robotic applications.
    • Investigate the system's generalization capabilities, ensuring its robust performance across different environments and task complexities.

How to apply

You can find all the necessary information here.