Motivation & Project Description Given the fact of the Telekom AG interest in Internet TV (IPTV) one of the key issues is   giving a user a complete  freedom to  interact with  the broadcasted  video. The  main idea  is that  the  user can arbitrary choose a new viewpoint from which he wants  to view the scene and to be in full control of changing the viewpoint  anytime he wants. It is  possible to imagine plenty of  scenarios where this will be very useful feature: sport events,  movies and entertainment programs. The system which will provide  such   a  service  has  to   be  based  on   multi-camera  system  which  film   the  scene simulateniously. The 3D representation of the  scene has to be reconstructed from those synchronized multiple videos in real time and multicasted to the multiple users. Having a complete  textured 3D representation of the scene one is able to  move a  virtual camera  around the  scene in  any freely  chosen viewpoint.   Having the 3D information and  the texture on the client side the user can freely change his view point while the scene is dynamically changing, to freeze the scene and move around and eventually augment the scene by changing the surroundings. The following topics will be covered by this project:

• 3D Reconstruction

   

  • Development of an advanced  algorithms for 3D reconstruction from multiple cameras
  • Enhance shape using photoconsistency and temporal coherence

   

• Multiple view  video compression, coding and interpolation

   

  • Efficient coding of multi-view video signals
  • Edge-adaptive interpolation of lightfield in between multiple cameras

   

• Camera callibration

   

  • Enable calibration from raw video of the network of cameras.

   

• Interaction

   

  • Vision based intuitive display and control of viewpoint for user.

   

Expected Outcome: The project consists of research and technology parts. The research should result in scientific publications and patents, while technology part should produce a demonstrator for T-Galleries.

Time Frame: 07/2007 – 07/2010

T-labs Team Members: Rahul Swaminathan, Slobodan Ilic, Vladan Velisavljevic, Michael Rohs

Partners: PERCEPTION, INRIA, Rhone-Alpes / HHI, Fraunhofer Institut, Berlin / GRASP Laboratory, University of Pennsylvania

Funding by: Deutsche Telekom Laboratories