In the past years our research group has been investigating algorithms that drastically depart from “Motion Compensated DPCM/Transform (MC-DPCM)”. Our prime intention is to allow ourselves a completely “fresh view” on how non-linear dependencies between pixels and motion in images and video can be described and harvested for compression. To this end we employ non-linear machine learning algorithms that explore dependencies between vast amounts of pixels in images and video sequences. Our current approaches are based on non-linear Kernel methods, Steered Mixture of Experts networks (MoE) and Restricted Boltzmann Machines and show strong resemblance to recent work on deep neural networks. Our experiments give hope that our networks may provide far better visual quality compared to DPCM/Transform approaches in the long run.
Our challenge is to efficiently identify and harvest longest-range correlations in images and video - to allow for leaps in compression. Our strategies completely depart from current JPEG and MPEG/ITU type compression approaches with block processing, block transforms and motion vectors. In our recent work we develop specifically designed SMoE Gating Networks for compression. These networks are based on Steered Mixture-of-Experts (SMoE) networks that distribute swarms of N steered “atoms” into arrays of image pixels (for images) or into 3D stacks of video pixels (for video). Simple “atoms” may comprise of steered 2D Gaussian Kernels (for images) or of steered 3D Gaussian Kernels (for video). Kernel parameters include location of individual Kernels as well as steering and bandwidth parameters.
Kernel regression has been proven successful for image denoising, deblocking and reconstruction. These techniques lay the foundation for new image coding opportunities. We introduce a novel compression scheme: The sparse Steered Mixtureof-Experts (SMoE) regression network for coding image and video data.
Motion Modeling for Motion Vector Coding in HEVC" Tok, M.; Sikora, T.submitted to the Picture Coding Symposium (PCS) 2015 Abstract During the standardization of HEVC, new motion information coding and prediction schemes such as temporal motion vector prediction have been investigated to reduce the spatial redundancy of motion vector fields used for motion compensated inter prediction. In this paper a general motion model based vector coding scheme is introduced. This scheme includes estimation, coding and dynamic recombination of parametric motion models to generate vector predictorsand merge candidates for all common HEVC inter coding settings. Bit rate reductions of up to 4.9% indicate that higher order motion models can increase the efficiency of motion information coding in modern hybrid video coding standards.
The following zip-files contain exemplary groundtruth motion vector data for the test sequences. Due to website restrictions the filesize is limited to 20MB. If you want to obtain the complete motion vector fields please contact esche@nue.tu-berlin.de.
The working group "Video Coding" deals with approaches of global and local motion estimation and compensation with the aim to improve existing video codecs like e.g. H.264 / AVC. Currently, the working group is active in particular within the framework of the ITU / ISO / IEC standardization effort "HEVC". This has already resulted in numerous publications and input documents for MPEG. The sub-projects listed below have been processed so far.
Der MPEG-4 ALS Standard gehört zur Familie der MPEG-4 Audiocodierstandards, die von der ISO (www.iso.org) herausgegeben werden. Im Gegensatz zu verlustbehafteten Verfahren wie MP3 und AAC, die lediglich die subjektiv empfundene Qualität zu erhalten versuchen, erlaubt die verlustlose Codierung jedoch eine exakte Wiederherstellung jedes einzelnen Bits der ursprüglichen Audiodaten. Das grundlegende Verfahren von MPEG-4 ALS wurde am Fachgebiet Nachrichtenübertragung der Technischen Universität Berlin entwickelt. Die erste Version des MPEG-4 ALS Standards wurde 2006 veröffentlicht, und die aktuelle Beschreibung ist inzwischen Teil der 4. Edition (2009) des übergreifenden MPEG-4 Audiostandards (ISO/IEC 14496-3:2009). Eine neue Version (RM23) der MPEG-4 ALS Referenzsoftware und des Codecs ist jetzt verfügbar. Mehr dazu hier (in English)... MPEG-4 ALS wird mittlerweile von FFmpeg, MPlayer, VLC Media Player und weiteren Anwendungen unterstützt. Mehr dazu hier (in Englisch)...
A Tensorflow-based implementation of the Steered Mixture-of-Experts (SMoE) image modelling approach described in the ICIP 2018 paper Regularized Gradient Descent Training of Steered Mixture of Experts for Sparse Image Representation. This repository contains the implementation of a reference class for training and regressing a SMoE model, an easy to use training script as well as a jupyter notebook which helps you getting started writing you own code with a more elaborate example.
We are delighted to announce that our paper "Steered mixture-of-experts for light field coding, depth estimation, and processing" won the Best Student Paper Award at the IEEE International Conference on Multimedia and Expo, 10.07.2017 - 14.07.2017. Congratulations to Ruben Verhack and the co-authors.
Congrats to Prof. Sikora and his Communication Systems Lab members at TU Berlin, Lieven Lange, Rolf Jongebloed, and the colleagues from Uni Ghent/iMinds Lab, Ruben Verhack (joint PhD between TUB & iMind Lab Uni Ghent), Prof. Peter Lambert and Dr. Glenn van Walllendael. The award was given for his work on Video Compression with Steered-Mixture-of-Experts Networks.
We are delighted to announce that our paper "Lossless Image Compression based on Kernel Least Mean Squares" won the Highly Recommended Paper Award at the IEEE Picture Coding Symposium, 31.05.2015 - 03.06.2015. Congratulations to Ruben Verhack and the co-authors.
We are delighted to announce that our paper "LOSSY IMAGE CODING IN THE PIXEL DOMAIN USING A SPARSE STEERING KERNEL SYNTHESIS APPROACH" won the Top 10% Paper Award at the IEEE International Conference on Image Processing, 27.10.2014 - 30.10.2014. Congratulations to Ruben Verhack and the co-authors.
2009