Video coding and networking for resource constrained applications

The problem

With the gradual expansion of both wired and wireless Internet to most corners of the globe, video communication is gaining significant prominence. While the effectiveness of video communication is critically dependent on sharing useful information, low bandwidth availability and network unreliability, common in scenarios where it is most needed, may compromise or completely undermine its usefulness. Existing techniques do not optimize the source coding or the channel coding decisions to accurately account for real-time channel conditions and their impact on the received signal, nor do they account for user requirements, resulting in sub-optimal use of resources.

Our solution

Our approach addresses the problem with a two-fold solution. First, we employ scalable coding to enhance streaming flexibility, where the video sequence is encoded into a single bit-stream consisting of multiple layers with progressively higher spatial, temporal, or quantization resolutions, but with an important novelty of higher quality layers being encoded optimally by accounting for all the available information from previously reconstructed frames and layers. Second, we accurately account for real-time network conditions by optimally estimating, at the encoder, the end-to-end distortion expected at the decoder, via an estimation-theoretic technique that accounts for all relevant effects of compression, error protection, error propagation, and concealment at the decoder.