98 lines
13 KiB
TeX
Executable File
98 lines
13 KiB
TeX
Executable File
\graphicspath{{010_Introduction}}
|
||
|
||
\glsreset{hevc}
|
||
\chapter{Introduction}
|
||
\label{Chapter:Intro}
|
||
|
||
\section{Context and motivation}
|
||
|
||
\glsreset{h264}
|
||
|
||
The developments in video compression and networking technologies over the recent years have made the deployment of many applications over the internet possible such as video conferencing, video streaming, video publishing, among others. The increasing diversity of services, the demands for high quality multimedia and the growing popularity of \ac{uhd} formats (\emph{e.g.}, $4k\times2k$ or $8k\times4k$ resolution and beyond) are pushing technologies forward and opening new research directions.
|
||
Moreover, the emergent video formats and increased resolutions are creating even stronger needs for coding efficiency superior to the MPEG-4/\ac{h264}~\cite{H264AVCStandard2003,H264OverviewJul2003} capabilities.
|
||
The need is even stronger when higher resolutions are needed by stereoscopic and multi-view videos, due to the increased number of camera views that need to be delivered simultaneously. Moreover, the traffic generated by mobile multimedia applications and the video-on-demand services of ever increasing popularity impose new challenges to the existing networks.
|
||
Thus, a new coding standard was developed by the ITU-T \ac{vceg} and the \ac{iso}/\ac{iec} \ac{mpeg} standardisation organisations, referred to as H.265 - \ac{hevc}~\cite{H265StandardFeb2018,HEVCOverviewDez2012}.
|
||
|
||
\glsunset{h264}
|
||
|
||
The \ac{hevc} is foreseen as the main video coding framework to extend current services towards higher resolution formats delivered with better compression efficiency. Similar to \ac{h264}, \ac{hevc} is a hybrid video codec based on predictive transform coding and an entropy coding step. The coding flexibility and efficiency of \ac{hevc}, results from the use of new coding tools and block structures, which enables new block partitions\cite{HEVCBlock2012}, improved prediction modes \cite{IntraCodingHEVC2012,AMPCodingHEVCJun2012} and new high-level features~\cite{HEVCHighLevelReferenceDez2012}, such as explicit reference picture management and new parameter sets. However, it is known that higher coding efficiency leads to lower robustness to transmission errors and data loss. Summarising, the new coding tools in \ac{hevc} bring certain compression efficiency~\cite{HEVCComparisonDec2012}, but also has some other disadvantages, such as increased complexity~\cite{PerformanceComplexityDez2012} and inherent reduction in error robustness.
|
||
|
||
\begin{figure}[tb]
|
||
\centering
|
||
%\includegraphics[width=0.5\columnwidth]{./010_Introduction/HEVCTransmissionStack}
|
||
\includegraphics[width=0.7\columnwidth]{./010_Introduction/HEVCTransmissionStackOverview}
|
||
\caption{System layer stack compatible with \ac{hevc} video transmission~\cite{HEVCLayerIntegrationDez2012}.}
|
||
\label{fig:HEVCStack}
|
||
\end{figure}
|
||
|
||
The error robustness characteristics of \ac{hevc} were analysed in \cite{StudyHEVCErrorsDez2012} confirming that, in general, the \ac{hevc} presents poor error resilience performance, despite its superior coding efficiency in comparison with previous standards.
|
||
Thus, when \ac{hevc} streams incur in transmission losses, this results in significant degradation of both objective and subjective quality~\cite{QoEHEVCMay2014,SubjectiveEvalSep2013}.
|
||
This is mostly due to the strong decoding dependencies imposed by the highly complex prediction modes of \ac{hevc}, which are selected according to a \mbox{rate-distortion~(R-D)} optimisation criteria, always assuming \mbox{error-free} transmission~\cite{ROPE2007}. A possible approach to encode robust video streams is to increase the error resilience of \ac{hevc} by including \mbox{error-aware} mechanisms in the coding process. As a result, the effect of error propagation due to prediction mismatch at the decoding side, will be limited.
|
||
|
||
The widespread use of \ac{hevc} in broadcast and multimedia delivery services, will be based on transport technologies such as RTP~\cite{HEVCStreamMay2012}, \mbox{MPEG-2 TS}~\cite{HEVCLayerIntegrationDez2012}, or more recent standards as MPEG-DASH~\cite{HEVCHTTPStreamAug2016}.
|
||
Recent developments also enable combinations of different technologies, e.g, \mbox{HTTP-based} streaming and \mbox{MPEG-2 TS}, to increase the flexibility of over-the-top (OTT) video services\cite{Broadcast2017}. For content delivery over heterogeneous networks, the MPEG Media Transport (MMT)~\cite{ATSCMMT2016} is another possible technology to be used with the \ac{hevc} standard. Besides the transport technologies,
|
||
the overall error robustness of video transmission may also take advantage of the increasing flexibility of multimedia systems to cope with heterogeneous networks, by using auxiliary streams to aid the video decoder to increase the quality of recovered lost slices~\cite{ErrorResilienceHeterogenousNetworks2016,HeterogeneousStreaming2016}. Moreover, error resilience techniques may also take advantage of the multi-stream based approach, where each stream allows different levels of robustness to errors~\mbox{\cite{HEVCHTTPStreamAug2016,PerfMediaTransport2017}}.
|
||
|
||
|
||
%Despite the different characteristics of the existing services and applications, the users' \ac{qoe} consistently depends on how the compressed streams are adapted to the diverse networking technologies, as well as on how the transmission errors and/or packet losses are handled at the application level ~\cite{HEVCStreamMay2012,HEVCLayerIntegrationDez2012,StudyHEVCErrorsDez2012}. As mentioned before, the \ac{hevc} standard is expected to replace the \ac{h264} in future video services.
|
||
|
||
|
||
%The system interface of \ac{hevc} is an important component
|
||
%in the media access chain, and a prerequisite for its successful
|
||
%deployment in various consumer electronics applications. We
|
||
%consider four broad categories of systems: broadcast delivery
|
||
%over digital television and Internet Protocol Television (IPTV)
|
||
%channels including video-on-demand (VoD), streaming over
|
||
%the Internet using the Hypertext Transport Protocol (HTTP),
|
||
%real-time communications such as video conferencing, and
|
||
%store-forward delivery based on a file or a physical memory
|
||
%delivered to the end user (such as: camcorded videos or Blu-
|
||
%ray Disks). To this end, different system technologies have
|
||
%been standardized. These technologies can be summarized as
|
||
%follows.
|
||
|
||
|
||
|
||
|
||
In summary, it is an emergent research problem to increase the adaptability of \ac{hevc} compressed streams for diverse network conditions and its robustness against transmission errors, in order to provide acceptable levels of \ac{qoe} to the end users at all times. The \ac{hevc} reference codec is still under development by multiple proponents from around the world, and there are open questions in terms of error resilience and techniques to increase the concealment performance on the decoder side. Although the \ac{ec} methods have been developed during the past years for \ac{h264}~\cite{2DRecursiveRefinement2010} and recently for \ac{hevc}~\cite{HEVCMCBlockAug2013}, a combined and optimised approach between encoder and decoder is yet to be developed, which would improve the users’ \ac{qoe} by exploiting shared information between the encoder and the decoder sides.
|
||
|
||
\section{\HIGHLIGHT{Main objectives and contributions}}
|
||
|
||
This research plan addresses robust error resilience and efficient error concealment for the forthcoming multimedia services, based on the recent \ac{hevc} standard. Thus, this work focuses on novel techniques to increase delivered quality of \ac{hevc} bitstreams in the following respects:
|
||
\begin{enumerate}
|
||
\item \textbf{evaluate the error robustness of \ac{hevc}:}
|
||
|
||
\item \textbf{source coding level:} At the encoder side, novel techniques to extend the existing error robustness features of the \ac{hevc} standard will be investigated in order to increase the received video quality over error-prone networks. The new coding features introduced in the standard increase the compression ratio, resulting in more complex dependencies that lead to increased error propagation rate.
|
||
|
||
\item \textbf{error concealment of the received video:} At the decoder side, the error concealment techniques improve the perceived video quality by reconstructing the missing information. The paradigm of error concealment should be evaluated at the encoder side, as part of the encoding process, in order to evaluate the concealment performance and share used information with the decoder to improve the overall performance of the transmission system.
|
||
|
||
\item \textbf{jointly error resilience and concealment:} the ultimate goal of this research work is to strike a balance between the designed error resilience and error concealment techniques so as to meet higher quality demands for different video contents characteristics and network conditions.
|
||
\end{enumerate}
|
||
|
||
%Finally, the ultimate goal of this research work is to strike a balance between the designed error resilience and error concealment techniques so as to meet a set of \ac{qos}/\ac{qoe} requirements.
|
||
%Since the ultimate quality judgement is made by end users, subjective tests will constitute a part of the research work in order to validate the objective gains reported as a result of the deployment of various proposed and implemented methods. Given the importance of the \ac{hevc} standard in the near future, the simulation scenarios will be defined based on practical service requirements, such as, bit rate allocation, transmission delay and required QoS.
|
||
|
||
\section{\HIGHLIGHT{Outline of this thesis}}
|
||
|
||
This thesis is organised as follows. The current chapter introduces the main research topics of this thesis, describing the motivations and objectives to achieve with this work, as well as the original contributions of this thesis.
|
||
|
||
The first two chapters review the \mbox{state-of-the-art} concepts related with this research work. Chapter~\ref{Chapter:HEVCReview} presents an overview of the \ac{hevc} standard, with some details in the concepts explored in this work, e.g., high-level features and temporal predictions. Chapter~\ref{Chapter:ErrorResilienceReview} reviews some of the work regarding robust video transmission proposed over the last years, since they provide relevant knowledge to the main objectives of this work.
|
||
|
||
Chapter~\ref{Chapter:HEVCErrorResilience} firstly presents an evaluation study of the performance of the \ac{hevc} standard under errors for various coding configuration. The study aims at finding the relevant tools introduced in the standard that highly affects the error robustness. Secondly, a new method is proposed to decrease the vulnerability of \ac{hevc} temporal prediction and decrease error propagation, leading to higher video quality in error-prone conditions.
|
||
|
||
Chapter~\ref{Chapter:ErrorResilienceTechniques} proposes a novel error resilience approach in the context of the \ac{hevc} standard. The motivation for this works is related with the poor error robustness performance show in Chapter~\ref{Chapter:HEVCErrorResilience}. Thus, a new method to distribute the use of reference frames was proposed to reduce the error propagation and improve the \ac{ec} performance. Moreover, to reduce the effect of mismatch motion predictions (also shown in Chapter~\ref{Chapter:HEVCErrorResilience}) a method based on selective motion vector redundancies is proposed. This method is able to improve the decoded video quality using a low amount of overhead.
|
||
|
||
Chapter~\ref{Chapter:ConcealmentAwareTechniques} addresses the problem of efficient \ac{ec} in the \ac{hevc}. This work was motivated by the requirements for more accurate \ac{ec} algorithms which are able to reduce the error propagation. Different \ac{ec} algorithms were taken into consideration in order to develop a concealment-aware coding scheme. The proposed scheme is based on a scalable coding approach where the best \ac{ec} methods to be used at the decoder are optimally determined at the encoder and signalled to the decoder. Such optimal EC\ac{ec} modes are found by simulating transmission losses followed by a \ac{rd} optimisation of the signalling rate and \ac{ec} distortion cost.
|
||
A generalised saliency-weighted distortion is used and the residue between coded frames and their EC substitutes is encoded using a rate-controlled enhancement layer. When data loss occurs the decoder uses the signalling information is used at the decoder, in case of data loss, to improve the reconstruction quality. The simulation results show that the proposed method achieves consistent quality gains in comparison with other reference methods and previous works.
|
||
|
||
|
||
Chapter~\ref{Chapter:EndToEndRobustSystem} ... \HERE
|
||
|
||
Chapter~\ref{Chapter:Conclusions} concludes this dissertation and presents some suggestions for future work.
|
||
|
||
Appendix~\ref{Appen:Sequences} illustrates the original test signals used in simulations throughout the thesis. This includes a partial set of the \ac{hevc} test material and other external test material.
|
||
Appendix~\ref{Appen:Papers} presents a summary of the published papers, describing the contributions of the research work of this thesis.
|
||
|
||
|
||
|