156 lines
12 KiB
TeX
156 lines
12 KiB
TeX
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
% CHAPTER
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
\chapter{\HIGHLIGHT{Optimal Robust Video Coding for High Efficient Video Coding}}
|
|
\label{Chapter:EndToEndRobustSystem}
|
|
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
%% SECTION %%
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
\section{Introduction}
|
|
|
|
|
|
In this chapter the error resilience and error concealment approaches proposed in the previous chapters, namely Chapter~\ref{Chapter:ErrorResilienceTechniques}~and~\ref{Chapter:ConcealmentAwareTechniques}, are combined to achieved a higher decoder video quality across different network conditions and applications.
|
|
This chapter presents a combined evaluation of different approaches and proposes a method to efficiently select the optimal one in terms of reconstructed quality.
|
|
|
|
This chapter is organised as follows. Section~\ref{Section6:Problem} presents the problem formulation and motivation for the method proposed in this chapter. \HIGHLIGHT{...}
|
|
Section~\ref{Section6:Summary} concludes this chapter.
|
|
|
|
\clearpage
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
%% SECTION %%
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
\section{Problem formulation}
|
|
\label{Section6:Problem}
|
|
|
|
In this research work different robust coding techniques were proposed to increase the overall performance of video transmission under error-prone conditions. The methods proposed used different algorithms, as reference picture selection, motion vector redundancies and error concealment signalling. In previous chapters theses methods were evaluated against existing techniques revealing their superior performance and efficiency to deal with error robustness problems in high efficient video coding, e.g., using the \ac{hevc} standard.
|
|
|
|
The evaluation of these methods revealed that they can efficiently increase the robustness to transmission errors and losses, leading to higher decoded video quality. Results in Chapter~\ref{Chapter:ErrorResilienceTechniques} show that the proposed reference picture selection algorithm introduces low overhead in comparison with existing techniques. However, the method proposed in Chapter~\ref{Chapter:ConcealmentAwareTechniques} achieves higher quality gains at cost of slightly higher bitrate increase.
|
|
Therefore, it is noticeable that each method might be optimised for different applications, video characteristics and network conditions.
|
|
|
|
To advance one step forward, a jointly robust coding method using different techniques can be used to increase the decoder video quality across difference conditions.
|
|
Such approach can be based on both the input contents and the network information, such as bitrate and packet loss conditions to optimise the robust coders to each application scenario. In order to overcome the above problem a jointly evaluation study is necessary to identify the most efficient techniques and to evaluate their performance for different conditions.
|
|
|
|
|
|
\subsection{Comparison of the proposed methods}
|
|
|
|
In order to develop a jointly robust coder to cope with transmission errors an evaluation study was carried out, to compare the error robustness performance of the proposed methods for different conditions with each others.
|
|
%This study extends the previous evaluations presented in Chapter~\ref{Chapter:ErrorResilienceTechniques}~and~\ref{Chapter:ConcealmentAwareTechniques}, as they target a wider range of test material and network conditions.
|
|
The experimental evaluation carried out in this section follows the same condition described in Section~\ref{Section3:ErrorResilienceEvaluation}.
|
|
%In order to perform a wider evaluation different bitrate and \acs{plr} were used. Table~\ref{Table6:TestVariables} illustrates the selected configurations. As shown in the table a total of 15 test sequences were used encoder using 5 different target bitrates. One should note that the main bitrate varies based on the test sequence and it is shown in third column of Table~\ref{TableTestSequences}. Different network conditions were evaluated using a total of 13 different \acs{plr}, with higher special focus on lower loss ratios, which are more relevant to modern video streaming applications.
|
|
In this experimental evaluation the following methods are evaluated, namely:
|
|
\begin{itemize}
|
|
\item the default \ac{hevc} decoder with frame-copy \ac{ec} (\REF), as reference method;
|
|
\item the proposed reference picture selection scheme (\EXP);
|
|
\item the \EXP method in combination with an \ac{ec} optimisation with 10\% of bitrate of the scalable enhancement layer (\EXPELTEN);
|
|
\item the same as above with 20\% of bitrate for the scalable enhancement layer (\EXPELTWEN).
|
|
\end{itemize}
|
|
|
|
|
|
\begin{figure}[!t]
|
|
\centering
|
|
\input{043_EndToEndRobustSystem/Results/Quality_PLR_Methods.tex}
|
|
\vspace{-2mm}
|
|
\caption{Average decoded video quality using the main target bitrate.}
|
|
\label{Figure6:AverageQualityMain}
|
|
\end{figure}
|
|
|
|
\subsubsection{Experimental results}
|
|
|
|
Figures~\ref{Figure6:AverageQualityMain}~and~\ref{Figure6:AverageQualityReduction} shows the average decoded video quality obtained from 50 different trials, measured with the \ac{psnr} quality metric. The figures shows the quality obtained for different loss conditions. Figure~\ref{Figure6:AverageQualityMain} shows the results obtained with the test sequence encoded using a constant target bitrate shown in the third column of Table~\ref{TableTestSequences}. Results on Figure~\ref{Figure6:AverageQualityReduction} were obtained using a lower amount of coded bits, corresponding to a 40\% reduction of the main target bitrate. This aims at finding the impact of the video and network bitrate on the performance of the proposed methods.
|
|
|
|
\begin{figure}[!t]
|
|
\centering
|
|
\input{043_EndToEndRobustSystem/Results/Quality_PLR_Methods_Reduced.tex}
|
|
\vspace{-2mm}
|
|
\caption{Average decoded video quality using a 40\% reduction of the main target bitrate}
|
|
\label{Figure6:AverageQualityReduction}
|
|
\end{figure}
|
|
|
|
Results on both figures show that the proposed methods (i.e., \EXP, \EXPELTEN and \EXPELTWEN) are all able to outperform the reference \ac{hevc} for high \acs{plr}. In case of the proposed reference picture selection method (\EXP), quality gains in comparison with the \REF case are achieved for lower loss ratio, i.e., \acs{plr} as low as 2\% for all test conditions shown.
|
|
Comparing the results for both target bitrates also shows that the proposed methods are able to achieve higher quality gains for higher bitrates. Therefore, the for a lower bitrate (see results of Figure~\ref{Figure6:AverageQualityReduction}) the proposed methods only outperform the reference method for slightly higher \acs{plr}. This is more noticeable in the \EXPELTWEN case.
|
|
|
|
Summarising, the performance of the proposed methods is not similar across different video content and target bitrate, which impose a challenge on which method should be applied to achieve the highest decoded video quality in case of transmission errors. In order to achieve the highest quality the optimal trade-off between auxiliary information and video bitrate should be obtained by selecting the optimal error robust coding algorithm.
|
|
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
%% SECTION %%
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
\section{Proposed architecture}
|
|
\label{Section6:ProposedMethod}
|
|
|
|
In this section the architecture of the proposed method is described. The goal of the proposed method is to obtain an optimal error robustness method for a wide range of applications, i.e., video content and network conditions.
|
|
In order to accomplish that, at the encoder-side, the input signals are coded using different levels of error protections. Subsequently, an optimisation algorithm is used to decide which bitstream should be transmitted at any given instant, depending of different parameters.
|
|
|
|
\begin{figure}[tb]
|
|
\centering
|
|
%\includegraphics[width=.9\columnwidth]{\ConcealmentAwareDir/Figures/BlockDiagram}
|
|
\input{043_EndToEndRobustSystem/Figures/JointlyRobustnessDiagram.tikz}
|
|
\caption{Architecture of the proposed scheme.}
|
|
\label{FIG6:BlockDiagram}
|
|
\FIGSVSPACE
|
|
\end{figure}
|
|
|
|
Figure~\ref{FIG6:BlockDiagram} illustrates the architecture of the proposed encoding scheme.
|
|
Firstly, a reference video coder is used to compress the bitstream using highly efficiency coding techniques in order to obtain the best \ac{rd} performance for error-free environments. This is used as the reference bitstream in the proposed approach.
|
|
Subsequently, a transcoding operation based on the reference bitstream and the input video is used to efficiently select the reference pictures used for inter-frame prediction in terms of error propagation. The reference picture selection is applied as proposed in Chapter~\ref{Chapter:ErrorResilienceTechniques}.
|
|
Moreover, a loss simulation and saliency-based \ac{ec} optimisation is also applied on the error robust bitstream. This is used to select the optimal \ac{ec} algorithm which should be used at the decoder side.
|
|
Since the optimal \ac{ec} mode might not be able to correctly reconstruct all the missing regions, extra residual information is multiplexed in the stream as a scalable \ac{el} using two different bitrates, i.e., with 10\% and 20\% of the total bitrate. This provides two different levels of error robustness. This \ac{ec} optimisation and the \ac{el} coding is applied as proposed in Chapter~\ref{Chapter:ConcealmentAwareTechniques}
|
|
|
|
The proposed approach results in four different bitstreams: (i) reference stream without error robustness; (ii) robust video coded using a reference picture selection algorithm which leads to lower error propagation in case of data-loss; (iii) robust video coding and decoding using optimal reference pictures and optimal \ac{ec} method signalling and 10\% of bitrate allocated for the scalable enhancement layer \ac{ec} residue; (iv) same as the previous bitstream but with 20\$ of the bitrate allocated for the scalable \ac{el}. These bitstreams are referred in Figure~\ref{FIG6:BlockDiagram} as \textit{Stream\_1} to \textit{Stream\_4}, respectively.
|
|
|
|
Finally, the optimal robust stream is selected using a selection algorithm based on
|
|
|
|
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
%% SECTION %%
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
\section{Error robustness optimisation method}
|
|
|
|
|
|
\section{Performance evaluation}
|
|
|
|
\begin{table}[t]
|
|
\centering
|
|
\caption{List of the test conditions tested.}
|
|
\label{Table6:TestVariables}
|
|
\TABLESVSPACE
|
|
\begin{tabular}{m{4cm}|y{4cm}|y{6cm}}
|
|
\toprule
|
|
\textbf{Sequence} & \textbf{Bitrate} & \textbf{\ac{plr}} \\ \midrule
|
|
Basketball Drill & Main (see Table~\ref{TableTestSequences}) & Error-free \\
|
|
Book Arrival & 10\% reduction & 0.5\% \\
|
|
Bosphorus & 20\% reduction & 1.0\% \\
|
|
BQSquare & 30\% reduction & 1.2\% \\
|
|
Cactus & 40\% reduction & 1.6\% \\
|
|
Four People & 50\% reduction & 2.0\% \\
|
|
Jockey & & 2.2\% \\
|
|
Kendo & & 2.4\% \\
|
|
Kimono & & 2.6\% \\
|
|
Kristen and Sara & & 2.8\% \\
|
|
Park Scene & & 3.0\% \\
|
|
People on Street & & 5.0\% \\
|
|
Race Horses & & 10.0\% \\
|
|
Tennis & & \\
|
|
Traffic & & \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\end{table}
|
|
|
|
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
%%% SECTION %%
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
%\section{Error concealment optimisation based on MV differences}
|
|
|
|
|
|
|
|
\subsection{Experimental results}
|
|
|
|
|
|
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
%% SECTION %%
|
|
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
|
\section{Summary}
|
|
\label{Section6:Summary}
|