jfmcarreira/doc_phd_thesis
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Files
8d252f0dbd53bdf8f67c728fcdd0ec1b512dd6aa
doc_phd_thesis/041_ImprovedErrorResilience/ImprovedErrorResilience.tex
T

520 lines
53 KiB
TeX
Raw Blame History

% Figure folders
\newcommand{\ImprovedErrorResilience}{./041_ImprovedErrorResilience}
\graphicspath{{\ImprovedErrorResilience/Results/}}
\chapter{A Two-stage Approach for Robust Coding and Streaming}
% \chapter{Error Resilience Techniques for \ac{hevc} Standard}
%\chapter{Two-stage Robust Video Coding}
\label{Chapter:ErrorResilienceTechniques}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% SECTION %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}
\glsreset{mv}
\glsreset{rd}
\glsreset{ra}
\glsreset{ld}
%The increased compression ratios achieved by the \ac{hevc} standard lead to reduced robustness of coded streams, with increased susceptibility to network errors and consequent video quality degradation, as shown in Chapter~\ref{Chapter:HEVCErrorResilience}.
Therefore, this chapter proposes a coding method, based on a two-stage approach, to improve the error robustness of \ac{hevc} streaming, by reducing temporal error propagation in case of frame loss.
This is accomplished by reducing the prediction mismatch that occurs at the decoder after frame loss through the following two stages: (i) at the encoding stage, the reference pictures are dynamically selected based on constraining conditions and Lagrangian optimisation, which distributes the use of reference pictures throughout each frame, by reducing the number of \acs{pu} that depend on a single reference; (ii) at the streaming stage, a \ac{mv} prioritisation algorithm, based on spatial dependencies, selects an optimal sub-set of \acs{mv} to be transmitted, redundantly, as side information to reduce mismatched \ac{mv} predictions at the decoder.
This chapter is organized as follows. Section~\ref{Section4:Problem} presents the problem definition and motivation for the method proposed in this chapter. Section~\ref{Section4:Architecture} provides an overview of the proposed method, describing the functional blocks that compose the two-stage approach for robust coding and streaming. Section~\ref{Section4:RefPicSel} describes the encoding stage of the proposed method, that is used to select the optimal reference picture for inter-prediction. Section~\ref{Section4:MVSelection} describes the streaming stage used to select a sub-set of \acs{mv} to reduce the mismatch motion predictions. Section~\ref{Section4:Performance} provides a performance evaluation of the proposed method, illustrating the advantages of each stage and how they are combined to achieve superior error robustness in \ac{hevc} standard. Finally, Section~\ref{Section4:Summary} concludes the chapter.
\clearpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% SECTION %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Problem definition}
\label{Section4:Problem}
As mentioned before \ac{hevc} bitstreams subject to transmission losses result in significant degradation of both objective and subjective quality.
This is mostly due to the strong decoding dependencies imposed by the highly complex prediction modes, which are selected according to a \ac{rd} optimisation criteria, assuming error-free transmission~\cite{ROPE2007}.
Therefore, high compression efficiency is achieved, but resilience to transmission errors is penalised.
\begin{figure}[tb]
\centering
\input{./041_ImprovedErrorResilience/Results/AverageRefPicUsage.tex}
\caption{Average usage of the reference frame 1 ($r_1$) for different sequences.}
\label{FIG4:ReferenceUsageRatio}
\end{figure}
Highly efficient video encoders support the use of multiple reference frames for inter-frame prediction, which are selected based on a \ac{rd} cost which may lead to an unbalance distribution.
In order to show evaluate the unbalanced is the reference frame selection attained by the \ac{rd} implemented in the reference \ac{hevc} encoder, different test sequences are encoded using two reference frames, referred to as $r_1$ and $r_2$, and its selection ratios are analysed. In this study, reference $r_1$ corresponds to the closest neighbouring frame and reference $r_2$ the second closest.
Figure~\ref{FIG4:ReferenceUsageRatio} illustrates the average usage percentage of reference $r_1$ for different sequences.
The remaining corresponds to those cases where reference $r_2$ were selected.
These results show that, in general, reference $r_1$ is selected more often than $r_2$.
This is mainly true for sequences with higher motion. For those sequences with lower motion but high spatial complexity, e.g., BQSquare (see \ac{si} in Table~\ref{TableTestSequences}), both reference frames equally (approximately) selected for temporal predictions, with $r_1$ being used for 58\% of cases and $r_2$ for 42\%.
This reveals that even with a configuration with two references the encoder will mainly use the temporally closer one as the reference for inter-frame predictions.
Therefore, if such reference frame is lost, a higher number of mismatched predictions will occur, inevitabely leading to higher error propagation.
Another technique that contributes to increased error propagation is the use of predictions in \ac{mv} coding~\cite{MotionCodingHEVC2011}, which contributes to increase the spatial and temporal dependencies.
Although the spatial \ac{mv} candidates are used more often, temporally predicted \acs{mv} are responsible for a greater impact on error propagation, as they propagate errors in the \ac{mv} decoding across different time instances.
To overcome these issues, in the next sections a two-fold approach is described to increase the error robustness in video streaming by jointly considering two different stages: (i) while encoding live video, by selecting robust coding modes and (ii) after encoding, by adding robust features to the compressed streams. While in the former this is accomplished by optimising coding parameters and decisions to increase robustness, in the latter some type of compressed domain processing is used to increase error robustness of pre-encoded streams, e.g., error resilient transcoding. Inevitably, in both cases this is achieved at the expense of some loss in coding efficiency, therefore the challenge is to find an optimum trade-off between robustness and bitrate overhead. In this chapter a two-fold approach is used to overcome the two problems mentioned above.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% SECTION %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Proposed two-stage architecture}
\label{Section4:Architecture}
The goal of the proposed method is to reduce the error propagation in decoded video in case of frame loss.
The underlying idea is to constrain inter-frame prediction dependencies, in order to minimise the distortion $D_t$ given by
\begin{equation}
D_t = \tilde f_t - \hat f_t, \text{when } \tilde f_{t-1} \neq \hat f_{t-1},
\label{eq1}
\end{equation}
where $\hat f_t$ and $\tilde f_t$ are the encoded and decoded frames, respectively, at instant $t$.
Figure \ref{fig:proposedMethod} illustrates the block diagram of the two-stage architecture proposed in this work.
At the encoding stage, a reference frame selection mechanism is used for each \ac{pu} within a coding tree unit (\ac{ctu}) forcing the use of different reference frames in adjacent \acs{ctu}. The decision criterion contributes to reduce the amount of predictions from a single reference frame, and forces to interleave different reference frames along all encoded \acs{pu}, based on \ac{rd} optimisation.
At the streaming stage, the coded stream is parsed to extract the \ac{mv} information, which allows ranking \acs{mv} according to a relevance criterion. Then, a sub-set of these \acs{mv} is selected to be transmitted as side information. The amount of selected \acs{mv} is not fixed and can be optimised for different content and network characteristics.
Note that the streaming stage is an independent and complementary subsystem to the encoder. Thus, it is able to operate at network nodes, including streaming servers delivering pre-encoded streams.
\begin{figure}[tb]
\centering
% \includegraphics[width=0.85\columnwidth]{041_ImprovedErrorResilience/Figures/BlockDiagram}
\input{041_ImprovedErrorResilience/Figures/BlockDiagram.tikz}
\caption{Functional blocks of the proposed method (\textsl{gray} blocks) in the context of the \ac{hevc} coding and streaming.}
\label{fig:proposedMethod}
\end{figure}
In case of frame loss these two stages have the common objective of helping the decoder to recover from mismatched predictions at the expense of a small increase in bitrate.
This is equivalent to reducing the error propagation by using two techniques: (i) reducing the mismatch of inter-frame predictions, by encoding each \ac{pu} from a different reference frame and (ii) reducing the mismatch of \ac{mv} predictions by sending the most important \acs{mv} as side information. These are the two stages of the proposed method described in the following sections.
\section{Encoding stage: Reference frame selection}
\label{Section4:RefPicSel}
The reference picture selection mechanism attempts to uniformly distribute the use of different reference pictures across the \ac{ctu} of a coded video frame.
In case of frame loss, the error propagation in the decoder is reduced because the number of \acs{ctu} predicted from the lost frame decreases and thus error concealment benefits as more data is correctly decoded from other frames.
The underlying principle of this method is to minimise the probability of using the same reference picture for neighbouring \acs{ctu}.
%For instance,
In a coding structure with two reference frames, as shown in Figure~\ref{fig:checkerboardRefSelection}, the reference frames $r_{1}$ and $r_{2}$ are used to predict the same number of \acs{ctu} in current frame~($C$), resulting in a checkerboard structure, where a different reference frame is used for predicting every other \ac{ctu} within the current frame.
If each reference frame is used to predict half of the pixels in the current frame, when any of these references is lost, the subsequent predicted frame is only partially distorted by mismatched predictions.
This is because only half of its predictions depend on the missing reference frame, which limits the overall error propagation.
Moreover, since the motion information of the predicted frame is encoded with not only reference to the missing frame but also to the past ones, those \acs{mv} that cross over a missing reference frame, pointing to the previous ones, are still useful for reconstructing correct \acs{mv} and also for error concealment.
In the limit, a straightforward implementation of the previous example leads to a simple interleaved uniform distribution of the reference frames over all \acs{ctu} in the current frame, forcing the checkerboard structure shown in Figure \ref{fig:checkerboardRefSelection}. In the case of two reference frames, 50\% of the \acs{ctu} would use one reference frame, while the other 50\% would use the other reference.
However, such solution would not take into consideration coding efficiency, since the selection process of the reference frame would not be \ac{rd} optimised for each \ac{pu} within a \ac{ctu}.
\begin{figure}[tb]
\centering
\input{\ImprovedErrorResilience/Figures/RefPicSelection.tikz}
\caption{Example of the reference picture selection applied to the frame $C$ based on the checkerboard at \ac{ctu} level (\acs{pu} represented by small partitions).}
\label{fig:checkerboardRefSelection}
\end{figure}
Therefore, a more efficient approach to improve error robustness was devised to dynamically choose the coding mode for each \ac{pu}, that ensures a balanced use of all possible reference frames $F=\{r_1,r_2,...r_S\}$ across the \acs{ctu} of the current frame $C$, also optimising the \ac{rd} cost.
This leads to the overall goal of finding the optimum R-D coding mode for each \ac{pu}, as the one which tends to minimise the standard deviation of the reference frames use count $\mathbf{N}_C(r_i)$, considering all encoded \acs{pu} of the current frame $C$, as follows:
\begin{equation}
\sigma_C=std\Big(\mathbf{N}_C(r_i) \Big), i=1, 2,\cdots S.
\label{eq:StdX}
\end{equation}
In this case $\sigma_C$ measures the deviation from the checkerboard solution, where $\sigma_C = 0$, since all possible reference frames $r_i\in F$ are uniformly used throughout the \acs{pu} of $C$.
To attain the goal defined above, the best coding mode for each \ac{pu} is chosen through a Lagrangian \ac{rd} optimisation that tends to minimise $\sigma_C$.
This is done by penalising (or benefiting) the \ac{rd} cost of those coding modes which intend to use reference frames that have already been used more (or less) frequently than their average use in previous \acs{pu} of the current frame.
For each \ac{pu}, this mechanism has the effect of reducing (or increasing) the probability of using each possible reference frame $r_i\in F$, according to the number of previous \acs{pu} that have used each one.
Such approach allows to select the reference frames, by independently processing each \ac{pu} in order to minimise the global standard deviation given by~Equation~(\ref{eq:StdX}). Thus, the optimum coding mode $\phi^*$, selected from a set of possible coding modes $M$, is derived based on the following Lagrangian optimisation:
\begin{align}
\phi^* &= arg \min_{\phi \in M} J(\phi) \\
J(\phi) &= \Big( D(\phi) + \lambda \times R(\phi) \Big) \times e^{W(r_{\phi})}
\label{eq:Lagrangian}
\end{align}
where $D(\phi)$ and $R(\phi)$ are the distortion and rate, respectively, associated with the coding mode $\phi$.
The exponential weighting factor $e^{W(r_{\phi})}$ is used to penalise (or benefit) the Lagrangian cost of a given coding mode $\phi$ associated to reference frame $r_\phi$.
$W(r_{\phi})$ is obtained according to the global and local deviation between the number of times that $r_\phi$ was used and the average use of all $r_i \in F$ across the \acs{pu} of current frame $C$, as follows,
\begin{equation}\label{eq:KFactor}
W(r_{\phi}) = \Big( \Delta_G(r_{\phi}) + \Delta_L(r_{\phi}) \Big)\times 2^{-T_{ID}} \times \gamma
\end{equation}
where $T_{ID}$ is given by the temporal hierarchy in \ac{hevc}, and $\gamma$ is a parameter to control the slope of ${W(r_\phi)}$.
The global and local deviations, $\Delta_G (r_\phi)$ and $\Delta_L (r_\phi)$, respectively, are computed as follows,
\begin{equation}\label{eq:KFactorXValue}
\Delta_\zeta(r_{\phi}) = \mathbf{N}_\zeta(r_\phi) - \frac{1}{S} \sum_{i=1}^{S}\mathbf{N}_\zeta(r_i), \hspace{2mm} \zeta=G,L.
\end{equation}
For $\zeta=G$, function $\Delta_G(r_\phi)$ measures the difference between the number of times that reference frame $r_\phi$ was used and the global average use of all $r_i \in F$ in the \acs{pu} of $C$ that were encoded before the current \ac{pu}. $\mathbf{N}_G(r_i)$ is the number of times that each reference frame $r_i$ was used across all previously encoded \acs{pu}. For $\zeta=L$, function $\Delta_L(r_\phi)$ only considers the three top-left neighbouring \acs{ctu} for counting the number of times each reference frame was used, which is expressed by $\mathbf{N}_L(r_i)$. Overall, Equation (\ref{eq:KFactorXValue}) is used to adjust the penalty weight defined by Equation (\ref{eq:KFactor}) in order to achieve an approximately uniform use of all possible reference frames. This corresponds to the general objective of minimising the values of $\Delta_\zeta(r_{\phi})$.
The optimum is reached when all reference frames are used to encode the same amount of \acs{pu} (i.e., $\Delta_G(r_{\phi})=\Delta_L(r_{\phi})=0$), and the original Lagrangian cost is not affected, i.e., $e^{W(r_{\phi})}=1$ in Equation (\ref{eq:Lagrangian}).
Note that an exponential cost increase is used in Equation (\ref{eq:Lagrangian}) in order to increase the penalty associated to any reference frame that tends to be used much more often than the average.
\begin{figure}[t]
\centering
\includegraphics[width=0.8\columnwidth]{\ImprovedErrorResilience/Figures/ExponentialWeight}
\caption{Exponential weight as a function of the global and local deviations}
\label{fig:weightfct}
\end{figure}
In Equation (\ref{eq:KFactor}), the $\gamma$ constant controls the slope of $W(r_\phi)$, leading to higher weights as the value of $\gamma$ increases.
On the one hand a very high value for $\gamma$ leads to the checkerboard pattern in the reference picture usage, as previously described.
On the other hand, a low value of $\gamma$ reduces the importance of the choice of the reference frames in the Lagrangian cost.
Therefore, the impact of the proposed method on the coding efficiency can be controlled through $\gamma$. Another way of controlling the impact of the proposed method is the use of the $T_{ID}$. This allows using lower weights in higher temporal layers, since such frames cannot be used as references for others with lower temporal ID, thus having less impact on error propagation.
Therefore, there is a reduction of the impact of the proposed method when hierarchical coding configuration is adopted without compromising the error robustness.
As an example, Figure~\ref{fig:weightfct} illustrates the exponential weight used in the proposed method for different values of $\gamma$ and $T_{ID}=0$. The horizontal axis represents the sum of $\Delta_G(r_\phi)$~plus~$\Delta_L(r_\phi)$.
For instance, for $\gamma=5$, when a reference frame $r_\phi$ is used to predict 3 more \acs{ctu} ($\Delta_G(r_\phi) + \Delta_L(r_\phi)=3$) than the other reference frames (all \acs{pu} within those \acs{ctu} use the same reference), then the Lagrangian cost is increased by $50\%$ ($e^{W(r_{\phi})}\approx1.5$).
\begin{figure*}[tb]
\centering
\newcommand{\figsize}{0.22\textwidth}
\resizebox{\textwidth}{!}{
\begin{tabular}{y{.24\textwidth}y{.24\textwidth}y{.24\textwidth}y{.24\textwidth}}
\small Original frame & \small \REF & \small \CHKB & \small \PROP \\
\includegraphics[width=\figsize]{SelectionExample/ORG} &
\includegraphics[width=\figsize]{SelectionExample/REF} &
\includegraphics[width=\figsize]{SelectionExample/CHKB} &
\includegraphics[width=\figsize]{SelectionExample/EXP} \\
& \footnotesize $r_{1}$: $61.8\%$ | $r_{2}$: $25.7\%$ & \footnotesize $r_{1}$: $42.2\%$ | $r_{2}$: $39.3\%$ & \footnotesize $r_{1}$: $45.22\%$ | $r_{2}$: $43.2\%$ \\
\end{tabular}
}
\caption{Example of the partitions and references pictures selected using the different encoding methods (the dark gray squares indicate that the \ac{mv} is using the reference $r_{1}$, light~gray squares corresponds to the reference $r_{2}$ and unfilled regions are intra-coded blocks).}
\label{fig:RefPicSelectionExample}
\FIGSVSPACE
\end{figure*}
Figure~\ref{fig:RefPicSelectionExample} illustrates the distribution of reference frames achieved by the proposed method across different \acs{pu}. This result is compared against a fixed checkerboard approach (\CHKB) and the reference \ac{hevc} (\REF). The dark-gray and light-gray squares indicate the reference frame in use, either $r_1$ or $r_2$, respectively. The white squares indicate intra-coded \acs{pu}. Below each method, the percentages of the image areas predicted by each reference frame are shown. As shown in this figure, the proposed method is able to distribute the use of the reference frames closely to the \CHKB method, i.e., $r_2$ is used as reference for approximately the same amount of \acs{pu} as $r_1$ (see use percentages of the predicted image). However, the predicted regions in the proposed method do not exhibit the same regular pattern as in \CHKB method.
It is noticeable in Figure~\ref{fig:RefPicSelectionExample} that for several \acs{ctu} the \PROP method is able to keep the same partitions and reference frames when compared to the optimal encoding (\REF). This can be observed, for example in the small partitions on the left, where the \PROP method uses the same reference frame as \REF. Thus, the proposed method is expected to achieve better coding efficiency than \CHKB, since more optimal coding modes may be chosen.
% TID aqui?
Summarising, by using Equations~(\ref{eq:Lagrangian})~and~(\ref{eq:KFactor}), the cost of using a given reference frame increases as the number of \acs{pu} using it for prediction increases.
However, the reference frames are not hard-selected without taking into account the R-D cost.
In general, it is more difficult to reach a uniform use of reference frames in complex sequences than in simpler ones.
Note that standard compatibility is maintained, since the proposed optimisation method does not require any syntax change. Moreover, since the proposed approach is employed at the Lagrangian optimisation already implemented in the \ac{hevc} reference software, the complexity of the encoder is not significantly affected.
\section{Streaming stage: \ac{mv} selection}
\label{Section4:MVSelection}
\glsreset{tmvp}
Since \ac{hevc} uses differential and predictive \ac{mv} coding, whenever a \ac{mv} is lost the subsequent ones will also be affected, until a refresh point (i.e., \ac{mv} coded on its own, without using \ac{mv} prediction) is reached to break the dependency chain.
Thus, the amount of error propagation strongly depends on whether the \ac{tmvp} is used.
In this streaming stage, to increase the number of possible refresh points and improve error robustness, a small number of \acs{mv} are selected to be transmitted as side information. The selection of these redundant \acs{mv} is based on a trade-off between the image area covered by such \acs{mv} and the number of bits to encode them.
The best tradeoff corresponds to the \acs{mv} that cover the largest image area using the lowest amount of encoding bits.
\def\MVDEPA{\mbox{(a)}\xspace}
\def\MVDEPB{\mbox{(b)}\xspace}
\def\MVDEPC{\mbox{(c)}\xspace}
Figure~\ref{fig:mvdependencies} illustrates how \acs{mv} have different importance in error propagation.
In this figure, each block corresponds to a \ac{pu}, which has an associated \ac{mv}. The arrows point to the \ac{pu} containing the \ac{mv} used as predictor. In the current frame ($f_0$), two \acs{pu} have temporally dependent \acs{mv}.
While the \ac{mv} of \ac{pu} \MVDEPA is used to predict a total of four other \acs{mv}, the \ac{mv} of \ac{pu} \MVDEPB is only used to predict one single \ac{mv}.
Thus, in case of data loss (e.g., frame $f_{-1}$), the \ac{mv} of \ac{pu} \MVDEPA has more impact on error propagation to the frame $f_{0}$ and consequently quality degradation than the \ac{mv} of \ac{pu} \MVDEPB, because of its higher number of dependent \acs{mv}.
For this reason, in the \ac{mv} selection process, the proposed method assigns higher importance to the \ac{mv} of \ac{pu} \MVDEPA than \ac{pu} \MVDEPB. Moreover, in Figure~\ref{fig:mvdependencies}, it can be observed that although the number of \acs{mv} predicted from the \acs{mv} of \ac{pu} \MVDEPC is higher than \ac{pu} \MVDEPA (6 and 4 \acs{mv}, respectively),
the image area that is dependent from the former is smaller that the latter. This is due to the variable size of \acs{pu}.
Therefore, the image area covered by all dependent \acs{pu} is separately considered in the \ac{mv} selection process.
\begin{figure}[tb]
\centering
\input{\ImprovedErrorResilience/Figures/MVDependencies.tikz}
\caption{Example of temporal and spatial \ac{mv} dependencies in \ac{hevc}.}
\label{fig:mvdependencies}
\end{figure}
Based on the evidence mentioned above, the best \acs{mv} to be encoded as side information for each frame are selected through the following procedure:
\begin{enumerate}
\item
the \acs{mv} with temporal dependencies are firstly selected as the most important set: $V$ = \{\emph{\ac{mv}: \ac{mv} is temporally dependent from others}\}.
\item
the $MVs \in V$ are ranked according to the number of spatial dependencies associated to each one.
\item
the optimal sub-set $U \subset V$ is found as the best trade-off between the image area covered by the dependent \acs{pu} and bitrate overhead, i.e., the one that maximises the cost function in Equation (\ref{eq:MVSelectOpt}).
\end{enumerate}
Considering $V$ the ordered set of \acs{mv} encoded for the current frame, i.e., $V=\{MV_1, MV_2, \cdots MV_s\}$ and $U \subset V$ always starting in the first vector, i.e., \mbox{$U_n=\{MV_1, MV_2, \cdots MV_n\}$} with $n\leq s$, the best set of \acs{mv} ($U_n^*)$ is selected based on the optimisation procedure given by Equation (\ref{eq:MVSelectOpt}). Note that any value of $n$ corresponds to a unique subset $U$.
\begin{equation}\label{eq:MVSelectOpt}
\newcommand{\argmax}{\operatornamewithlimits{arg\ max}}
U_n^* = \argmax_{n\leq s} \left\{ \sum_{i=1}^n A_{D}(i) - \alpha \frac{t_C-t_I}{T_{I}} R_U \right\},
\end{equation}
where $R_U$ is the amount of bits required to encode the $n$ \acs{mv} of $U$ and $A_{D}(i)$ is the total area of the \acs{pu} whose \acs{mv} are dependently encoded from $\ac{mv}_i$:
\begin{equation}
A_{D}(i) = \sum_{j=1}^{DV_i} ( w_j\times h_j ),\hspace{2mm} i = 1, 2 \cdots s,
\end{equation}
where $DV_i$ is the number of \acs{mv} dependent from $MV_i$ and $w_j$ and $h_j$ are the width and height of the corresponding \ac{pu}.
A weighting factor is used to control the amount of bitrate overhead ($R_U$), by increasing the cost of the number of encoded bits based on the duration of the error propagation.
It is given by the ratio between the distance from the current frame ($t_c$) to the next refresh frame ($t_I$) and the \ac{idr} period ($T_{I}$), combined with the control parameter $\alpha$.
This weight varies in the range $\left[0;1 \right]$, achieving the lowest value for intra-coded frames and the maximum for the frame located immediately before an intra-coded one.
In the proposed approach, for higher values of $\alpha$, less redundant \acs{mv} are selected.
As pointed out before, this streaming stage operates on coded data and may be implemented at any point of the network, to increase robustness without fully decoding the stream. Note that the parsing operation represented in Figure~\ref{fig:proposedMethod} does not include decoding of the video frames. Furthermore, only one frame delay is introduced because the \ac{mv} selection process is performed on a frame-by-frame basis.
To maintain compatibility with the \ac{hevc} standard, the redundant motion information is transmitted using the \ac{sei} \ac{nal} units \cite{HEVCHighLevelReferenceDez2012}. Thus, this information can be multiplexed into the coded bitstream without affecting the compressed video.
The subset of selected \acs{mv} is independently encoded, ensuring that no reference information is needed in the decoder to properly decode the redundant \acs{mv}. In this work arithmetic coding~\cite{ArithmeticCodingJun1994} was used for the side-information, but any other entropy coding method can be used.
\section{Performance evaluation}
\label{Section4:Performance}
In this section the robustness of the proposed two-stage architecture is evaluated and discussed for two cases: (i) only using the encoding stage and (ii) using the streaming stage in addition to the encoding stage. In the former case, identified as ``\PROP'', only the dynamic reference picture selection method described in Section~\ref{Section4:RefPicSel} is evaluated. In the latter case, identified as ``\DYN'', besides the previous method of reference picture selection, redundant \acs{mv} are encoded as described in Section~\ref{Section4:MVSelection}, resulting in a combination of both stages.
As references for comparison, three other methods are used: (i) a fixed checkerboard approach (\CHKB), (ii) the reference picture generation method based on leaky predictions proposed in \cite{ROPERefGenJan2010}, identified as ``\AVG'', and (iii) long-term reference frames (\LONG). The \LONG method only uses the key frames as reference, i.e., in the \ac{ra} configuration only one out of eight frames is used as reference for prediction, allowing a more robust transmission, as the reference frames are less likely to be hit by errors.
The experimental setup used in this section is the same as the one previously described in Section~\ref{Section3:ErrorResilienceEvaluation}.
Two test cases were considered for simulation: (i) in \emph{\LOSTALL} all frames may be equally affected by packet loss, including \ac{idr} frames; (ii) in \emph{\KEEPINTRA} \ac{idr} frames are assumed to be prioritised in the network and delivered error-free, or using some combined approach as recently proposed in \cite{ErrorResilienceHeterogenousNetworks2016}.
At the decoder side, error concealment is based on motion compensation from previously decoded frames. The motion field of a missing frame is obtained applying \ac{mv} extrapolation from the two closest neighbours. Afterwards, the final \acs{mv} are selected based on the residual energy of their original prediction~\cite{HEVCMCResidueDec2013}.
The evaluation of the encoding-stage is carried out separately from the streaming-stage.
%This section presents an evaluation study about the error robustness of HEVC using different sequences and coding configurations. Table~\ref{tab:sequences} presents a summary of the main characteristics of the video sequences used in this work. Each sequence comprises 240 frames and the whole set covers different types of motion and texture complexity. This table also shows the spatial information (SI) and the temporal information (TI), as defined in \cite{ITUTP9111998}. The experimental results for this study were obtained by using the HM reference software, version $16.2$.
%The sequences were encoded with all coding modes enabled and following two recommended prediction structures: Low-Delay (P-frames with 4 references frames) and \ac{ra} (B-frames with 2 reference frames in each list)~\cite{HEVCCommonTests2012}. These prediction structures cover the use of P-frames and hierarchical B-frames~\cite{H264ScalableOverviewSep2007}, which are commonly used in HEVC streaming~\cite{RealTimeHEVC2016}.
%To simulate a broadcast transmission an \ac{idr} period of $32$ frames was used and the filtering operations were disabled at slices boundaries in order to keep the slices self-contained. The remaining configurations have been kept at their default values as given in the common test conditions~\cite{HEVCCommonTests2012}.
%In order to achieve high quality images for all coded sequences (i.e., PSNR around 40 dB) different bitrates were used.
%At the decoder side, error concealment is based on motion compensation from previously decoded frames. The motion field of a missing frame is obtained applying MV extrapolation from the two closest neighbours. Afterwards, the final MVs are selected based on the residual energy of their original prediction~\cite{HEVCMCResidueDec2013}.
%To simulate the lossy transmission environment, each frame is encoded into several slices, each one packetized into one NAL unit, which are then transmitted as the payload of independent packets.
%
%In our experiments a whole packet is lost whenever an error occurs, originating a lost slice. Random packet loss was simulated using a two-state Markov model with the following average burst length 1.24, 1.47, 1.83 and 2.05, and a maximum burst length of 2, 3, 5 and 7, for packet loss ratios (PLR) of 1\%, 3\%, 5\%, 10\%, respectively.
\subsection{Encoding stage}
The performance of the encoding stage is evaluated on its own in order to only determine the improvements achieved by the dynamic reference frame selection method.
In this case, the reference frames were reconstructed with distortion, as a result of non-perfect error concealment.
Since the encoding stage does not deal with errors in temporal \ac{mv} predictions (this is handled by the streaming-stage), in this evaluation the temporal \ac{mv} predictor is disabled.
This is necessary to avoid masking the results with the errors caused by wrong \ac{mv} predictions.
The performance of the encoding stage (\PROP) was evaluated using $\gamma = 5$.
This value was chosen to allow increasing the Lagrangian cost by about $50\%$, when a reference frame is used for prediction of more than three \acs{ctu} in comparison to the usage of the other reference frames (see Figure~\ref{fig:weightfct}). Nevertheless, further results for a wide range of $\gamma$ values are also presented.
%\HERE
\subsubsection{Quality evaluation under error-prone conditions}
\begin{figure}[tb]
\centering
\footnotesize
\newcommand{\CurrFigSize}{0.23\columnwidth}
\resizebox{\textwidth}{!}{
\begin{tabular}{y{.23\columnwidth}y{.23\columnwidth}|y{.23\columnwidth}y{.23\columnwidth}}
\multicolumn{2}{c|}{Basketball Drill} & \multicolumn{2}{c}{Race Horses} \\
\AVG & \PROP & \AVG & \PROP \\
\includegraphics[width=\CurrFigSize]{Figure_6_1} &
\includegraphics[width=\CurrFigSize]{Figure_6_2} &
\includegraphics[width=\CurrFigSize]{Figure_6_3} &
\includegraphics[width=\CurrFigSize]{Figure_6_4} \\
PSNR = 31.93 dB & PSNR = 33.83 dB & PSNR = 24.12 dB & PSNR = 31.93 dB \\
\end{tabular}
}
\caption{Zoom of recovered Frame \#10 after a loss event at Frame \#9 for Basketball Drill and Race Horses sequence.}
\label{fig:RecoveredExample}
\end{figure}
The error robustness achieved by the encoding-stage was firstly evaluated in regards to the error propagation resulting from a single loss event in the \ac{ld} configuration.
Figure~\ref{fig:RecoveredExample} shows the visual impact of error propagation on a region of the decoded Frame \#10 when Frame \#9 is affected by errors, for Basketball Drill and Race Horses sequences (the \ac{psnr} corresponds to the entire frame).
The proposed method is compared with the method~in~\cite{ROPERefGenJan2010} as both aim at reducing the error propagation.
The method~in~\cite{ROPERefGenJan2010} uses filtered frames as reference for predictions, therefore, in case of errors, the effects of mismatch predictions are significantly reduced by the filtering operation (see Sub-Section~\ref{SECTION2:LeakyPred}).
As shown in this figure, the proposed method is able to reduce the impact of mismatch decoding in comparison with the method~in~\cite{ROPERefGenJan2010}. Although both methods still reveal some predictions mismatch leading to inaccurate reconstructed of the objects contours, the proposed method is able to achieve a smoother reconstruction, which leads to higher quality in terms of \ac{psnr}. This is because, in frame \#10, the predictions depending on the missing frame \#9 are not used throughout the entire frame, thus, approximately half of the predictions are not affected by errors. In contrast, the method~in~\cite{ROPERefGenJan2010} requires several filtering operations to be applied (i.e, different reference frames need to be filtered) in order to reduce the effect of mismatch predictions.
Summarising, the reconstructed frames illustrated in Figure~\ref{fig:RecoveredExample} reveal the higher efficiency achieved by distributing the use of the references frames.
\begin{figure}[t]
\renewcommand{\FIGSIZE}{0.8\columnwidth}
\centering
% \includegraphics[width=\FIGSIZE]{\ImprovedErrorResilience/Results/PSNRversusFrameKendo}
\input{041_ImprovedErrorResilience/Results/PSNRversusFrameKendo.tex}
\caption{Error propagation when errors affect Frame \#7 in Kendo sequence with an \ac{idr} period of 32 frames.}
\label{fig:PSNRvsFrameForRefPicSel}
\end{figure}
Figure~\ref{fig:PSNRvsFrameForRefPicSel}~shows the \ac{psnr} over a \ac{gop} with an \ac{idr} period of 32 frames, using the same constant bitrate for all streams.
This figure shows that all methods are able to outperform the reference \ac{hevc} (\REF), gaining approximately $3$~dB in reconstruction quality.
The similar quality levels achieved by \CHKB and \PROP reveal that the dynamic selection of the reference frames used in \PROP is more robust because a better trade-off between coding efficiency and error resilience is attained.
Moreover, results in Figure~\ref{fig:PSNRvsFrameForRefPicSel} are in line with the reconstruction quality shown in Figure~\ref{fig:RecoveredExample}, confirming that the proposed method is able to outperform the reference frame generation technique in~\cite{ROPERefGenJan2010}.
%Since the proposed method is more efficient than the others for error-free transmission (see~Table~\ref{tab:bd_error_free}), and it also provides better error robustness, it can be considered as an efficient solution to better deal with transmission errors.
Overall, the method proposed for the encoding stage is able to increase the error robustness of \ac{hevc} without severely compromising the coding efficiency.
\input{./041_ImprovedErrorResilience/Results/TableResultsRandomLossRefPicSelection.tex}
Further tests were run to evaluate the effectiveness of the proposed method under various packet loss rates for both test cases, i.e., with and without \ac{idr} frame loss, \LOSTALL and \KEEPINTRA, respectively.
The use of long-term reference pictures (\LONG) is only tested for the \ac{ra} configuration, where key frames are available, while method \cite{ROPERefGenJan2010} is tested for the \ac{ld} configuration.
For each test condition, 50 trials were performed and the average quality (\ac{psnr}) obtained across all trials for both coding configurations are shown in Table~\ref{tab:AverageQualityCHKBEXPLowDelay}~and~\ref{tab:AverageQualityCHKBEXPRandomAccess}, for the \ac{ld} and \ac{ra}, respectively.
In both tables the absolute \ac{psnr} is shown for the reference \ac{hevc} case (\REF), while the \ac{psnr} difference is presented for the other cases.
Results show that the proposed method is able to outperform other methods for the same loss conditions.
An average quality gain up to $2.26$~dB is obtained for the \ac{ld} configuration (see Table~\ref{tab:AverageQualityCHKBEXPLowDelay}), in comparison to the \REF, while the fixed checkerboard pattern approach (\CHKB) only achieves up to $2.15$~dB.
In \cite{ROPERefGenJan2010}, the use of interpolated references in the encoding loop helps to reduce error propagation, at a cost of poor predictions, thus degrading the coding efficiency. This is the main reason that justifies a lower overall performance of the method proposed in \cite{ROPERefGenJan2010} (see results for ``\AVG'').
The proposed method is also able to outperform the \ac{hevc} reference software when an hierarchical coding structure is used (see results for \ac{ra} configuration in Table~\ref{tab:AverageQualityCHKBEXPRandomAccess}), which implicitly limits the error propagation.
In comparison to the reference \ac{hevc}, quality improvements up to $1.42$~dB at PLR=$10\%$ are obtained for Kendo sequence.%, while the average gain across all test sequences is 1~dB.
When comparing results for both test cases, it is noticeable that the proposed method is able to outperform the reference methods both when \ac{idr} frame loss is allowed or not (\LOSTALL and \KEEPINTRA, respectively). As the \ac{plr} increases, the gains of the proposed method decrease in comparison with the case where the \ac{idr} frames are delivered error-free (i.e., case \KEEPINTRA), due to the higher error propagation of the missing \ac{idr} frame. Moreover, the quality gains are lower in the \LOSTALL case when comparing with \KEEPINTRA for the same \ac{plr}, for both the \ac{ld} and \ac{ra}.
Taking into account the above results and the spatio-temporal information shown in Table~\ref{TableTestSequences}, one can observe that the performance of the proposed method is lower for sequences with high spatial details and low temporal activity (e.g., BQSquare and Park Scene), which indicates higher effectiveness for video content with higher motion activity.
%\HIGHLIGHT{complete with analysis of both cases}
\begin{figure}[!b]
\renewcommand{\FIGSIZE}{0.8\columnwidth}
\centering
%\includegraphics[width=\FIGSIZE,page=1]{\ImprovedErrorResilience/Results/PSNRversusGama}
%\includegraphics[width=\FIGSIZE,page=2]{\ImprovedErrorResilience/Results/PSNRversusGama}
\input{./041_ImprovedErrorResilience/Results/PSNRversusGama.tex}
\caption{Influence of the $\gamma$ parameter in the average quality gains, comparing with \REF, for different \acs{plr}.}
\label{fig:QualityVsGamma}
\end{figure}
\subsubsection{Influence of parameter $\gamma$}
The influence of the $\gamma$ parameter used in Equation (\ref{eq:KFactor}) on the video quality was also evaluated, in order to check how the cost constrains affect the performance of the proposed method.
Figure \ref{fig:QualityVsGamma} shows the average quality gains of the proposed method in comparison to the reference \ac{hevc} (\REF) across all test sequences, for different values of $\gamma$.
For the error-free case (PLR=0\%), the results in this figure show that increasing $\gamma$ leads to a decrease in the coding efficiency, especially in the \ac{ld} configuration.
The reason is that higher values of $\gamma$ lead to higher values of $W(r_{\phi})$, therefore the reference picture selection cost is severely constrained by the exponential weights.
Since $T_{ID}$ is constant for all frames in the \ac{ld}, higher weights are used for all frames ($T_{ID}$ is used to reduce the weights in case of hierarchical coding) and the encoder is not able to select the best reference frame in \c{rd} sense.
In the case where packet loss occurs, higher values of $\gamma$ lead to improved error resilience.
Since higher weights are used in the \ac{rd} cost, the encoder is constrained to distribute the use of the reference frames of different \acs{pu}, reducing the error propagation.
For very high values (e.g., $\gamma=1000$) the encoder is forced to uniformly select the reference frames, resulting in a fixed checkerboard pattern.
Overall, the $\gamma$ parameter is able to control the reference frame selection constrains,
%, which are selected by a modified \ac{rd} optimisation, ,
thus achieving better trade-off between improved error resilience and loss of coding efficiency.
\subsubsection{BD-PSNR under error-free conditions}
This sub-section evaluates the \ac{rd} penalty incurred by the proposed method in comparison with \CHKB, as well as the other reference methods.
Table~\ref{tab:bd_error_free} shows the Bjontegaard's Delta PSNR (BD-PSNR) values compared to the reference \ac{hevc} encoder for different coding configurations in error-free decoding. Four \acs{qp} were used from the common test conditions: 22, 27, 32, 37.
\begin{table}[tb]
\centering
\newcommand{\resultswidth}{1.5cm}
\caption{BD-PSNR (dB) in error-free conditions.}
\TABLESVSPACE
\footnotesize
\label{tab:bd_error_free}
\begin{tabular}{p{3cm}|x{\resultswidth}|x{\resultswidth}|x{\resultswidth}|x{\resultswidth}|x{\resultswidth}|x{\resultswidth}}
\toprule
\multirow{2}{*}{\centering\textbf{Sequence}} & \multicolumn{3}{c|}{\textbf{Low-Delay}} & \multicolumn{3}{c}{\textbf{Random-Access}} \\
& \AVG & \CHKB & \PROP & \LONG & \CHKB & \PROP \\ \hline
Basketball Drill & -1.14 & -1.15 & -0.51 & -0.85 & -0.29 & -0.22 \\ \hline
Kendo & -2.10 & -1.39 & -0.99 & -0.91 & -0.39 & -0.28 \\ \hline
Park Scene & -1.79 & -1.38 & -0.81 & -0.68 & -0.38 & -0.28 \\ \hline
Traffic & -1.91 & -1.21 & -0.71 & -0.55 & -0.36 & -0.21 \\ \hline
Bosphorus & -0.97 & -0.95 & -1.10 & -0.64 & -0.20 & -0.20 \\ \hline
\textbf{Average} & \textbf{-1.58} & \textbf{-1.22} & \textbf{-0.82} & \textbf{-0.73} & \textbf{-0.32} & \textbf{-0.24} \\
\bottomrule
\end{tabular}
\end{table}
The results in Table~\ref{tab:bd_error_free} indicate a small loss of coding efficiency of the proposed method, as well as, the reference methods tested.
The fixed checkerboard approach leads to an average quality reduction of $1.22$~dB for the \ac{ld} ($0.32$~dB for \ac{ra}), while the proposed method only loses $0.82$~dB and $0.24$~dB in the \ac{ld} and \ac{ra} configurations, respectively. This results clearly shows that dynamically selecting the reference frames is more efficient that using a fixed pattern, confirming the previous results for error-prone conditions.
Moreover, in comparison to \LONG and the method in~\cite{ROPERefGenJan2010}, the proposed method is still more efficient, consistently achieving better results. Comparing, the results for both error-free and error-prone conditions, one may conclude that the proposed method not only achieves higher quality gains, but also does reduces the impact on coding efficiency.
Overall, the loss in the quality compared to the reference \ac{hevc} is considered acceptable, given the increase in robustness obtained in lossy transmission demonstrated in the previous sub-sections.
%As expected, this is due to the sub-optimal \ac{rd} optimisation of the proposed method, because the cost function also includes the cost associated to the choice of the reference frame, in addition to the rate and distortion.
\subsubsection{Complexity overhead}
The implementation of this method slightly increases the complexity of a standard non-robust encoder. Therefore, the computational complexity has also been considered as a performance metric in this research, which is measured as the encoding time in a controlled hardware platform. The reference used for comparison with other methods is the \ac{hevc} reference software encoding time, running on the same platform with the same coding configuration.
The relative complexity increase of the robust encoder using \PROP, \CHKB and the method presented in \cite{ROPERefGenJan2010} is shown in Table~\ref{tab:cpu_time}.
As shown in this table, the method presented in \cite{ROPERefGenJan2010} is the most time consuming because it requires post-processing of the encoded frames in order to interpolate new references.
The proposed scheme, using a reference picture selection, presents a lower complexity increase than \cite{ROPERefGenJan2010}, while the fixed checkerboard scheme (\CHKB) presents a slightly lower complexity \PROP method.
This is due to the fact that in the \CHKB method there is no \ac{rd} optimisation process for the reference frame selection (fixed pattern is used), thus reducing the overall computational complexity.
\begin{table}[tb]
\newcommand{\resultswidth}{2.2cm}
\centering
\caption{Relative CPU times of tested methods.}\TABLESVSPACE
\label{tab:cpu_time}
\footnotesize
\begin{tabular}{p{4cm}|x{\resultswidth}|x{\resultswidth}|x{\resultswidth}}
\toprule
\multirow{2}{*}{\centering\textbf{Sequence}} & \multicolumn{3}{c}{\textbf{Method}} \\
& \AVG & \CHKB & \PROP \\ \hline
Basketball Drill & +4.19\% & +2.13\% & +2.93\% \\ \hline
Kendo & +4.96\% & +1.83\% & +2.05\% \\ \hline
Park Scene & +4.98\% & +3.90\% & +4.04\% \\ \hline
Traffic & +8.65\% & +2.28\% & +3.05\% \\
\bottomrule
\end{tabular}
\end{table}
\subsection{Streaming stage}
In this section the coding performance that results from transmitting redundant \ac{mv} (streaming stage) in combination with the reference picture selection scheme (encoding stage) is presented.
In these experimental tests the temporal \ac{mv} candidates are enabled and \acs{mv} are dynamically added as side-information into the stream, as described in Section \ref{Section4:MVSelection}.
Different values of $\alpha$ are used to achieve different overhead ratios. All streams were encoded with the same total bitrate, including the redundant bits, in order to make a fair quality comparison.
\subsubsection{Dynamic MV selection performance}
\begin{figure}[!b]
\centering
\newcommand{\CurrFigSize}{0.49\columnwidth}
\includegraphics[width=\CurrFigSize,page=1]{\ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf}
\includegraphics[width=\CurrFigSize,page=2]{\ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf}
\includegraphics[width=\CurrFigSize,page=3]{\ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf}
\includegraphics[width=\CurrFigSize,page=4]{\ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf}
\includegraphics[width=\CurrFigSize,page=5]{\ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf}
\includegraphics[width=\CurrFigSize,page=6]{\ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf}
\caption{Decoded video quality for different amounts of redundancy, selected using a fixed approach and the proposed method.}
\label{fig:QualityVsRedundancyMVSelection}
\end{figure}
Firstly the quality obtained by the proposed method, which dynamically selecting the amount of redundant \acs{mv}, is compared to cases where a fixed amount of \acs{mv} per frame are transmitted (i.e., the most important 10\%, 30\% and 50\% $\ac{mv} \in V$). This cased are referred to as "\DYN" and "\FIXED", respectively.
%In the case where variable redundancy is allowed, \acs{mv} are selected using the proposed dynamic method (Dyn) and a fixed approach (Fixed).
Figure~\ref{fig:QualityVsRedundancyMVSelection} illustrates the relation between the quality obtained (\ac{psnr}) and the amount of redundancy used.
The results indicate that for approximately the same amount of redundant bits are used in both cases, however by using the proposed method to dynamically select the \acs{mv} to be used as side-information (\DYN) is able to outperform the fixed case for all test sequences.
Results also show that for both approaches the decoded video quality increases as the redundancy of the side-information increases, since a higher number of mismatch \ac{mv} predictions are able to be recovered.
It is noticeable in Figure~\ref{fig:QualityVsRedundancyMVSelection} that for higher \acs{plr}, the proposed method is able to achieve higher gains. This reveals that for increasing data loss it is more important optimise the amount of \acs{mv} selected per frame in order to use more \acs{mv} in those frames which have a higher impact of the error propagation.
For instance, the proposed method is able to gain up to $1.5$~dB for the BQSquare sequence for a PLR=10\$.
For lower \acs{plr}, since fewer frames are affected by data losses, the use of a fixed approach achieves similar performance as the proposed method (\DYN).
\subsubsection{Quality evaluation}
Table~\ref{tab:AverageQualityRedundancies} presents the average \ac{psnr} of decoded video for different \ac{plr} (columns~4~to~8) and for various percentages of \ac{mv} redundancy (third column). The absolute quality values are shown for the reference case (\NOREDUND) and \ac{psnr} gains are shown for thee different levels of redundancy introduced by the proposed method (\DYN).
This study also conducted for the two different cases, with and without \ac{idr} loss.
It can be confirmed from the results of Table~\ref{tab:AverageQualityRedundancies} that, for each sequence, as $\alpha$ decreases more redundant \acs{mv} are used leading to higher overhead rations. This results in higher objective video quality for all tested sequences.
This confirms that redundant \acs{mv} are able to minimise the error propagation, which is accomplished at a cost of an acceptable increase in the bitrate, since only a sub-set of \acs{mv} is transmitted.
The use of redundant \acs{mv} yields better video quality for \ac{plr} as low as $1\%$, achieving higher quality gains when \ac{idr} frames are not lost (\KEEPINTRA). This is because of the higher quality degradation when \ac{idr} frames are lost, which is not due to mismatched \ac{mv} predictions and thus cannot be reduced by transmitting extra \acs{mv} as side-information.
The use the proposed method achieves higher quality for sequences with higher motion activity and which makes use of inter-prediction more often, i.e., BQSquare and Traffic sequences (see inter ratio in Table~\ref{TableTestSequences}), leading to a maximum gain of $3.93$~dB for \LOSTALL ($7.82$~dB for \KEEPINTRA).
\begin{table}[!t]
\centering
\caption{Average quality using redundant \ac{mv} selected by the proposed approach.}
\TABLESVSPACE
\label{tab:AverageQualityRedundancies}
\TABLESVSPACE
\input{\ImprovedErrorResilience/Results/TableResultsRandomLossMvRedundancies.tex}
\end{table}
In summary, the proposed method is able to select the most relevant motion \ac{mv} information and when combined with a reference picture selection scheme the problem of mismatch \ac{mv} predictions is mitigated.
Moreover, the dynamic approach devised to select the best \acs{mv} also contributes to increase the overall performance.
Thus, using the proposed two-stage approach to recover temporal predictions consistently improves the error robustness of \ac{hevc}.
\section{Summary}
\label{Section4:Summary}
In this chapter a two-stage approach was proposed to increase the error robustness of \ac{hevc} streaming and reduce the error propagation in case of packet loss.
A constrained coding approach was devised to select reference frames and dynamically distribute temporal dependencies.
This is jointly used with a controlled amount of side information in coded streams, comprising a small set of the most relevant \acs{mv}, to minimise \ac{mv} mismatch at the decoder in the presence of frame loss.
As can be concluded from the results, this method contributes to reduce the temporal dependencies with consistent quality gains for different \ac{plr} and coding configurations, only incurring in a small drop of coding efficiency.
Overall, the proposed approach is an effective method for coping with video transmission over error-prone networks.
Reference in New Issue View Git Blame Copy Permalink