Review all chapters and start conclusions
This commit is contained in:
1 parent
461b560b17
commit
8d3a4c72ff
20 files changed
+304
-427
No files matched your search
@@ -51,7 +51,7 @@ In order to evaluate the reference frame selection method attained by the \ac{rd
|
||||
Figure~\ref{FIG4:ReferenceUsageRatio} illustrates the average usage percentage of reference $r_1$ for different sequences.
|
||||
The remaining corresponds to those cases where reference $r_2$ is selected.
|
||||
These results show that, in general, reference $r_1$ is selected more often than $r_2$.
|
||||
This is mainly true for sequences with higher motion.
|
||||
This is mainly true for sequences with higher motion.
|
||||
For those sequences with lower motion, but high spatial complexity, e.g., BQSquare (see \ac{si} in Table~\ref{TableTestSequences}), both reference frames are equally (approximately) selected for temporal predictions, with $r_1$ being used by 58\% of cases and $r_2$ by 42\%.
|
||||
This reveals that even with a configuration using only two references the encoder mainly uses the temporally closer one as the reference for inter-frame predictions.
|
||||
Therefore, if such reference frame is lost, a higher number of mismatched predictions will occur, inevitabely leading to higher error propagation.
|
||||
@@ -59,7 +59,7 @@ Therefore, if such reference frame is lost, a higher number of mismatched predic
|
||||
Another technique that contributes to increased error propagation is the use of predictions in \ac{mv} coding~\cite{MotionCodingHEVC2011}, which contributes to increase the spatial and temporal dependencies.
|
||||
Although the spatial \ac{mv} candidates are used more often, temporally predicted \acs{mv} are responsible for a greater impact on error propagation, as they propagate errors in the \ac{mv} decoding across different time instances.
|
||||
|
||||
To overcome these issues, in the next sections a two-fold approach is described to increase the error robustness in video streaming by jointly considering two different stages: (i) while encoding live video, by selecting robust coding modes and (ii) after encoding, by adding robust features to the compressed streams. While in the former this is accomplished by optimising coding parameters and decisions to increase robustness, in the latter some type of compressed domain processing is used to increase error robustness of pre-encoded streams, e.g., error resilient transcoding. Inevitably, in both cases this is achieved at the expense of some loss in coding efficiency, therefore the challenge is to find an optimum trade-off between robustness and bitrate overhead.
|
||||
To overcome these issues, in the next sections a two-fold approach is described to increase the error robustness in video streaming by jointly considering two different stages: (i) while encoding live video, by selecting robust coding modes and (ii) after encoding, by adding robust features to the compressed streams. While in the former this is accomplished by optimising coding parameters and decisions to increase robustness, in the latter some type of compressed domain processing is used to increase error robustness of pre-encoded streams, e.g., error resilient transcoding. Inevitably, in both cases this is achieved at the expense of some loss in coding efficiency, therefore the challenge is to find an optimum trade-off between robustness and bitrate overhead.
|
||||
%In this chapter a two-fold approach is used to overcome the two problems mentioned above.
|
||||
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
@@ -76,8 +76,8 @@ D_t = \tilde f_t - \hat f_t, \text{when } \tilde f_{t-1} \neq \hat f_{t-1},
|
||||
\end{equation}
|
||||
where $\hat f_t$ and $\tilde f_t$ are the encoded and decoded frames, respectively, at instant $t$.
|
||||
|
||||
Figure \ref{fig:proposedMethod} illustrates the block diagram of the two-stage architecture proposed in this work.
|
||||
At the encoding stage, a reference frame selection mechanism is used for each \ac{pu} within a \ac{ctu}, by forcing the use of different reference frames for adjacent \acs{ctu}.
|
||||
Figure~\ref{fig:proposedMethod} illustrates the block diagram of the two-stage architecture proposed in this work.
|
||||
At the encoding stage, a reference frame selection mechanism is used for each \ac{pu} within a \ac{ctu}, by forcing the use of different reference frames for adjacent \acs{ctu}.
|
||||
The decision criterion contributes to reduce the amount of predictions from a single reference frame and forces to interleave different reference frames along all encoded \acs{pu}, based on \ac{rd} optimisation.
|
||||
At the streaming stage, the coded stream is parsed to extract the \ac{mv} information, which allows ranking \acs{mv} according to a relevance criterion. Then, a sub-set of these \acs{mv} is selected to be transmitted as side information. The amount of selected \acs{mv} is not fixed and can be optimised for different content and network characteristics.
|
||||
Note that the streaming stage is an independent and complementary subsystem of the encoder. Thus, it is able to operate at network nodes, including streaming servers delivering pre-encoded streams.
|
||||
@@ -108,7 +108,7 @@ In a coding structure with two reference frames, as shown in Figure~\ref{fig:che
|
||||
If each reference frame is used to predict half of the pixels in the current frame, when any of these references is lost, the subsequent predicted frame is only partially distorted by mismatched predictions.
|
||||
This is because only half of its predictions depend on the missing reference frame, which limits the overall error propagation.
|
||||
Moreover, since the motion information of the predicted frame is encoded using as reference not only the missing frame but also the past ones, those \acs{mv} that cross over a missing reference frame, pointing to the previous ones, are still useful for reconstructing correct \acs{mv} and also for error concealment.
|
||||
In the limit, a straightforward implementation of the previous example leads to a simple interleaved uniform distribution of the reference frames over all \acs{ctu} in the current frame, forcing the checkerboard structure shown in Figure \ref{fig:checkerboardRefSelection}. In the case of two reference frames, 50\% of the \acs{ctu} would use one reference frame, while the other 50\% would use the other reference.
|
||||
In the limit, a straightforward implementation of the previous example leads to a simple interleaved uniform distribution of the reference frames over all \acs{ctu} in the current frame, forcing the checkerboard structure shown in Figure~\ref{fig:checkerboardRefSelection}. In the case of two reference frames, 50\% of the \acs{ctu} would use one reference frame, while the other 50\% would use the other reference.
|
||||
However, such solution would not take into consideration coding efficiency, since the selection process of the reference frame would not be \ac{rd} optimised for each \ac{pu} within a \ac{ctu}.
|
||||
|
||||
\begin{figure}[tb]
|
||||
@@ -372,7 +372,7 @@ Taking into account the above results and the spatio-temporal information shown
|
||||
|
||||
|
||||
The influence of the $\gamma$ parameter used in Equation (\ref{eq:KFactor}) on the video quality was also evaluated, in order to check how the cost constrains affect the performance of the proposed method.
|
||||
Figure \ref{fig:QualityVsGamma} shows the average quality gains of the proposed method in comparison to the reference \ac{hevc} (\REF) across all test sequences, for different values of $\gamma$.
|
||||
Figure~\ref{fig:QualityVsGamma} shows the average quality gains of the proposed method in comparison to the reference \ac{hevc} (\REF) across all test sequences, for different values of $\gamma$.
|
||||
For the error-free case (PLR=0\%), the results in this figure show that increasing $\gamma$ leads to a decrease in the coding efficiency, especially in the \ac{ld} configuration.
|
||||
The reason is that higher values of $\gamma$ lead to higher values of $W(r_{\phi})$, therefore the reference picture selection cost is severely constrained by the exponential weights.
|
||||
Since $T_{ID}$ is constant for all frames in the \ac{ld}, higher weights are used for all frames ($T_{ID}$ is used to reduce the weights in case of hierarchical coding) and the encoder is not able to select the best reference frame in the \ac{rd} sense.
|
||||
@@ -451,7 +451,7 @@ This is due to the fact that in the \CHKB method there is no \ac{rd} optimisatio
|
||||
\subsection{Streaming stage}
|
||||
|
||||
In this section the coding performance that results from transmitting redundant \ac{mv} (streaming stage) in combination with the reference picture selection scheme (encoding stage) is presented.
|
||||
In these experimental tests the temporal \ac{mv} candidates are enabled and \acs{mv} are dynamically added as side-information into the stream, as described in Section \ref{Section4:MVSelection}.
|
||||
In these experimental tests the temporal \ac{mv} candidates are enabled and \acs{mv} are dynamically added as side-information into the stream, as described in Section~\ref{Section4:MVSelection}.
|
||||
Different values of $\alpha$ are used to achieve different overhead ratios. All streams were encoded with the same total bitrate, including the redundant bits, in order to make a fair quality comparison.
|
||||
|
||||
|
||||
@@ -464,7 +464,7 @@ Figure~\ref{fig:QualityVsRedundancyMVSelection} illustrates the relation between
|
||||
The results indicate that for approximately the same amount of redundant bits are used in both cases, however by using the proposed method to dynamically select the \acs{mv} to be used as side-information (\DYN) is able to outperform the fixed case for all test sequences.
|
||||
Results also show that for both approaches the decoded video quality increases as the redundancy of the side-information increases, since a higher number of mismatch \ac{mv} predictions are able to be recovered.
|
||||
|
||||
\begin{figure}[tb]
|
||||
\begin{figure}[!t]
|
||||
\centering
|
||||
\newcommand{\CurrFigSize}{0.49\columnwidth}
|
||||
\includegraphics[width=\CurrFigSize,page=1]{\ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf}
|
||||
@@ -477,14 +477,12 @@ Results also show that for both approaches the decoded video quality increases a
|
||||
\label{fig:QualityVsRedundancyMVSelection}
|
||||
\end{figure}
|
||||
|
||||
It is noticeable in Figure~\ref{fig:QualityVsRedundancyMVSelection} that for higher \acs{plr}, the proposed method is able to achieve higher gains.
|
||||
It is noticeable in Figure~\ref{fig:QualityVsRedundancyMVSelection} that for higher \acs{plr}, the proposed method is able to achieve higher gains.
|
||||
This reveals that when data loss increases it is more important optimise the amount of \acs{mv} selected per frame, in order to use more \acs{mv} in those frames that have a higher impact on the error propagation.
|
||||
For instance, the proposed method is able to gain up to $1.5$~dB for the BQSquare sequence subject to a PLR=10\%.
|
||||
For lower \acs{plr}, since fewer frames are affected by data losses, the use of a fixed approach achieves similar performance as the proposed method (\DYN).
|
||||
|
||||
|
||||
|
||||
|
||||
\subsubsection{Quality evaluation}
|
||||
|
||||
Table~\ref{tab:AverageQualityRedundancies} presents the average \ac{psnr} of decoded video for different \ac{plr} (columns~4~to~8) and for various percentages of \ac{mv} redundancy (third column). The absolute quality values are shown for the reference case (\NOREDUND) and \ac{psnr} gains are shown for three different levels of redundancy introduced by the proposed method (\DYN).
|
||||
@@ -495,6 +493,10 @@ This confirms that redundant \acs{mv} are able to minimise the error propagation
|
||||
The use of redundant \acs{mv} yields better video quality for \ac{plr} as low as $1\%$, achieving higher quality gains when \ac{idr} frames are not lost (\KEEPINTRA). This is because of the higher quality degradation when \ac{idr} frames are lost, which is not due to mismatched \ac{mv} predictions, and thus cannot be reduced by transmitting extra \acs{mv} as side-information.
|
||||
The use the proposed method also achieves higher quality for sequences with higher motion activity that makes use of inter-prediction more often, i.e., BQSquare and Traffic sequences (see Inter ratio in Table~\ref{TableTestSequences}), leading to a maximum gain of $3.93$~dB for \LOSTALL ($7.82$~dB for \KEEPINTRA).
|
||||
|
||||
In summary, the proposed method is able to select the most relevant motion \ac{mv} information and when combined with a reference picture selection scheme the problem of mismatch \ac{mv} predictions is mitigated.
|
||||
Moreover, the dynamic approach devised to select the best \acs{mv} also contributes to increase the overall performance.
|
||||
Thus, using the proposed two-stage approach to recover temporal predictions consistently improves the error robustness of \ac{hevc}.
|
||||
|
||||
\begin{table}[!t]
|
||||
\centering
|
||||
\caption{Average quality using redundant \ac{mv} selected by the proposed approach.}
|
||||
@@ -504,21 +506,15 @@ The use the proposed method also achieves higher quality for sequences with high
|
||||
\input{\ImprovedErrorResilience/Results/TableResultsRandomLossMvRedundancies.tex}
|
||||
\end{table}
|
||||
|
||||
In summary, the proposed method is able to select the most relevant motion \ac{mv} information and when combined with a reference picture selection scheme the problem of mismatch \ac{mv} predictions is mitigated.
|
||||
Moreover, the dynamic approach devised to select the best \acs{mv} also contributes to increase the overall performance.
|
||||
Thus, using the proposed two-stage approach to recover temporal predictions consistently improves the error robustness of \ac{hevc}.
|
||||
|
||||
\section{Summary}
|
||||
\label{Section4:Summary}
|
||||
|
||||
In this chapter a two-stage approach was proposed to increase the error robustness of \ac{hevc} streaming and reduce the error propagation in case of packet loss.
|
||||
A constrained coding approach was devised to select reference frames and dynamically distribute temporal dependencies.
|
||||
This is jointly used with a controlled amount of side information in coded streams, comprising a small set of the most relevant \acs{mv}, in order to minimise \ac{mv} mismatch at the decoder in the presence of frame loss.
|
||||
This is jointly used with a controlled amount of side information in coded streams, comprising a small set of the most relevant \acs{mv}, in order to minimise \ac{mv} mismatch at the decoder in the presence of frame loss. The use of \acs{mv} as side information was firstly proposed and evaluated in the publication~\ref{Conf:MotionVectorRedundancies} and the reference frame selection was further discussed in the publication~\ref{Conf:Chkb}. Subsequently, a wider analysis was carried out and published in \ref{Journal:TwoStage}.
|
||||
As can be concluded from the results, this method contributes to reduce the temporal dependencies with consistent quality gains for different \ac{plr} and coding configurations, only incurring in a small drop of coding efficiency.
|
||||
Overall, the proposed approach is an effective method to cope with video transmission over error-prone networks.
|
||||
|
||||
|
||||
\todo{Papers here??}
|
||||
|
||||
|
||||
|
||||
Reference in new issue
Block a user