Fix some compound word

This commit is contained in:
jfmcarreira committed 2018-09-06 14:56:36 +01:00
1 parent 16b070607f
commit 1c13a4e48e
9 files changed
+309 -278

No files matched your search

@@ -34,8 +34,8 @@ This chapter is organized as follows. Section~\ref{Section4:Problem} presents th
As mentioned before \ac{hevc} bitstreams subject to transmission losses result in significant degradation of both objective and subjective quality.
This is mostly due to the strong decoding dependencies imposed by the highly complex prediction modes, which are selected according to a \ac{rd} optimisation criteria, assuming error-free transmission~\cite{ROPE2007}.
Therefore, high compression efficiency is achieved, but resilience to transmission errors is penalised.
This is mostly due to the strong decoding dependencies imposed by the highly complex prediction modes, which are selected according to a \ac{rd} optimisation criteria, assuming error-free transmission~\cite{ROPE2007}.
Therefore, high compression efficiency is achieved, but resilience to transmission errors is penalised.
\begin{figure}[tb]
\centering
@@ -46,15 +46,15 @@ Therefore, high compression efficiency is achieved, but resilience to transmissi
Highly efficient video encoders support the use of multiple reference frames for inter-frame prediction, which are selected based on a \ac{rd} cost which may lead to an unbalance distribution.
In order to show evaluate the unbalanced is the reference frame selection attained by the \ac{rd} implemented in the reference \ac{hevc} encoder, different test sequences are encoded using two reference frames, referred to as $r_1$ and $r_2$, and its selection ratios are analysed. In this study, reference $r_1$ corresponds to the closest neighbouring frame and reference $r_2$ the second closest.
Figure~\ref{FIG4:ReferenceUsageRatio} illustrates the average usage percentage of reference $r_1$ for different sequences.
The remaining corresponds to those cases where reference $r_2$ were selected.
Figure~\ref{FIG4:ReferenceUsageRatio} illustrates the average usage percentage of reference $r_1$ for different sequences.
The remaining corresponds to those cases where reference $r_2$ were selected.
These results show that, in general, reference $r_1$ is selected more often than $r_2$.
This is mainly true for sequences with higher motion. For those sequences with lower motion but high spatial complexity, e.g., BQSquare (see \ac{si} in Table~\ref{TableTestSequences}), both reference frames equally (approximately) selected for temporal predictions, with $r_1$ being used for 58\% of cases and $r_2$ for 42\%.
This is mainly true for sequences with higher motion. For those sequences with lower motion but high spatial complexity, e.g., BQSquare (see \ac{si} in Table~\ref{TableTestSequences}), both reference frames equally (approximately) selected for temporal predictions, with $r_1$ being used for 58\% of cases and $r_2$ for 42\%.
This reveals that even with a configuration with two references the encoder will mainly use the temporally closer one as the reference for inter-frame predictions.
Therefore, if such reference frame is lost, a higher number of mismatched predictions will occur, inevitabely leading to higher error propagation.
Another technique that contributes to increased error propagation is the use of predictions in \ac{mv} coding~\cite{MotionCodingHEVC2011}, which contributes to increase the spatial and temporal dependencies.
Another technique that contributes to increased error propagation is the use of predictions in \ac{mv} coding~\cite{MotionCodingHEVC2011}, which contributes to increase the spatial and temporal dependencies.
Although the spatial \ac{mv} candidates are used more often, temporally predicted \acs{mv} are responsible for a greater impact on error propagation, as they propagate errors in the \ac{mv} decoding across different time instances.
To overcome these issues, in the next sections a two-fold approach is described to increase the error robustness in video streaming by jointly considering two different stages: (i) while encoding live video, by selecting robust coding modes and (ii) after encoding, by adding robust features to the compressed streams. While in the former this is accomplished by optimising coding parameters and decisions to increase robustness, in the latter some type of compressed domain processing is used to increase error robustness of pre-encoded streams, e.g., error resilient transcoding. Inevitably, in both cases this is achieved at the expense of some loss in coding efficiency, therefore the challenge is to find an optimum trade-off between robustness and bitrate overhead. In this chapter a two-fold approach is used to overcome the two problems mentioned above.
@@ -65,7 +65,7 @@ To overcome these issues, in the next sections a two-fold approach is described
\section{Proposed two-stage architecture}
\label{Section4:Architecture}
The goal of the proposed method is to reduce the error propagation in decoded video in case of frame loss.
The goal of the proposed method is to reduce the error propagation in decoded video in case of frame loss.
The underlying idea is to constrain inter-frame prediction dependencies, in order to minimise the distortion $D_t$ given by
\begin{equation}
D_t = \tilde f_t - \hat f_t, \text{when } \tilde f_{t-1} \neq \hat f_{t-1},
@@ -95,11 +95,11 @@ This is equivalent to reducing the error propagation by using two techniques: (i
\label{Section4:RefPicSel}
The reference picture selection mechanism attempts to uniformly distribute the use of different reference pictures across the \ac{ctu} of a coded video frame.
In case of frame loss, the error propagation in the decoder is reduced because the number of \acs{ctu} predicted from the lost frame decreases and thus error concealment benefits as more data is correctly decoded from other frames.
The reference picture selection mechanism attempts to uniformly distribute the use of different reference pictures across the \ac{ctu} of a coded video frame.
In case of frame loss, the error propagation in the decoder is reduced because the number of \acs{ctu} predicted from the lost frame decreases and thus error concealment benefits as more data is correctly decoded from other frames.
The underlying principle of this method is to minimise the probability of using the same reference picture for neighbouring \acs{ctu}.
%For instance,
%For instance,
In a coding structure with two reference frames, as shown in Figure~\ref{fig:checkerboardRefSelection}, the reference frames $r_{1}$ and $r_{2}$ are used to predict the same number of \acs{ctu} in current frame~($C$), resulting in a checkerboard structure, where a different reference frame is used for predicting every other \ac{ctu} within the current frame.
If each reference frame is used to predict half of the pixels in the current frame, when any of these references is lost, the subsequent predicted frame is only partially distorted by mismatched predictions.
This is because only half of its predictions depend on the missing reference frame, which limits the overall error propagation.
@@ -187,8 +187,8 @@ Figure~\ref{fig:RefPicSelectionExample} illustrates the distribution of referenc
It is noticeable in Figure~\ref{fig:RefPicSelectionExample} that for several \acs{ctu} the \PROP method is able to keep the same partitions and reference frames when compared to the optimal encoding (\REF). This can be observed, for example in the small partitions on the left, where the \PROP method uses the same reference frame as \REF. Thus, the proposed method is expected to achieve better coding efficiency than \CHKB, since more optimal coding modes may be chosen.
% TID aqui?
Summarising, by using Equations~(\ref{eq:Lagrangian})~and~(\ref{eq:KFactor}), the cost of using a given reference frame increases as the number of \acs{pu} using it for prediction increases.
However, the reference frames are not hard-selected without taking into account the R-D cost.
Summarising, by using Equations~(\ref{eq:Lagrangian})~and~(\ref{eq:KFactor}), the cost of using a given reference frame increases as the number of \acs{pu} using it for prediction increases.
However, the reference frames are not hard-selected without taking into account the R-D cost.
In general, it is more difficult to reach a uniform use of reference frames in complex sequences than in simpler ones.
Note that standard compatibility is maintained, since the proposed optimisation method does not require any syntax change. Moreover, since the proposed approach is employed at the Lagrangian optimisation already implemented in the \ac{hevc} reference software, the complexity of the encoder is not significantly affected.
@@ -198,7 +198,7 @@ Note that standard compatibility is maintained, since the proposed optimisation
\glsreset{tmvp}
Since \ac{hevc} uses differential and predictive \ac{mv} coding, whenever a \ac{mv} is lost the subsequent ones will also be affected, until a refresh point (i.e., \ac{mv} coded on its own, without using \ac{mv} prediction) is reached to break the dependency chain.
Since \ac{hevc} uses differential and predictive \ac{mv} coding, whenever a \ac{mv} is lost the subsequent ones will also be affected, until a refresh point (i.e., \ac{mv} coded on its own, without using \ac{mv} prediction) is reached to break the dependency chain.
Thus, the amount of error propagation strongly depends on whether the \ac{tmvp} is used.
In this streaming stage, to increase the number of possible refresh points and improve error robustness, a small number of \acs{mv} are selected to be transmitted as side information. The selection of these redundant \acs{mv} is based on a trade-off between the image area covered by such \acs{mv} and the number of bits to encode them.
The best tradeoff corresponds to the \acs{mv} that cover the largest image area using the lowest amount of encoding bits.
@@ -209,7 +209,7 @@ The best tradeoff corresponds to the \acs{mv} that cover the largest image area
Figure~\ref{fig:mvdependencies} illustrates how \acs{mv} have different importance in error propagation.
In this figure, each block corresponds to a \ac{pu}, which has an associated \ac{mv}. The arrows point to the \ac{pu} containing the \ac{mv} used as predictor. In the current frame ($f_0$), two \acs{pu} have temporally dependent \acs{mv}.
While the \ac{mv} of \ac{pu} \MVDEPA is used to predict a total of four other \acs{mv}, the \ac{mv} of \ac{pu} \MVDEPB is only used to predict one single \ac{mv}.
While the \ac{mv} of \ac{pu} \MVDEPA is used to predict a total of four other \acs{mv}, the \ac{mv} of \ac{pu} \MVDEPB is only used to predict one single \ac{mv}.
Thus, in case of data loss (e.g., frame $f_{-1}$), the \ac{mv} of \ac{pu} \MVDEPA has more impact on error propagation to the frame $f_{0}$ and consequently quality degradation than the \ac{mv} of \ac{pu} \MVDEPB, because of its higher number of dependent \acs{mv}.
For this reason, in the \ac{mv} selection process, the proposed method assigns higher importance to the \ac{mv} of \ac{pu} \MVDEPA than \ac{pu} \MVDEPB. Moreover, in Figure~\ref{fig:mvdependencies}, it can be observed that although the number of \acs{mv} predicted from the \acs{mv} of \ac{pu} \MVDEPC is higher than \ac{pu} \MVDEPA (6 and 4 \acs{mv}, respectively),
the image area that is dependent from the former is smaller that the latter. This is due to the variable size of \acs{pu}.
@@ -261,9 +261,9 @@ In this section the robustness of the proposed two-stage architecture is evaluat
As references for comparison, three other methods are used: (i) a fixed checkerboard approach (\CHKB), (ii) the reference picture generation method based on leaky predictions proposed in \cite{ROPERefGenJan2010}, identified as ``\AVG'', and (iii) long-term reference frames (\LONG). The \LONG method only uses the key frames as reference, i.e., in the \ac{ra} configuration only one out of eight frames is used as reference for prediction, allowing a more robust transmission, as the reference frames are less likely to be hit by errors.
The experimental setup used in this section is the same as the one previously described in Section~\ref{Section3:ErrorResilienceEvaluation}.
The experimental setup used in this section is the same as the one previously described in Section~\ref{Section3:ErrorResilienceEvaluation}.
Two test cases were considered for simulation: (i) in \emph{\LOSTALL} all frames may be equally affected by packet loss, including \ac{idr} frames; (ii) in \emph{\KEEPINTRA} \ac{idr} frames are assumed to be prioritised in the network and delivered error-free, or using some combined approach as recently proposed in \cite{ErrorResilienceHeterogenousNetworks2016}.
At the decoder side, error concealment is based on motion compensation from previously decoded frames. The motion field of a missing frame is obtained applying \ac{mv} extrapolation from the two closest neighbours. Afterwards, the final \acs{mv} are selected based on the residual energy of their original prediction~\cite{HEVCMCResidueDec2013}.
At the decoder-side, error concealment is based on motion compensation from previously decoded frames. The motion field of a missing frame is obtained applying \ac{mv} extrapolation from the two closest neighbours. Afterwards, the final \acs{mv} are selected based on the residual energy of their original prediction~\cite{HEVCMCResidueDec2013}.
The evaluation of the encoding-stage is carried out separately from the streaming-stage.
@@ -272,7 +272,7 @@ The evaluation of the encoding-stage is carried out separately from the streamin
%The sequences were encoded with all coding modes enabled and following two recommended prediction structures: Low-Delay (P-frames with 4 references frames) and \ac{ra} (B-frames with 2 reference frames in each list)~\cite{HEVCCommonTests2012}. These prediction structures cover the use of P-frames and hierarchical B-frames~\cite{H264ScalableOverviewSep2007}, which are commonly used in HEVC streaming~\cite{RealTimeHEVC2016}.
%To simulate a broadcast transmission an \ac{idr} period of $32$ frames was used and the filtering operations were disabled at slices boundaries in order to keep the slices self-contained. The remaining configurations have been kept at their default values as given in the common test conditions~\cite{HEVCCommonTests2012}.
%In order to achieve high quality images for all coded sequences (i.e., PSNR around 40 dB) different bitrates were used.
%At the decoder side, error concealment is based on motion compensation from previously decoded frames. The motion field of a missing frame is obtained applying MV extrapolation from the two closest neighbours. Afterwards, the final MVs are selected based on the residual energy of their original prediction~\cite{HEVCMCResidueDec2013}.
%At the decoder-side, error concealment is based on motion compensation from previously decoded frames. The motion field of a missing frame is obtained applying MV extrapolation from the two closest neighbours. Afterwards, the final MVs are selected based on the residual energy of their original prediction~\cite{HEVCMCResidueDec2013}.
%To simulate the lossy transmission environment, each frame is encoded into several slices, each one packetized into one NAL unit, which are then transmitted as the payload of independent packets.
%
%In our experiments a whole packet is lost whenever an error occurs, originating a lost slice. Random packet loss was simulated using a two-state Markov model with the following average burst length 1.24, 1.47, 1.83 and 2.05, and a maximum burst length of 2, 3, 5 and 7, for packet loss ratios (PLR) of 1\%, 3\%, 5\%, 10\%, respectively.
@@ -283,13 +283,13 @@ The evaluation of the encoding-stage is carried out separately from the streamin
The performance of the encoding stage is evaluated on its own in order to only determine the improvements achieved by the dynamic reference frame selection method.
In this case, the reference frames were reconstructed with distortion, as a result of non-perfect error concealment.
Since the encoding stage does not deal with errors in temporal \ac{mv} predictions (this is handled by the streaming-stage), in this evaluation the temporal \ac{mv} predictor is disabled.
Since the encoding stage does not deal with errors in temporal \ac{mv} predictions (this is handled by the streaming-stage), in this evaluation the temporal \ac{mv} predictor is disabled.
This is necessary to avoid masking the results with the errors caused by wrong \ac{mv} predictions.
The performance of the encoding stage (\PROP) was evaluated using $\gamma = 5$.
The performance of the encoding stage (\PROP) was evaluated using $\gamma = 5$.
This value was chosen to allow increasing the Lagrangian cost by about $50\%$, when a reference frame is used for prediction of more than three \acs{ctu} in comparison to the usage of the other reference frames (see Figure~\ref{fig:weightfct}). Nevertheless, further results for a wide range of $\gamma$ values are also presented.
%\HERE
%\HERE
\subsubsection{Quality evaluation under error-prone conditions}
\begin{figure}[tb]
@@ -312,7 +312,7 @@ This value was chosen to allow increasing the Lagrangian cost by about $50\%$,
\end{figure}
The error robustness achieved by the encoding-stage was firstly evaluated in regards to the error propagation resulting from a single loss event in the \ac{ld} configuration.
Figure~\ref{fig:RecoveredExample} shows the visual impact of error propagation on a region of the decoded Frame \#10 when Frame \#9 is affected by errors, for Basketball Drill and Race Horses sequences (the \ac{psnr} corresponds to the entire frame).
Figure~\ref{fig:RecoveredExample} shows the visual impact of error propagation on a region of the decoded Frame \#10 when Frame \#9 is affected by errors, for Basketball Drill and Race Horses sequences (the \ac{psnr} corresponds to the entire frame).
The proposed method is compared with the method~in~\cite{ROPERefGenJan2010} as both aim at reducing the error propagation.
The method~in~\cite{ROPERefGenJan2010} uses filtered frames as reference for predictions, therefore, in case of errors, the effects of mismatch predictions are significantly reduced by the filtering operation (see Sub-Section~\ref{SECTION2:LeakyPred}).
As shown in this figure, the proposed method is able to reduce the impact of mismatch decoding in comparison with the method~in~\cite{ROPERefGenJan2010}. Although both methods still reveal some predictions mismatch leading to inaccurate reconstructed of the objects contours, the proposed method is able to achieve a smoother reconstruction, which leads to higher quality in terms of \ac{psnr}. This is because, in frame \#10, the predictions depending on the missing frame \#9 are not used throughout the entire frame, thus, approximately half of the predictions are not affected by errors. In contrast, the method~in~\cite{ROPERefGenJan2010} requires several filtering operations to be applied (i.e, different reference frames need to be filtered) in order to reduce the effect of mismatch predictions.
@@ -342,14 +342,14 @@ Further tests were run to evaluate the effectiveness of the proposed method unde
The use of long-term reference pictures (\LONG) is only tested for the \ac{ra} configuration, where key frames are available, while method \cite{ROPERefGenJan2010} is tested for the \ac{ld} configuration.
For each test condition, 50 trials were performed and the average quality (\ac{psnr}) obtained across all trials for both coding configurations are shown in Table~\ref{tab:AverageQualityCHKBEXPLowDelay}~and~\ref{tab:AverageQualityCHKBEXPRandomAccess}, for the \ac{ld} and \ac{ra}, respectively.
In both tables the absolute \ac{psnr} is shown for the reference \ac{hevc} case (\REF), while the \ac{psnr} difference is presented for the other cases.
Results show that the proposed method is able to outperform other methods for the same loss conditions.
Results show that the proposed method is able to outperform other methods for the same loss conditions.
An average quality gain up to $2.26$~dB is obtained for the \ac{ld} configuration (see Table~\ref{tab:AverageQualityCHKBEXPLowDelay}), in comparison to the \REF, while the fixed checkerboard pattern approach (\CHKB) only achieves up to $2.15$~dB.
In \cite{ROPERefGenJan2010}, the use of interpolated references in the encoding loop helps to reduce error propagation, at a cost of poor predictions, thus degrading the coding efficiency. This is the main reason that justifies a lower overall performance of the method proposed in \cite{ROPERefGenJan2010} (see results for ``\AVG'').
The proposed method is also able to outperform the \ac{hevc} reference software when an hierarchical coding structure is used (see results for \ac{ra} configuration in Table~\ref{tab:AverageQualityCHKBEXPRandomAccess}), which implicitly limits the error propagation.
In comparison to the reference \ac{hevc}, quality improvements up to $1.42$~dB at PLR=$10\%$ are obtained for Kendo sequence.%, while the average gain across all test sequences is 1~dB.
When comparing results for both test cases, it is noticeable that the proposed method is able to outperform the reference methods both when \ac{idr} frame loss is allowed or not (\LOSTALL and \KEEPINTRA, respectively). As the \ac{plr} increases, the gains of the proposed method decrease in comparison with the case where the \ac{idr} frames are delivered error-free (i.e., case \KEEPINTRA), due to the higher error propagation of the missing \ac{idr} frame. Moreover, the quality gains are lower in the \LOSTALL case when comparing with \KEEPINTRA for the same \ac{plr}, for both the \ac{ld} and \ac{ra}.
Taking into account the above results and the spatio-temporal information shown in Table~\ref{TableTestSequences}, one can observe that the performance of the proposed method is lower for sequences with high spatial details and low temporal activity (e.g., BQSquare and Park Scene), which indicates higher effectiveness for video content with higher motion activity.
Taking into account the above results and the spatio-temporal information shown in Table~\ref{TableTestSequences}, one can observe that the performance of the proposed method is lower for sequences with high spatial details and low temporal activity (e.g., BQSquare and Park Scene), which indicates higher effectiveness for video content with higher motion activity.
%\HIGHLIGHT{complete with analysis of both cases}
@@ -369,10 +369,10 @@ Taking into account the above results and the spatio-temporal information shown
The influence of the $\gamma$ parameter used in Equation (\ref{eq:KFactor}) on the video quality was also evaluated, in order to check how the cost constrains affect the performance of the proposed method.
Figure \ref{fig:QualityVsGamma} shows the average quality gains of the proposed method in comparison to the reference \ac{hevc} (\REF) across all test sequences, for different values of $\gamma$.
For the error-free case (PLR=0\%), the results in this figure show that increasing $\gamma$ leads to a decrease in the coding efficiency, especially in the \ac{ld} configuration.
For the error-free case (PLR=0\%), the results in this figure show that increasing $\gamma$ leads to a decrease in the coding efficiency, especially in the \ac{ld} configuration.
The reason is that higher values of $\gamma$ lead to higher values of $W(r_{\phi})$, therefore the reference picture selection cost is severely constrained by the exponential weights.
Since $T_{ID}$ is constant for all frames in the \ac{ld}, higher weights are used for all frames ($T_{ID}$ is used to reduce the weights in case of hierarchical coding) and the encoder is not able to select the best reference frame in \c{rd} sense.
In the case where packet loss occurs, higher values of $\gamma$ lead to improved error resilience.
In the case where packet loss occurs, higher values of $\gamma$ lead to improved error resilience.
Since higher weights are used in the \ac{rd} cost, the encoder is constrained to distribute the use of the reference frames of different \acs{pu}, reducing the error propagation.
For very high values (e.g., $\gamma=1000$) the encoder is forced to uniformly select the reference frames, resulting in a fixed checkerboard pattern.
Overall, the $\gamma$ parameter is able to control the reference frame selection constrains,
@@ -418,7 +418,7 @@ Overall, the loss in the quality compared to the reference \ac{hevc} is consider
\subsubsection{Complexity overhead}
The implementation of this method slightly increases the complexity of a standard non-robust encoder. Therefore, the computational complexity has also been considered as a performance metric in this research, which is measured as the encoding time in a controlled hardware platform. The reference used for comparison with other methods is the \ac{hevc} reference software encoding time, running on the same platform with the same coding configuration.
The relative complexity increase of the robust encoder using \PROP, \CHKB and the method presented in \cite{ROPERefGenJan2010} is shown in Table~\ref{tab:cpu_time}.
The relative complexity increase of the robust encoder using \PROP, \CHKB and the method presented in \cite{ROPERefGenJan2010} is shown in Table~\ref{tab:cpu_time}.
As shown in this table, the method presented in \cite{ROPERefGenJan2010} is the most time consuming because it requires post-processing of the encoded frames in order to interpolate new references.
@@ -446,8 +446,8 @@ This is due to the fact that in the \CHKB method there is no \ac{rd} optimisatio
\subsection{Streaming stage}
In this section the coding performance that results from transmitting redundant \ac{mv} (streaming stage) in combination with the reference picture selection scheme (encoding stage) is presented.
In these experimental tests the temporal \ac{mv} candidates are enabled and \acs{mv} are dynamically added as side-information into the stream, as described in Section \ref{Section4:MVSelection}.
In this section the coding performance that results from transmitting redundant \ac{mv} (streaming stage) in combination with the reference picture selection scheme (encoding stage) is presented.
In these experimental tests the temporal \ac{mv} candidates are enabled and \acs{mv} are dynamically added as side-information into the stream, as described in Section \ref{Section4:MVSelection}.
Different values of $\alpha$ are used to achieve different overhead ratios. All streams were encoded with the same total bitrate, including the redundant bits, in order to make a fair quality comparison.