Final revised version
This commit is contained in:
1 parent
59d007205a
commit
3b63e37cc4
14 files changed
+119
-118
No files matched your search
@@ -44,7 +44,7 @@ There are some studies presented in the past regarding the error robustness of t
|
||||
A subjective evaluation study is presented in~\cite{SubjectiveEvalSep2013,QoEHEVCMay2014} revealing that packet loss ratios higher than $3\%$ have significant effect on the perceived video quality.
|
||||
The vulnerability of \ac{mv} prediction is investigated and its impact on the error robustness is evaluated in \cite{DynamicTMVP2011}.
|
||||
Moreover, in the same study, a method to introduce \ac{mv} refresh points is proposed, achieving average \ac{psnr} gains of approximately 3~dB.
|
||||
Summarising, the decrease in error robustness is mostly due to the strong data dependencies imposed by the highly complex prediction modes of \ac{hevc}. Since these are selected according to a rate distortion optimisation criterion that assumes \ac{plf} transmission, the decoder faces different problems whenever errors or data loss occur in the transmission network \cite{ROPE2007}.
|
||||
Summarising, the decrease in error robustness is mostly due to the strong data dependencies imposed by the highly complex prediction modes of \ac{hevc}. Since these are selected according to a rate distortion optimisation criterion that assumes packet loss-free transmission, the decoder faces different problems whenever errors or data loss occur in the transmission network \cite{ROPE2007}.
|
||||
|
||||
The following sub-sections present an error robustness evaluation of \ac{hevc} focused on the impact of the novel coding techniques introduced in the standard, extending the generic studies presented in previous works. This study provides relevant insights for research directions and motivation to investigate solutions for the error robustness problem in \ac{hevc}.
|
||||
The following case-studies are covered in this experimental evaluation:
|
||||
@@ -63,7 +63,7 @@ The experimental setup used in this study to evaluate the error robustness of th
|
||||
Table~\ref{TableTestSequences} presents a summary of the main characteristics of the test sequences. Appendix~\ref{Appen:Sequences} shows a single frame of each test sequence.
|
||||
Considering a test sequence with $N$ frames $f(t)$ the pixel values (Luma component) of the frame at instant $t$, the spatial and temporal complexity of the video sequences was measured by using the following metrics \cite{ITUTP9102008}:
|
||||
\begin{itemize}
|
||||
\item {\ac{si}} based on Sobel filter. Each video frame is firstly filtered, then the standard deviation ($std$) of the pixel values is computed. Finally, the maximum value over time ($\maxop_{t\in [1..N]}$), is chosen to represent the \ac{si} metric. This process can be represented by the following equation:
|
||||
\item {\ac{si}} based on Sobel filter. Each video frame is firstly filtered, then the standard deviation ($std$) of the pixel values is computed. Finally, the maximum value over time ($\maxop_{t\in [1..N]}$\{ \}), is chosen to represent the \ac{si} metric. This process can be represented by the following equation:
|
||||
\begin{equation}
|
||||
% SI = max_{time}\Big\{ std \big( Sobel\left(f_t\right) \big) \Big\}.
|
||||
SI = \maxop_{t\in [1..N]} \Big\{ std \big( sobel\left(f\left(t\right)\right) \big) \Big\}.
|
||||
@@ -102,12 +102,12 @@ Moreover, the ratio of inter-coded \acs{cu} of each coded sequence using the def
|
||||
\end{tabular}
|
||||
\end{table}
|
||||
|
||||
To simulate the lossy transmission environment, each frame is divided into several slices and each slice encoded and packetized into one \ac{nal} unit, which are then transmitted as the payload of independent packets.
|
||||
To simulate the lossy transmission environment, each frame was divided into several slices and each slice encoded and packetized into one \ac{nal} unit, which were then transmitted as the payload of independent packets.
|
||||
In these simulations a whole packet is lost whenever an error occurs, originating a lost slice.
|
||||
\HIGHLIGHT{Random packet loss was simulated using a two-state Markov model, derived from the classical Gilbert-Elliot \cite{GilbertModel_1960, ElliotModel_1963}, which can be used to describe error patterns in real time video applications~\cite{GilbertElliotModelPlr_2008}. In this study a simplified version of the Gilbert-Elliot model was used by only defining the loss probability without control over burst loss events.
|
||||
The application of this model resulted in loss patterns with approximately the same average burst loss as the ones provided in \cite{JCTVCLossSim_2012}.
|
||||
For each test condition, 50 runs are tested and the average quality is measured.
|
||||
Four different \acs{plr} with the characteristics presented in Table~\ref{TAB3:LossPatterns} are used.}
|
||||
For each test condition, 50 runs were tested and the average quality was measured.
|
||||
Four different \acs{plr} with the characteristics presented in Table~\ref{TAB3:LossPatterns} were used.}
|
||||
The remaining configurations have been kept at their default values, as given in the common test conditions~\cite{HEVCCommonTests2012}.
|
||||
|
||||
|
||||
@@ -125,14 +125,14 @@ In the first set of experiments the impact of the maximum \ac{cu} size in the de
|
||||
\end{table}
|
||||
|
||||
|
||||
Table~\ref{Table3:Quality_BlockSize} shows the average \ac{psnr} (dB) obtained for the three test conditions and for three different packet loss ratios. Results for the \ac{plf} scenario shows a consistent quality increase as the maximum \ac{cu} size increases,
|
||||
Table~\ref{Table3:Quality_BlockSize} shows the average \ac{psnr} (dB) obtained for the three test conditions and for three different packet loss ratios. Results for the \NOLOSS case shows a consistent quality increase as the maximum \ac{cu} size increases,
|
||||
% up to a maximum of 1.41~dB for the Tennis sequence
|
||||
when comparing the maximum \ac{cu} size of $64\times64$ with $16\times16$.
|
||||
This confirms the principle used by the \ac{hevc} standard, whereby larger coding units lead to higher coding efficiency.
|
||||
For the Tennis sequence, which has the highest \ac{ti} parameter (see Table~\ref{TableTestSequences}), a maximum of 1.41~dB is achieved.
|
||||
Moreover, it is also noticeable that sequences with higher amount of inter-coded blocks take less advantage of the larger \acs{cu}.
|
||||
This is noticeable in the results of the BQSquare sequence (i.e., highest ratio of inter-coded blocks and lowest quality gain in the \ac{plf} case).
|
||||
Overall, comparing the results for the \ac{plf} and data loss cases, it is noticeable that similar quality gains are obtained, mostly due to the superior coding efficiency of larger \acs{cu}.
|
||||
This is noticeable in the results of the BQSquare sequence (i.e., highest ratio of inter-coded blocks and lowest quality gain in the \NOLOSS case).
|
||||
Overall, comparing the results with and without packet loss, it is noticeable that similar quality gains are obtained, mostly due to the superior coding efficiency of larger \acs{cu}.
|
||||
In most cases the quality gains slightly decrease for increasing \acs{plr}.
|
||||
Therefore, one may conclude that the error robustness of \ac{hevc} streams is not affected by limiting the maximum \ac{cu} size.
|
||||
|
||||
@@ -177,7 +177,7 @@ Figure~\ref{Figure3:QualityvsIntra} shows the average decoded video quality (\ac
|
||||
Results in the figure indicate that as the distance between intra frames increases, the decoded video quality significantly decreases.
|
||||
The quality decreasing is especially noticeable for intra periods higher than 32 frames, indicating that temporal distances greater than approximately one second are not suitable for video streaming under error-prone conditions.
|
||||
When a high temporal distance between intra frames is used, the quality decreasing is more noticeable in the People On Street and BQSquare sequences, which have higher spatial complexity (see values for \ac{si}), leading to a reduction of 13 dB and 14 dB, respectively.
|
||||
As spatial complexity increases, the prediction residue energy also increases, which leads to more erroneous information in case of data loss, reducing the video quality under errors. \HIGHLIGHT{Finally, based on the confidence interval one may conclude that for different error patterns the average \ac{psnr} of all frames does not significantly changes (maximum interval of 1 dB approximately).}
|
||||
As spatial complexity increases, the prediction residue energy also increases, which leads to more erroneous information in case of data loss, reducing the video quality under errors. \HIGHLIGHT{Finally, based on the confidence interval one may conclude that for different error patterns the average \ac{psnr} of all frames does not significantly change. A maximum variation of 1 dB, approximately, is observed in Figure~\ref{Figure3:QualityvsIntra}.}
|
||||
|
||||
One should also note from the results of Figure~\ref{Figure3:QualityvsIntra} that, using a small distance between intra frames is not always more efficient.
|
||||
Since intra frames require higher amount of bits, the overall coding efficiency is reduced, resulting in lower overall quality in case of data loss, when comparing with other cases.
|
||||
@@ -197,7 +197,7 @@ A comparison of the quality obtained for different partition modes is also carri
|
||||
\begin{figure}[!t]
|
||||
\centering
|
||||
\input{./031_ErrorResilienceEvaluation/QualityEvaluation/PSNRvsSlices.tex}
|
||||
\caption{Decoded video quality for different slice partitioning modes.}
|
||||
\caption{Decoded video quality with a 95\% confidence interval for different slice partitioning modes.}
|
||||
\label{Figure3:SlicePartitioning}
|
||||
\end{figure}
|
||||
|
||||
@@ -211,7 +211,7 @@ Although the use of smaller slices increases the overhead required for slice hea
|
||||
This is due to the lack of refresh points within a frame, resulting in errors that affect larger regions of the frame, which are propagated to the subsequent inter predicted frames.
|
||||
|
||||
The results for of \textit{SS} (slice partitioning disabled) reveals that this approach also leads to lower error robustness when comparing with those cases where the frames are divided into multiple slices.
|
||||
\HIGHLIGHT{Moreover, in this case the 95\% confidence interval increases, due to the fact that each packet corresponds to a whole frame, which vary in size. Therefore, the amount of lost data varies depending on the affected frame, leading to a different impact of the recovered quality.}
|
||||
\HIGHLIGHT{In this case the 95\% confidence interval is greater than the others shown in Figure~\ref{Figure3:SlicePartitioning}. This is because the packet size is variable, which implies higher variability in the amount of lost information.}
|
||||
Although higher coding efficiency is achieved in this mode, the lower error robustness leads to an overall poor performance under frame or slice loss conditions.
|
||||
It is worthwhile to notice that the results for Kendo sequence do not follow the main trend.
|
||||
This can be explained by the low bitrate required to compress the video signal, which is the lowest when compared with the other test sequences (see Table~\ref{TableTestSequences}).
|
||||
@@ -280,7 +280,7 @@ Moreover, the slice partitioning mode used only 1200 bytes per slice, which intr
|
||||
|
||||
The second set of experiments aimed to find the influence of losing temporal motion predictors on the error resilience of \ac{hevc}.
|
||||
In order to perform this test, a single frame loss in enforced, and the error propagation is evaluated for two configurations: (i) with the temporal \ac{mv} candidate enabled and (ii) disabled.
|
||||
Figure~\ref{Figure3:PSNRvsFrameErrorFreeConc} illustrates the error propagation, when a single frame is lost. For this particular experiment an intra-refreshing period of 32 frames was used to evaluate the impact on long error propagation periods. This is commonly used in quality evaluation of erroneous video decoding and follows the reference conditions~\cite{HEVCCommonTests2012}.
|
||||
Figure~\ref{Figure3:PSNRvsFrameErrorFreeConc} illustrates the error propagation, when a single frame is lost. \HIGHLIGHT{For this particular experiment an intra-refreshing period of 32 frames was used to evaluate the impact on long error propagation periods. This is commonly used in quality evaluation of erroneous video decoding and follows the reference conditions~\cite{HEVCCommonTests2012}.}
|
||||
The results show that when the \ac{tmvp} mode is enabled, the loss of Frame \#6 leads to significant reduction in the reconstruction quality of the subsequent frames.
|
||||
On the contrary, when the temporal \ac{mv} candidate is disabled, the motion information on subsequent frames is not affected by the loss of a single frame and significantly higher quality is obtained.
|
||||
This is the effect of breaking the temporal dependencies between \acs{mv}, which prevents erroneous \acs{mv} to propagate throughout the set of dependent frames.
|
||||
@@ -295,7 +295,7 @@ This is the effect of breaking the temporal dependencies between \acs{mv}, which
|
||||
|
||||
In order to further evaluate the impact of the temporal \ac{mv} dependencies on the error propagation, the \ac{hevc} streams were subject to random packet losses.
|
||||
Table~\ref{Table3:AvgPSNRWithoutTMVP} presents the average quality results, measured using the \ac{psnr}.
|
||||
As expected, the results indicate that in the \ac{plf} case higher quality is achieved by enabling the \ac{tmvp} candidate, since higher compression efficiency is obtained because more \ac{mv} predictors are used.
|
||||
As expected, the results indicate that in the those cases without packet loss (\NOLOSS) higher quality is achieved by enabling the \ac{tmvp} candidate, since higher compression efficiency is obtained because more \ac{mv} predictors are used.
|
||||
This is true for both test conditions, \emph{i.e.}, \ac{ld} and \ac{ra}.
|
||||
However, in the presence of errors, the video quality significantly decreases due to low error robustness when temporal \ac{mv} candidates are used.
|
||||
Alternatively, average quality up to 5.17 dB of \ac{psnr} is achieved when the temporal \ac{mv} predictors are disabled in the \ac{ld} configuration.
|
||||
@@ -348,7 +348,7 @@ In the proposed approach, a temporal \ac{mv} candidate is marked as unsuitable,
|
||||
\centering
|
||||
% \includegraphics[width=0.6\columnwidth]{./031_ErrorResilienceEvaluation/DynamicTmvp/DynamicTMVP}
|
||||
\input{./031_ErrorResilienceEvaluation/DynamicTmvp/DynamicTMVP.tikz}
|
||||
\caption{Proposed approach to constrain the temporal \ac{mv} dependencies (arrow point to the MV predictor).}
|
||||
\caption{Proposed approach to constrain the temporal \ac{mv} dependencies (the arrows point to the MV predictors).}
|
||||
\label{ProposedDynamicMethod}
|
||||
\FIGSVSPACE
|
||||
\end{figure}
|
||||
@@ -416,7 +416,7 @@ Therefore, since the proposed method achieves the lowest \ac{psnr} standard dev
|
||||
Table~\ref{Table3:AverageResultsProposed} shows the Bjontegaard's average \ac{psnr} increase \cite{BJResults2001}.
|
||||
The results are obtained taking the standard \ac{hevc} encoder as reference.
|
||||
The results presented in Table~\ref{Table3:AverageResultsProposed} show that the proposed method presents practically the same rate-distortion performance when compared to the \TIDR method.
|
||||
In general, the average quality, in the \ac{plf} case, only decreases 0.028 dB in comparison to the \TIDR method and 0.034 dB in comparison to the reference encoder.
|
||||
In general, the average quality, in the \NOLOSS case, only decreases 0.028 dB in comparison to the \TIDR method and 0.034 dB in comparison to the reference encoder.
|
||||
However, the proposed method widely outperforms the reference \TIDR technique in terms of error resiliency performance, achieving in average higher overall quality under error-prone conditions.
|
||||
The effectiveness of the proposed method increases at higher packet loss rates (\emph{e.g.},~$5\%$), achieving an average gain of 0.64 dB ($1.34 - 0.70$), and a maximum gain of 1 dB ($1.44-0.43$) for the Book Arrival sequence , when compared to the \TIDR technique. Higher gains are achieved when compared with the reference \ac{hevc} encoder, as shown in Table~\ref{Table3:AverageResultsProposed}.
|
||||
|
||||
|
||||
Reference in new issue
Block a user