Fix some compound word
This commit is contained in:
1 parent
16b070607f
commit
1c13a4e48e
9 files changed
+309
-278
No files matched your search
@@ -14,9 +14,9 @@
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
\section{Introduction}
|
||||
|
||||
The use of \ac{hevc} in different type of application and video services over networks that are prone to errors and packet losses, requires mechanisms to minimise the effect o such probabilistic events. This chapter starts by presenting an evaluation study about the error resilience of \ac{hevc} in Section~\ref{Section3:ErrorResilienceEvaluation} , with the aim of defining the weak elements of coded streams and level of error robustness under different transmission loss conditions. The evaluation study covers different coding configuration to evaluate the individual impact of some new coding tools introduced, e.g., increased partition size, intra-refreshing, slice partitioning and advanced motion vector prediction.
|
||||
The use of \ac{hevc} in different type of application and video services over networks that are prone to errors and packet losses, requires mechanisms to minimise the effect o such probabilistic events. This chapter starts by presenting an evaluation study about the error resilience of \ac{hevc} in Section~\ref{Section3:ErrorResilienceEvaluation} , with the aim of defining the weak elements of coded streams and level of error robustness under different transmission loss conditions. The evaluation study covers different coding configuration to evaluate the individual impact of some new coding tools introduced, e.g., increased partition size, intra-refreshing, slice partitioning and advanced motion vector prediction.
|
||||
In Section~\ref{Section3:DynamicMVRefreshHEVC} a novel method to increase the error robustness of the \ac{hevc} \ac{mv} coding is presented and evaluated against existing techniques. The method proposed introduces refresh point in the motion prediction used in the \ac{hevc}, which significant reduces the impact of mismatch \ac{mv} predictions.
|
||||
Finally, Section~\ref{Section3:Summary} concludes this chapter.
|
||||
Finally, Section~\ref{Section3:Summary} concludes the chapter.
|
||||
|
||||
\clearpage
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
@@ -27,7 +27,7 @@ Finally, Section~\ref{Section3:Summary} concludes this chapter.
|
||||
|
||||
\subsection{Motivation and objectives}
|
||||
|
||||
As mentioned before the \ac{hevc} standard has improved its predecessor in terms of coding efficiency,
|
||||
As mentioned before the \ac{hevc} standard has improved its predecessor in terms of coding efficiency,
|
||||
higher complexity and data dependencies, mostly resulting from advanced predictions structures.
|
||||
Although the increased complexity may be compensated with more efficient algorithms and faster hardware technologies, the impact of higher coding efficiency on reducing error robustness strongly affects the video quality under packet loss.
|
||||
|
||||
@@ -35,12 +35,12 @@ A critical aspect related to the performance of robust video streaming methods i
|
||||
Since \ac{hevc} streaming is expected to use known transport technologies, such as RTP~\cite{HEVCStreamMay2012}, MPEG-2 TS~\cite{HEVCLayerIntegrationDez2012}, or more recent standards as MPEG-DASH~\cite{HEVCHTTPStreamAug2016} and MPEG Media Transport\cite{ATSCMMT2016}, the error detection capabilities of the underlying transport protocols have significant impact on the overall system performance.
|
||||
In general, the error detection operation at the transport layers is decoupled from the video coding layer.
|
||||
To cope with transmission errors the encoding side implements robust coding methods, while the efficient recovery of lost video data is implemented in the decoder through error concealment algorithms.
|
||||
Between them, signalling between different protocol layers ensures that the decoder is given the necessary information to identify the lost slices/frames in the video stream.
|
||||
Therefore, the error robustness of video streams can be investigated on its own, assuming any possible form of streaming technologies and error detection.
|
||||
Between them, signalling between different protocol layers ensures that the decoder is given the necessary information to identify the lost slices/frames in the video stream.
|
||||
Therefore, the error robustness of video streams can be investigated on its own, assuming any possible form of streaming technologies and error detection.
|
||||
This is the main focus of the study presented in this section.
|
||||
|
||||
There are some studies presented in the past regarding the error robustness of the \ac{hevc} standard which are worth to be mention as they provide relevant insights for the research of carried out in this research work. In~\cite{StudyHEVCErrorsDez2012,HEVCWireless_2016} both the coding efficiency and error robustness of the \ac{hevc} are evaluated and compared against the \ac{h264}. Experimental results confirm that \ac{hevc} has reduced error robustness despite its increased coding efficiency.
|
||||
A subjective evaluation study is presented in~\cite{SubjectiveEvalSep2013,QoEHEVCMay2014} revealing that packet loss ratios higher than $3\%$ have significant effect on the perceived video quality.
|
||||
There are some studies presented in the past regarding the error robustness of the \ac{hevc} standard which are worth to be mention as they provide relevant insights for the research of carried out in this research work. In~\cite{StudyHEVCErrorsDez2012,HEVCWireless_2016} both the coding efficiency and error robustness of the \ac{hevc} are evaluated and compared against the \ac{h264}. Experimental results confirm that \ac{hevc} has reduced error robustness despite its increased coding efficiency.
|
||||
A subjective evaluation study is presented in~\cite{SubjectiveEvalSep2013,QoEHEVCMay2014} revealing that packet loss ratios higher than $3\%$ have significant effect on the perceived video quality.
|
||||
%In both works it is concluded that the robustness of \ac{hevc} decreases for sequences complex motion.
|
||||
The vulnerability of \ac{mv} prediction was investigated and its impact on the error robustness was evaluated \cite{DynamicTMVP2011}. Moreover, a method to introduce \ac{mv} refresh points was proposed, achieving average \ac{psnr} gains of approximately 3~dB.
|
||||
Summarising, the decrease in error robustness is mostly due to the strong data dependencies imposed by the highly complex prediction modes of \ac{hevc}. Since these are selected according to a rate distortion optimisation criterion, assuming error-free transmission, the decoder faces a different problem whenever errors or data loss occur in the transmission network \cite{ROPE2007}.
|
||||
@@ -79,12 +79,12 @@ The values of \ac{si} and \ac{ti} calculated for the test sequences are shown in
|
||||
For the simulation carried out in this study the HM reference software, version $16.2$, was used \cite{HEVCReferenceSoftware162}.
|
||||
The sequences were encoded with an \ac{idr} period of $16$ frames and all the coding modes enabled, following two recommended configurations: \ac{ld}, i.e., P-frames with 4 references frames, and \ac{ra}, i.e., B-frames with 2 reference frames in each list~\cite{HEVCCommonTests2012}.
|
||||
These prediction structures cover the use of P-frames and hierarchical B-frames~\cite{H264ScalableOverviewSep2007}, which are commonly used in HEVC streaming~\cite{RealTimeHEVC2016}.
|
||||
In order to achieve high quality images for all coded sequences (i.e., \ac{psnr} around 40 dB) different bitrates were used, as listed in Table~\ref{TableTestSequences}. The filtering operations were disabled at slices boundaries in order to keep the slices self-contained.
|
||||
In order to achieve high quality images for all coded sequences (i.e., \ac{psnr} around 40 dB) different bitrates were used, as listed in Table~\ref{TableTestSequences}. The filtering operations were disabled at slices boundaries in order to keep the slices self-contained.
|
||||
Moreover, the ratio of inter-coded blocks of each coded sequence using the default coding parameters is also shown in the third column of the table to illustrate the predictions distribution.
|
||||
|
||||
|
||||
%The following bitrates were selected to achieve high quality (i.e., around 40 dB): 1.0 Mbits/s (Kendo), 1.5 Mbits/s (Book Arrival), 4.0 Mbits/s (BQSquare, Park Scene and Race Horses) and 12.0 Mbits/s (People on Street).
|
||||
% At the decoder side, error concealment is based on motion compensation from previously decoded frames.
|
||||
% At the decoder-side, error concealment is based on motion compensation from previously decoded frames.
|
||||
% The motion field of a missing frame is obtained applying \ac{mv} extrapolation from the two closest neighbours.
|
||||
% Afterwards, the final \acs{mv} are selected based on the residual energy of their original prediction~\cite{HEVCMCResidueDec2013}.
|
||||
To simulate the lossy transmission environment, each frame is encoded into several slices, each one packetized into one \ac{nal} unit, which are then transmitted as the payload of independent packets.
|
||||
@@ -97,17 +97,17 @@ The filtering operations were disabled at slices boundaries in order to keep the
|
||||
|
||||
\subsection{Evaluation of the impact of the partition units size}
|
||||
|
||||
%The \ac{hevc} standard introduced new partitioning schemes to allow coding improvements in high resolution videos. These partition structures, named \glsfirst{cu}, are allowed to be as large as $64\times64$ pixels.
|
||||
%The \ac{hevc} standard introduced new partitioning schemes to allow coding improvements in high resolution videos. These partition structures, named \glsfirst{cu}, are allowed to be as large as $64\times64$ pixels.
|
||||
In the first set of experiments the impact of the maximum \ac{cu} size in the decoded video quality is investigated. In this experiment the maximum \ac{cu} size is set to three different sizes, i.e., 16$\times$16, 32$\times$32 and 64$\times$64 pixels. The lower size is targeting the \ac{mb} size used in the \ac{h264}.
|
||||
|
||||
Table~\ref{Table3:Quality_BlockSize} shows the average \ac{psnr} obtained for the three test conditions and for four different packet loss ratios. Results for the error-free scenario shows a consistent quality increase as the maximum \ac{cu} size increases, up to a maximum of 1.41 dB for the Tennis sequence when comparing the maximum \ac{cu} size of $64\times64$ with $16\times16$.
|
||||
Table~\ref{Table3:Quality_BlockSize} shows the average \ac{psnr} obtained for the three test conditions and for four different packet loss ratios. Results for the error-free scenario shows a consistent quality increase as the maximum \ac{cu} size increases, up to a maximum of 1.41~dB for the Tennis sequence when comparing the maximum \ac{cu} size of $64\times64$ with $16\times16$.
|
||||
This confirms the principle used by the \ac{hevc} standard, that allowing larger coding units lead to higher coding efficiency, achieving a maximum of 1.41~dB for the Tennis sequence which has the higher \ac{ti} parameter (see Table~\ref{TableTestSequences}). Moreover, it is also noticeable that sequences with higher amount of inter-coded blocks take less advantage of the larger \acs{cu}, by the results of the BQSquare sequence (i.e., highest ratio of inter-coded blocks and lowest quality gain in the error-free case).
|
||||
Overall, comparing the results for the error-free and data loss cases it is noticeable that similar quality gains are obtained, which are achieved due to the superior coding efficiency of larger \acs{cu}. In most of the cases the quality gains slightly decrease for increasing \acs{plr}.
|
||||
Therefore, one may conclude that the error robustness of \ac{hevc} streams is not affected by limiting the maximum \ac{cu} size.
|
||||
Overall, comparing the results for the error-free and data loss cases it is noticeable that similar quality gains are obtained, which are achieved due to the superior coding efficiency of larger \acs{cu}. In most of the cases the quality gains slightly decrease for increasing \acs{plr}.
|
||||
Therefore, one may conclude that the error robustness of \ac{hevc} streams is not affected by limiting the maximum \ac{cu} size.
|
||||
%\todo{Complete discussion of partition unit size}
|
||||
|
||||
|
||||
%Results for the case of data loss indicate that using larger coding units leads to higher video quality.
|
||||
%Results for the case of data loss indicate that using larger coding units leads to higher video quality.
|
||||
%
|
||||
%that larger coding units should be maintained. Even though larger frame partitions lead to larger coding dependencies, it will not affect the robustness to packet loss and error propagation. This is mainly due to the fact that error propagation occurs mostly in complex and moving regions, which normally are coded using smaller partition units.
|
||||
|
||||
@@ -119,7 +119,7 @@ Therefore, one may conclude that the error robustness of \ac{hevc} streams is no
|
||||
\input{./031_ErrorResilienceEvaluation/QualityEvaluation/TableCUSizes.tex}
|
||||
\end{table}
|
||||
|
||||
Figure~\ref{Figure3:BlockPartitionBask} illustrates an example of the frame partitions for a region of the Basketball Drill sequence for two cases: (i) maximum \ac{cu} size of $64\times64$ and (i) maximum \ac{cu} size of $16\times16$.
|
||||
Figure~\ref{Figure3:BlockPartitionBask} illustrates an example of the frame partitions for a region of the Basketball Drill sequence for two cases: (i) maximum \ac{cu} size of $64\times64$ and (i) maximum \ac{cu} size of $16\times16$.
|
||||
As shown, in the moving regions, e.g., moving player, the partitions obtained for both cases have a similar pattern, mainly composed of \acs{cu} with small size. Larger \acs{cu} are only used in the floor which has a lower spatial complexity and suffers small changes between frames, therefore, easier to compress with $64\times64$ \acs{cu}. Moreover, those regions can be recovered more accurately using neighbouring information (spatial and temporal) correctly decoded, thus not incurring in severe error propagation.
|
||||
Therefore, the overall error robustness is not affected by reducing the maximum \ac{cu} size. One may conclude that the use of larger coding units is able to improve the overall \ac{hevc} efficiency in error-free and error prone applications.
|
||||
|
||||
@@ -156,8 +156,8 @@ Therefore, the overall error robustness is not affected by reducing the maximum
|
||||
|
||||
\subsection{Evaluation of the impact of intra refreshing}
|
||||
|
||||
In this sub-section the relation between the temporal distance of consecutive intra-coded frames, i.e., period of the intra frames, and the video quality under transmission errors is evaluated.
|
||||
As the intra-coded frames have no dependencies to prior coded pictures, they introduce a refreshing point into the bitstream, which stops temporal error propagation along several frames or slices.
|
||||
In this sub-section the relation between the temporal distance of consecutive intra-coded frames, i.e., period of the intra frames, and the video quality under transmission errors is evaluated.
|
||||
As the intra-coded frames have no dependencies to prior coded pictures, they introduce a refreshing point into the bitstream, which stops temporal error propagation along several frames or slices.
|
||||
This allows the decoder to recover from frame/slice loss and continue error-free decoding seamlessly.
|
||||
|
||||
\begin{figure}[!t]
|
||||
@@ -168,21 +168,21 @@ This allows the decoder to recover from frame/slice loss and continue error-free
|
||||
\label{Figure3:QualityvsIntra}
|
||||
\end{figure}
|
||||
|
||||
Figure~\ref{Figure3:QualityvsIntra} shows the average decoded video quality (\ac{psnr}) for different distance of intra frames, covering \acs{plr} between 3\% and 5\%.
|
||||
Results in the figure indicate that as the distance between intra frames increases, the decoded video quality significantly decreases.
|
||||
Figure~\ref{Figure3:QualityvsIntra} shows the average decoded video quality (\ac{psnr}) for different distance of intra frames, covering \acs{plr} between 3\% and 5\%.
|
||||
Results in the figure indicate that as the distance between intra frames increases, the decoded video quality significantly decreases.
|
||||
The quality decreasing is especially noticeable for intra periods higher than 32 frames, indicating that temporal distances greater than approximately one second are not suitable for video streaming under error-prone conditions.
|
||||
When a high temporal distance between intra frames is used, the quality decreasing is more noticeable in the People On Street and BQSquare sequences, which have higher spatial complexity (see values for \ac{si}), leading to a reduction of 13 dB and 14 dB, respectively. As spatial complexity increases the encoder rely more on temporal predictions making the bitstream more prone to error propagation.
|
||||
When a high temporal distance between intra frames is used, the quality decreasing is more noticeable in the People On Street and BQSquare sequences, which have higher spatial complexity (see values for \ac{si}), leading to a reduction of 13 dB and 14 dB, respectively. As spatial complexity increases the encoder rely more on temporal predictions making the bitstream more prone to error propagation.
|
||||
%\todo{Complete discussion of intra refreshing}
|
||||
|
||||
One should also note from the results of Figure~\ref{Figure3:QualityvsIntra} that using a small distance between intra frames is not always more efficient.
|
||||
Since intra frames require higher amount of bits, the overall coding efficiency is reduced, resulting in an overall quality that decreases even in data loss conditions.
|
||||
This is clear noticeable for the Four People sequence, where increasing the intra period from 8 to 16 leads to a quality increasing of 1 dB for 5\% of \ac{plr}.
|
||||
One should also note from the results of Figure~\ref{Figure3:QualityvsIntra} that using a small distance between intra frames is not always more efficient.
|
||||
Since intra frames require higher amount of bits, the overall coding efficiency is reduced, resulting in an overall quality that decreases even in data loss conditions.
|
||||
This is clear noticeable for the Four People sequence, where increasing the intra period from 8 to 16 leads to a quality increasing of 1 dB for 5\% of \ac{plr}.
|
||||
For the remaining sequences it can be concluded that using sixteen frames (not higher than half a second) between refresh points (i.e., intra frames) achieves an acceptable efficiency under errors. This provides an efficient trade-off between compression efficiency and error robustness in case of data loss.
|
||||
|
||||
\subsection{Evaluation for using different slice partitioning schemes}
|
||||
|
||||
The partitioning of the \acs{ctu} into different slices increase the flexibility of video streams to be partitioned into different packets, thus adapting them to the video transmission system.
|
||||
By partitioning the coded video frame into different slices spatial refresh points are introduced, limiting the error propagation across different \acs{ctu}.
|
||||
The partitioning of the \acs{ctu} into different slices increase the flexibility of video streams to be partitioned into different packets, thus adapting them to the video transmission system.
|
||||
By partitioning the coded video frame into different slices spatial refresh points are introduced, limiting the error propagation across different \acs{ctu}.
|
||||
In this sub-section the amount of bytes per slice is evaluated, in order to evaluate the influence of different schemes on the error robustness of \ac{hevc} streams. The quality obtained different mode is compared in order to find the most efficient that suits for video transmission under data loss conditions.
|
||||
|
||||
\begin{figure}[!t]
|
||||
@@ -194,11 +194,11 @@ In this sub-section the amount of bytes per slice is evaluated, in order to eval
|
||||
\label{Figure3:SlicePartitioning}
|
||||
\end{figure}
|
||||
|
||||
Figure~\ref{Figure3:SlicePartitioning} shows the average \ac{psnr} for four difference slice partitioning modes and different \acs{plr}.
|
||||
The first three modes are characterised by a different amount of bytes per slice (B/S), namely 1200, 2400 and 4800 bytes, respectively. In these cases each frame has a variable number of slices.
|
||||
The last mode uses a single slice per frame (referred to as \textit{SS}), corresponding to the case where slice partitioning is disabled.
|
||||
Figure~\ref{Figure3:SlicePartitioning} shows the average \ac{psnr} for four difference slice partitioning modes and different \acs{plr}.
|
||||
The first three modes are characterised by a different amount of bytes per slice (B/S), namely 1200, 2400 and 4800 bytes, respectively. In these cases each frame has a variable number of slices.
|
||||
The last mode uses a single slice per frame (referred to as \textit{SS}), corresponding to the case where slice partitioning is disabled.
|
||||
Results in the figure reveals a clear impact of the slice partitioning mode on the error robustness of the \ac{hevc}, as the quality degradation for increasing \ac{plr} significantly changes for each mode.
|
||||
Comparing the cases with fixed amount of bytes per slice, it is clearly noticeable that when higher amount of bytes are used the quality degradation increases. This is true for the results of most of the test sequences shown in Figure~\ref{Figure3:SlicePartitioning}.
|
||||
Comparing the cases with fixed amount of bytes per slice, it is clearly noticeable that when higher amount of bytes are used the quality degradation increases. This is true for the results of most of the test sequences shown in Figure~\ref{Figure3:SlicePartitioning}.
|
||||
Although the use of smaller slices increases the overhead used for slice headers, the superior error robustness of achieved with 1200 B/S is able to overcome the inherent degradation on coding efficiency.
|
||||
This is due to the lack of refresh points within a frame, resulting in errors that affects larger regions of the frame which are propagated to the subsequent inter predicted frames.
|
||||
|
||||
@@ -210,7 +210,7 @@ Results for Kendo sequence does not follow the main trend. This can be explained
|
||||
\subsection{Motion vector coding impact on error propagation}
|
||||
|
||||
|
||||
In \ac{hevc}, the new \ac{mv} prediction modes known as \ac{amvp} and Merge Mode~\cite{MotionCodingHEVC2011} may be significant contributors to poor performance under transmission errors. These modes are characterised by coding the motion information using previously encoded \acs{mv}, which obviously leads to prediction mismatches at the decoder side when such neighbouring information is missing due to network losses.
|
||||
In \ac{hevc}, the new \ac{mv} prediction modes known as \ac{amvp} and Merge Mode~\cite{MotionCodingHEVC2011} may be significant contributors to poor performance under transmission errors. These modes are characterised by coding the motion information using previously encoded \acs{mv}, which obviously leads to prediction mismatches at the decoder-side when such neighbouring information is missing due to network losses.
|
||||
Therefore, in this sub-section the impact of using spatial and temporal \ac{mv} candidates on the decoded video quality under errors is evaluated.
|
||||
|
||||
\subsubsection{Spatial \ac{mv} candidates}
|
||||
@@ -223,9 +223,9 @@ Therefore, in this sub-section the impact of using spatial and temporal \ac{mv}
|
||||
\label{Figure3:AvgPSNRMergeCandidates}
|
||||
\end{figure}
|
||||
|
||||
The first set of experiments aimed to investigate the impact of the amount of spatial \ac{mv} candidates in the video quality.
|
||||
The first set of experiments aimed to investigate the impact of the amount of spatial \ac{mv} candidates in the video quality.
|
||||
As mentioned before, \ac{hevc} allows up to five candidates to be used, which can be provided solely by spatial neighbours. In order to check the impact of the \ac{mv} candidates, different coding configurations were used, each one characterised by different amounts of spatial \ac{mv} candidates, i.e., 1, 3 and 5 \acs{mv}.
|
||||
Figure~\ref{Figure3:AvgPSNRMergeCandidates} shows the average quality results for the Basketball Drill and People On Street sequences under different packet loss conditions.
|
||||
Figure~\ref{Figure3:AvgPSNRMergeCandidates} shows the average quality results for the Basketball Drill and People On Street sequences under different packet loss conditions.
|
||||
The results show that, besides the number of \ac{mv} candidates, similar quality decreasing is achieved as the \ac{plr} increases. Therefore, one may conclude that varying the number of available \ac{mv} candidate does not affect the error resilience of the \ac{hevc}.
|
||||
|
||||
The occurrence rate for each spatial \ac{mv} candidate was analysed in order to check whether they are uniformly used.
|
||||
@@ -276,7 +276,7 @@ On the contrary, when the temporal \ac{mv} candidate is disabled, the motion inf
|
||||
|
||||
In order to further evaluate the impact of the temporal \ac{mv} dependencies on the error propagation, the \ac{hevc} streams were subject to random packet losses.
|
||||
Table~\ref{Table3:AvgPSNRWithoutTMVP} presents the average quality results, measured using the \ac{psnr}.
|
||||
As expected, the results indicate that in error-free conditions higher quality is achieved by enabling the \ac{tmvp} candidate, since higher compression efficiency is obtained because more \ac{mv} predictors are used.
|
||||
As expected, the results indicate that in error-free conditions higher quality is achieved by enabling the \ac{tmvp} candidate, since higher compression efficiency is obtained because more \ac{mv} predictors are used.
|
||||
This is true for all both test conditions, \emph{i.e.}, both \ac{ld} and \ac{ra}.
|
||||
However, in the presence of errors, the video quality significantly decreases due to low error robustness when temporal \ac{mv} candidates are used. Alternatively, average quality up to 5.17 dB of \ac{psnr} when the temporal \ac{mv} predictors are disabled is achieved in the \ac{ld} configuration.
|
||||
The results also show that the negative impact of using temporal \ac{mv} dependencies is smaller when B-frames are used. For the Random-Access, the average gain of disabling the \ac{tmvp} is 0.78 dB.
|
||||
@@ -294,14 +294,14 @@ Moreover, the results also indicate that for higher loss ratios the negative imp
|
||||
|
||||
\subsection{Remarks}
|
||||
|
||||
Overall, the results presented in the previous sub-sections show that \ac{hevc} has poor robustness against transmission errors, resulting in significant quality degradation, though dependent still on the video content and coding options.
|
||||
As shown, some coding options, such as slice partition and intra refreshing period, have high correlation with the decoded video quality when transmission errors affect the compressed streams.
|
||||
Overall, the results presented in the previous sub-sections show that \ac{hevc} has poor robustness against transmission errors, resulting in significant quality degradation, though dependent still on the video content and coding options.
|
||||
As shown, some coding options, such as slice partition and intra refreshing period, have high correlation with the decoded video quality when transmission errors affect the compressed streams.
|
||||
Therefore, these are important aspects to take into consideration in the design an implementation of broadcasting video services and applications.
|
||||
|
||||
One of the main improvements of the \ac{hevc} standard is the support for larger coding units up to $64\times64$, which does not have a negative impact the error robustness of coded video streams, indicating that they should be allowed to achieve higher coding performance.
|
||||
One of the main improvements of the \ac{hevc} standard is the support for larger coding units up to $64\times64$, which does not have a negative impact the error robustness of coded video streams, indicating that they should be allowed to achieve higher coding performance.
|
||||
Regarding the use of neighbouring information for \ac{mv} coding, on the one hand the spatial \ac{mv} candidates do not significantly decrease the error robustness. On the other hand, the use of temporal \ac{mv} candidates significantly contribute to increase the temporal dependencies, leading to more severe quality degradation in the presence of packet loss.
|
||||
|
||||
Based on these evidences, the next section presents an efficient method to improve the error robustness of \ac{hevc}.
|
||||
Based on these evidences, the next section presents an efficient method to improve the error robustness of \ac{hevc}.
|
||||
The proposed method is based on the underlying principle of reducing the mismatch of \ac{mv} predictions, by selectively disable some \ac{mv} predictions at the encoder-side, reducing the overall temporal dependencies.
|
||||
|
||||
|
||||
@@ -313,19 +313,19 @@ The proposed method is based on the underlying principle of reducing the mismatc
|
||||
|
||||
%\subsection{Problem formulation}
|
||||
|
||||
The aforementioned results revealed low error resilience in motion prediction because any error is easily propagated through several frames.
|
||||
The aforementioned results revealed low error resilience in motion prediction because any error is easily propagated through several frames.
|
||||
Based on this evidence, this section proposes a method to make intelligent use of temporal \ac{mv} candidates during the motion estimation process, in order to decrease the temporal dependency, and improve the error resiliency without penalising the \ac{rd} performance.
|
||||
|
||||
\subsection{Proposed method}
|
||||
|
||||
\glsreset{rd}
|
||||
|
||||
The new approach to deal with temporal motion vector dependencies is accomplished by modifying the \ac{rd} optimisation that is used to select the best motion vector for inter-prediction.
|
||||
The new approach to deal with temporal motion vector dependencies is accomplished by modifying the \ac{rd} optimisation that is used to select the best motion vector for inter-prediction.
|
||||
The objective is to increase the number of \acs{mv} that can be independently decoded to improve the error recovery capability after a frame loss event.
|
||||
|
||||
The proposed method also overcomes the limitation of the method presented in \cite{DynamicTMVP2011}, which break the motion information dependencies between frames.
|
||||
Instead of disabling the temporal motion candidate at the frame level, the proposed method aims to remove the dependencies at the \ac{cu} level.
|
||||
To achieve this goal, the temporal dependency of all \ac{mv} candidates is firstly analysed for each block, in order to select on the suitable candidates to be used as predictors for the current \ac{mv}.
|
||||
Instead of disabling the temporal motion candidate at the frame level, the proposed method aims to remove the dependencies at the \ac{cu} level.
|
||||
To achieve this goal, the temporal dependency of all \ac{mv} candidates is firstly analysed for each block, in order to select on the suitable candidates to be used as predictors for the current \ac{mv}.
|
||||
Based on the analysis of the temporal dependency, some temporal candidates are marked as unsuitable predictors of the current \ac{mv}, and for this reason are not selected in the Merge mode. In the proposed approach, a temporal \ac{mv} candidate is marked as unsuitable based on the following criteria:
|
||||
\begin{enumerate}
|
||||
\item if it is encoded based on another temporal \ac{mv} candidate from a previously encoded frame;
|
||||
@@ -341,8 +341,8 @@ Based on the analysis of the temporal dependency, some temporal candidates are m
|
||||
\FIGSVSPACE
|
||||
\end{figure}
|
||||
|
||||
Figure \ref{ProposedDynamicMethod} can be used to explain the proposed method, illustrating different blocks and different \ac{mv} dependencies represented with arrows. Note that the squares illustrate a generic block size; the representation of the temporal dependencies is simplified, and in some cases may not correspond to the exact co-located block in the previous frame. In frame $f_{t-1}$ there are several \acs{mv} encoded using spatial predictions and one \ac{mv} encoded using a temporal prediction (arrow pointing to $f_{t-2}$). In frame $f_{t}$, the \ac{mv} corresponding to the block (1) may use the temporal \ac{mv} candidate, since the co-located block in $f_{t-1}$ is not temporally dependent.
|
||||
However, in the proposed scheme the \ac{mv} prediction marked with the arrow (2) is not allowed, thus preventing the propagation of temporal dependencies.
|
||||
Figure \ref{ProposedDynamicMethod} can be used to explain the proposed method, illustrating different blocks and different \ac{mv} dependencies represented with arrows. Note that the squares illustrate a generic block size; the representation of the temporal dependencies is simplified, and in some cases may not correspond to the exact co-located block in the previous frame. In frame $f_{t-1}$ there are several \acs{mv} encoded using spatial predictions and one \ac{mv} encoded using a temporal prediction (arrow pointing to $f_{t-2}$). In frame $f_{t}$, the \ac{mv} corresponding to the block (1) may use the temporal \ac{mv} candidate, since the co-located block in $f_{t-1}$ is not temporally dependent.
|
||||
However, in the proposed scheme the \ac{mv} prediction marked with the arrow (2) is not allowed, thus preventing the propagation of temporal dependencies.
|
||||
Moreover, the block corresponding to the arrow (3) cannot use the temporal \ac{mv} candidate since it already depends on a \ac{mv} that was previously encoded using the temporal candidate. The proposed method reduces the temporal dependencies of the \glspl{mv} at the block level, providing improved error robustness to the encoded streams. Using this approach, the dependencies are selectively removed, improving the error recovery, without fully disabling the temporal \gls{mv} candidates for a given frame. Therefore, coding efficiency is not significantly penalised.
|
||||
|
||||
|
||||
@@ -361,8 +361,8 @@ The performance of the proposed scheme (\PROP) was evaluated against two other a
|
||||
|
||||
\subsubsection{Error propagation under single loss events}
|
||||
|
||||
Firstly, the performance of the proposed method was measured under a single loss event (i.e., only one frame is affected by errors).
|
||||
Figure \ref{Figure3:ProposedMethodEvaluationSingle} shows the error propagation for the different methods.
|
||||
Firstly, the performance of the proposed method was measured under a single loss event (i.e., only one frame is affected by errors).
|
||||
Figure \ref{Figure3:ProposedMethodEvaluationSingle} shows the error propagation for the different methods.
|
||||
The results show a significant quality difference between the proposed method, the \TIDR method, and the reference \ac{hevc}.
|
||||
When the temporal \ac{mv} candidate is disabled (proposed and \TIDR method) more error robustness can be achieved.
|
||||
The results also show the relevance of using the proposed method, as the \TIDR method is only able to recover from the frame loss after the refreshing frame, \emph{i.e.}, frame \#8, especially for the BQSquare sequence
|
||||
@@ -377,13 +377,14 @@ The results also show the relevance of using the proposed method, as the \TIDR m
|
||||
\input{./031_ErrorResilienceEvaluation/DynamicTmvp/TableDynamicTMVPPSNRSTD.tex}
|
||||
\end{table}
|
||||
|
||||
Secondly, further simulations were performed to evaluate the effectiveness of the proposed error resiliency method under realistic packet loss patterns.
|
||||
Secondly, further simulations were performed to evaluate the effectiveness of the proposed error resiliency method under realistic packet loss patterns.
|
||||
%As in previous experiments, each coded frame is transmitted in one packet.
|
||||
Table~\ref{Table3:AverageStdPSNR} illustrates the average standard deviation of the PSNR, for three of the test sequences.
|
||||
The results show that increasing the \ac{plr} leads to higher quality deviations, and the impact of errors in the subjective quality experience is higher.
|
||||
However, reducing the temporal dependencies of the \acs{mv} using the proposed method decreases the quality differences variation, achieving the lowest value for all scenarios tested. The proposed method is able to reduce the quality difference variations up to 0.94 dB in Kendo sequence (3$\%$ of packet loss) compared to the~\TIDR~method.
|
||||
Table~\ref{Table3:AverageStdPSNR} illustrates the average standard deviation of the PSNR, for three of the test sequences.
|
||||
The results show that increasing the \ac{plr} leads to higher quality deviations, and the impact of errors in the subjective quality experience is higher.
|
||||
However, reducing the temporal dependencies of the \acs{mv} using the proposed method decreases the quality differences variation, achieving the lowest value for all scenarios tested. The proposed method is able to reduce the quality difference variations up to 0.94 dB in Kendo sequence (3$\%$ of packet loss) compared to the~\TIDR~method.
|
||||
Since the proposed method achieves the lowest \ac{psnr} standard deviation, one may conclude that it is able to reduce the quality variations leading to a better visual quality.
|
||||
|
||||
\HIGHLIGHT{vantagem do std}
|
||||
%\HIGHLIGHT{vantagem do std}
|
||||
|
||||
\begin{table}[tbp]
|
||||
\centering
|
||||
@@ -404,10 +405,10 @@ The effectiveness of the proposed method increases at higher packet loss rates (
|
||||
\section{Summary}
|
||||
\label{Section3:Summary}
|
||||
|
||||
In this chapter, the error resilience performance of \ac{hevc} was studied in detail under error prone conditions.
|
||||
In this chapter, the error resilience performance of \ac{hevc} was studied in detail under error prone conditions.
|
||||
The simulation results revealed that new coding tools introduced in \ac{hevc} to improve coding efficiency are also responsible for higher error propagation.
|
||||
This motivates further research on new techniques to be developed for highly efficient encoders and decoders, in order to increase error resiliency and to improve the overall performance of video broadcast.
|
||||
Then, a new approach to deal with the weak robustness of \ac{mv} coding was proposed, addressing \ac{mv} dependencies at \ac{cu} level.
|
||||
The proposed method selectively disables the temporal \ac{mv} candidates, in order to break the dependencies across more than one frame. The experimental evaluation was able to demonstrate that the proposed method increases the error robustness and outperforms existing state-of-the-art methods and the reference codec.
|
||||
The results also show that the proposed method is able to provide a good trade-off between coding efficiency and error resilience.
|
||||
Then, a new approach to deal with the weak robustness of \ac{mv} coding was proposed, addressing \ac{mv} dependencies at \ac{cu} level.
|
||||
The proposed method selectively disables the temporal \ac{mv} candidates, in order to break the dependencies across more than one frame. The experimental evaluation was able to demonstrate that the proposed method increases the error robustness and outperforms existing state-of-the-art methods and the reference codec.
|
||||
The results also show that the proposed method is able to provide a good trade-off between coding efficiency and error resilience.
|
||||
Overall. these new findings demonstrate that selective usage of temporal \ac{mv} candidates is an efficient approach to enhance error resilience in high efficiency video encoders.
|
||||
Reference in new issue
Block a user