Final revised version

This commit is contained in:
jfmcarreira committed 2018-12-27 23:17:31 +00:00
1 parent 59d007205a
commit 3b63e37cc4
14 files changed
+119 -118

No files matched your search

@@ -104,7 +104,7 @@ However, such solution would not take into consideration coding efficiency, sinc
\begin{figure}[tb]
\centering
\input{./041_ImprovedErrorResilience/Figures/RefPicSelection.tikz}
\caption{Example of the reference picture selection applied to the frame $C$ based on the checkerboard at \ac{ctu} level (\acs{pu} are represented by small partitions).}
\caption{Example of the reference picture selection applied to the current frame based on the checkerboard at \ac{ctu} level (\acs{pu} are represented by small partitions).}
\label{fig:checkerboardRefSelection}
\end{figure}
@@ -114,7 +114,7 @@ This leads to the overall goal of finding the optimum R-D coding mode for each \
\sigma_C=std\Big(\mathbf{N}_C(r_i) \Big), i=1, 2,\cdots S,
\label{eq:StdX}
\end{equation}
where $\sigma_C$ measures the deviation from the checkerboard solution, which corresponds to $\sigma_C = 0$, since all possible reference frames $r_i\in F$ are uniformly used throughout the \acs{pu} of $C$.
\HIGHLIGHT{where $\sigma_C$ measures the deviation from the checkerboard solution, which corresponds to $\sigma_C = 0$, since all possible reference frames $r_i\in F$ are uniformly used throughout the \acs{pu} of $C$.}
To attain the goal defined above, the best coding mode for each \ac{pu} is chosen through a Lagrangian \ac{rd} optimisation that tends to minimise $\sigma_C$.
This is done by penalising (or benefiting) the \ac{rd} cost of those coding modes which intend to use reference frames that have already been used more (or less) frequently than their average use in previous \acs{pu} of the current frame.
@@ -125,7 +125,7 @@ Such approach allows to select the reference frames, by independently processing
J(\phi) &= \Big( D(\phi) + \lambda \times R(\phi) \Big) \times e^{W(r_{\phi})},
\label{eq:Lagrangian}
\end{align}
where $D(\phi)$ and $R(\phi)$ are the distortion and rate, respectively, associated with the coding mode $\phi$ and $J(\phi)$ is the trade-off between distortion and rate.
\HIGHLIGHT{where $D(\phi)$ and $R(\phi)$ are the distortion and rate, respectively, associated with the coding mode $\phi$. $J(\phi)$ is Lagrangian cost that measures the the trade-off between distortion and rate.}
The exponential weighting factor $e^{W(r_{\phi})}$ is used to penalise (or benefit) the Lagrangian cost of a given coding mode $\phi$ associated to reference frame $r_\phi$.
$W(r_{\phi})$ is obtained according to the global and local deviation between the number of times that $r_\phi$ was used and the average use of all $r_i \in F$ across the \acs{pu} of current frame $C$, as follows,
\begin{equation}\label{eq:KFactor}
@@ -170,7 +170,7 @@ For instance, for $\gamma=5$, when a reference frame $r_\phi$ is used to predict
& \footnotesize $r_{1}$: $61.8\%$ | $r_{2}$: $25.7\%$ & \footnotesize $r_{1}$: $42.2\%$ | $r_{2}$: $39.3\%$ & \footnotesize $r_{1}$: $45.22\%$ | $r_{2}$: $43.2\%$ \\
\end{tabular}
}
\caption{Example of the partitions and references pictures selected using the different encoding methods (the dark gray squares indicate that the \ac{mv} is using the reference $r_{1}$, light~gray squares corresponds to the reference $r_{2}$ and unfilled regions are intra-coded blocks).}
\caption{Example of the partitions and references pictures selected using the different encoding methods (the dark gray squares indicate that the \ac{mv} is using the reference $r_{1}$, light~gray squares correspond to the reference $r_{2}$ and unfilled regions are intra-coded blocks).}
\label{fig:RefPicSelectionExample}
\FIGSVSPACE
\end{figure*}
@@ -232,10 +232,9 @@ where $R_U$ is the amount of bits required to encode the $n$ \acs{mv} of $U$ and
A_{D}(i) = \sum_{j=1}^{DV_i} ( w_j\times h_j ),\hspace{2mm} i = 1, 2 \cdots s,
\end{equation}
where $DV_i$ is the number of \acs{mv} dependent from $MV_i$, and $w_j$ and $h_j$ are the width and height of the corresponding \ac{pu}.
A weighting factor is used to control the amount of bitrate overhead ($R_U$), by increasing the cost of the number of encoded bits based on the duration of the error propagation.
This duration is given by the ratio between the distance from the current frame ($t_C$) to the previous refresh frame ($t_I$) and the \ac{idr} period ($T_{I}$).
This weight varies in the range $\left[0;1 \right]$, achieving the lowest value for intra-coded frames and the maximum for the frame located immediately before an intra-coded one.
Moreover, in the proposed approach, the parameter $\alpha$ is used to globally control the redundancy for the entire stream, resulting in higher values of $\alpha$, less redundant \acs{mv} are selected.
\HIGHLIGHT{A weighting factor is used to control the amount of bitrate overhead ($R_U$), by increasing the cost of the number of encoded bits based on the duration of the error propagation.
This duration ($t_C-t_I / T_{I}$) is given by the ratio between the distance from the current frame ($t_C$) to the previous refresh frame ($t_I$) and the \ac{idr} period ($T_{I}$) and varies in the range $\left[0;1 \right]$, achieving the lowest value for intra-coded frames and the maximum for the frame located immediately before an intra-coded one.
The parameter $\alpha$ is used as a global control for redundancy of the entire stream, where higher values of $\alpha$ correspond to less redundant \acs{mv}.}
As pointed in Section~\ref{Section4:Architecture}, this streaming stage operates on coded data and may be implemented at any point of the network, to increase robustness without fully decoding the stream. Note that the parsing operation represented in Figure~\ref{fig:proposedMethod} does not include decoding of the video frames. Furthermore, only one frame delay is introduced because the \ac{mv} selection process is performed on a frame-by-frame basis.
To maintain compatibility with the \ac{hevc} standard, the redundant motion information is transmitted using the \ac{sei} \ac{nal} units \cite{HEVCHighLevelReferenceDez2012}. Thus, this information can be multiplexed into the coded bitstream without affecting the compressed video.
@@ -285,7 +284,7 @@ This value was chosen to allow increasing the Lagrangian cost by about $50\%$,
PSNR = 31.93 dB & PSNR = 33.83 dB & PSNR = 24.12 dB & PSNR = 31.93 dB \\
\end{tabular}
}
\caption{Zoom of recovered Frame \#10 after a loss event at Frame \#9 for Basketball Drill and Race Horses sequence.}
\caption{Zoom of recovered Frame \#10 after a loss event at Frame \#9 for Basketball Drill and Race Horses sequences.}
\label{fig:RecoveredExample}
\end{figure}
@@ -303,9 +302,8 @@ Summarising, the reconstructed frames illustrated in Figure~\ref{fig:RecoveredEx
\begin{figure}[t]
\renewcommand{\FIGSIZE}{0.8\columnwidth}
\centering
% \includegraphics[width=\FIGSIZE]{./041_ImprovedErrorResilience/Results/PSNRversusFrameKendo}
\input{041_ImprovedErrorResilience/Results/PSNRversusFrameKendo.tex}
\caption{Error propagation when errors affect Frame \#7 in Kendo sequence with an \ac{idr} period of 32 frames for the proposed reference frame selection algorithm in comparison with existing methods.}
\input{041_ImprovedErrorResilience/Results/PSNRversusFrameKendo.tex}
\caption{Error propagation when errors affect Frame \#7 in Kendo sequence (\ac{idr} period of 32 frames) for the proposed reference frame selection algorithm in comparison with existing methods.}
\label{fig:PSNRvsFrameForRefPicSel}
\end{figure}
@@ -342,20 +340,21 @@ Taking into account the above results and the spatio-temporal information shown
%\includegraphics[width=\FIGSIZE,page=1]{./041_ImprovedErrorResilience/Results/PSNRversusGama}
%\includegraphics[width=\FIGSIZE,page=2]{./041_ImprovedErrorResilience/Results/PSNRversusGama}
\input{./041_ImprovedErrorResilience/Results/PSNRversusGama.tex}
\caption{Influence of the $\gamma$ parameter in the average quality gains, comparing the proposed method with \REF, for different \acs{plr}.}
\caption{Influence of the $\gamma$ parameter in the average quality gains: comparison between the proposed method and \REF, for different \acs{plr}.}
\label{fig:QualityVsGamma}
\end{figure}
The influence of $\gamma$ parameter used in (\ref{eq:KFactor}) on the video quality was also evaluated, in order to check how the cost constrains affect the performance of the proposed method.
Figure~\ref{fig:QualityVsGamma} shows the average quality gains of the proposed method in comparison to the reference \ac{hevc} (\REF) across all test sequences, for different values of $\gamma$.
For the \ac{plf} case, the results in this figure show that increasing $\gamma$ leads to a decrease in the coding efficiency, especially in the \ac{ld} configuration.
When no packets are lost, the results in this figure show that increasing $\gamma$ leads to a decrease in the coding efficiency, especially in the \ac{ld} configuration.
The reason is that higher values of $\gamma$ lead to higher values of $W(r_{\phi})$, therefore the reference picture selection cost is severely constrained by the exponential weights.
Since $T_{ID}$ is constant for all frames in the \ac{ld}, higher weights are used for all frames ($T_{ID}$ is used to reduce the weights in case of hierarchical coding) and the encoder is not able to select the best reference frame in the \ac{rd} sense.
In the case where packet loss occurs, higher values of $\gamma$ lead to improved error resilience.
Since higher weights are used in the \ac{rd} cost, the encoder is constrained to distribute the use of the reference frames of different \acs{pu}, reducing the error propagation.
For very high values (e.g., $\gamma=1000$) the encoder is forced to uniformly select the reference frames, resulting in a fixed checkerboard pattern.
\HIGHLIGHT{Overall, the $\gamma$ parameter is able to control the reference frame selection constrains,
thus achieving better trade-off between improved error resilience and loss of coding efficiency. The value of this parameter should be selected accordingly with the requirements and condition of the streaming application, e.g., network bandwidth or packet loss ratio.}
Overall, the $\gamma$ parameter is able to control the reference frame selection constrains,
which allows to achieve a better trade-off between improved error resilience and loss of coding efficiency.
The value of $\gamma$ is a design parameter that can be selected according to the application requirements and networking conditions, e.g., available bandwidth or packet loss probability. A possible objective criterion is to choose $\gamma$ based on the predefined increase percentage of the Lagrangian cost and balancing constrains of reference frame usage. In this work 50\% increase of the Lagrangian cost was defined as adequate whenever the unbalance of the reference frame usage is greater than 3 CTUs (see Figure~\ref{fig:weightfct}).
\subsubsection{BD-PSNR under packet loss free conditions}
@@ -366,7 +365,7 @@ Table~\ref{tab:bd_error_free} shows the Bjontegaard's Delta PSNR (BD-PSNR) valu
\begin{table}[tb]
\centering
\newcommand{\resultswidth}{1.5cm}
\caption{BD-PSNR (dB) in comparison with the reference HEVC for the proposed reference frame selection algorithm (\PROP) in comparison with existing methods, for the case without errors.}
\caption{BD-PSNR (dB) of the proposed reference frame selection algorithm (\PROP) in comparison with existing methods for the case where no packets are lost.}
\TABLESVSPACE
\footnotesize
\label{tab:bd_error_free}