jfmcarreira/doc_phd_thesis
1
0 Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Final version 0

Browse Source
This commit is contained in:
jfmcarreira committed 2018-09-18 00:52:08 +01:00
1 parent 9bdc2ee860
commit e7388f2314
8 files changed
+88 -98
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

043_EndToEndRobustSystem/EndToEndRobustSystem.tex
+36 -42
View File
@@ -75,18 +75,18 @@ The remaining configuration was kept as described in Section~\ref{Section3:Error
\subsubsection{Preliminary simulation results and discussion}
\begin{figure}[!t]
\centering
\input{043_EndToEndRobustSystem/Results/Quality_PLR_Methods.tex}
\vspace{-4mm}
\caption{Average decoded video quality for the bitstreams encoded with the main bitrate (bps).}
\label{Figure6:AverageQualityMain}
\end{figure}
Figures~\ref{Figure6:AverageQualityMain}~and~\ref{Figure6:AverageQualityReduction} show the average decoded video quality (\ac{psnr}) obtained for different \acs{plr}.
Figure~\ref{Figure6:AverageQualityMain} shows the results obtained with the test sequence encoded using a constant target bitrate listed in Table~\ref{Table6:TestMaterialCharacteristics}, referred to as the main bitrate.
Results on Figure~\ref{Figure6:AverageQualityReduction} were obtained by reducing the main bitrate by 40\%. This is used to evaluate the impact of the network bandwidth on the decoded video quality. The results shown in the figures correspond to the same test sequences, in order to allow an accurate and fair comparison.
\begin{figure}[!t]
\centering
\input{043_EndToEndRobustSystem/Results/Quality_PLR_Methods.tex}
\vspace{-4mm}
\caption{Average decoded video quality for the bitstreams encoded with the main bitrate (bps).}
\label{Figure6:AverageQualityMain}
\end{figure}
\begin{figure}[!t]
@@ -221,19 +221,7 @@ Transmission networks with higher loss ratios require a higher bit allocation fo
\end{enumerate}
%To characterise the input video signal the following parameters are used: \ac{si} and \ac{ti}~\cite{ITUTP9102008}.
%These parameters are obtained as described in Chapter~\ref{Chapter:HEVCErrorResilience}, and aim at evaluating the spatial complexity and motion complexity of the video scenes, which have relevant impact on the error robustness of the coded streams.
%As shown, test material with higher \ac{si} or \ac{ti} suffer from higher quality degradation as the \ac{plr} increases, e.g., Book Arrival and Tennis sequences.
These parameters were compared against the optimal stream, selected from \SMEC to \SMEXPELTWEN.
This aims at finding the correlation between the input parameters and the output variable, i.e., target stream.
In this study the streams are characterised by a number between 1 and 4.
Table~\ref{Table6:Correlation} shows the absolute Person Correlation coefficient obtained across all sequences, bitrates and \acs{plr}.
The table shows the results for the parameters previously described, as well as, a new parameter obtained from the average magnitude of the optical flow measured using the method in~\cite{DeepFlow2013} (\textit{OPT\_FLOW}).
Results in Table~\ref{Table6:Correlation} show that the selected parameters are all related with the optimal streams, with the ratio of inter-coded blocks achieving the highest correlation. Moreover, one can also notice that the parameters related with the motion activity, i.e., \ac{ti} and \textit{AVG\_MV}, are also correlated with the output variable.
Since both \textit{AVG\_MV} and \textit{OPT\_FLOW} represent similar information (motion insensitive), only \textit{AVG\_MV} was used in the proposed method, due to its higher Person Correlation coefficient.
\begin{table}[b]
\begin{table}[tb]
\centering
\caption{Pearson Correlation coefficient between the different parameters and the optimal robust coding techniques (optimal stream).}
\label{Table6:Correlation}
@@ -252,9 +240,14 @@ Since both \textit{AVG\_MV} and \textit{OPT\_FLOW} represent similar informatio
\end{tabular}
\end{table}
These parameters were compared against the optimal stream, selected from \SMEC to \SMEXPELTWEN.
This aims at finding the correlation between the input parameters and the output variable, i.e., target stream.
In this study the streams are characterised by a number between 1 and 4.
Table~\ref{Table6:Correlation} shows the absolute Person Correlation coefficient obtained across all sequences, bitrates and \acs{plr}.
The table shows the results for the parameters previously described, as well as, a new parameter obtained from the average magnitude of the optical flow measured using the method in~\cite{DeepFlow2013} (\textit{OPT\_FLOW}).
Results in Table~\ref{Table6:Correlation} show that the selected parameters are all related with the optimal streams, with the ratio of inter-coded blocks achieving the highest correlation. Moreover, one can also notice that the parameters related with the motion activity, i.e., \ac{ti} and \textit{AVG\_MV}, are also correlated with the output variable.
Since both \textit{AVG\_MV} and \textit{OPT\_FLOW} represent similar information (motion insensitive), only \textit{AVG\_MV} was used in the proposed method, due to its higher Person Correlation coefficient.
% Podia se citar este paper
% \cite{IntraRefreshICIP2009}
\subsection{Neural network}
@@ -266,13 +259,6 @@ Mainly, a \ac{nn} consists of three different layers: the input layer which rece
In the past \acs{nn} have been widely used in video processing applications such as, quality evaluation~\cite{VideoQualityNN_TNN2006}, video analysis~\cite{VideoAnalysisNN_2018} and video coding~\cite{RateDistortionNN_VCIP2017,HEVCComplexityNN_TIP2018}.
In the proposed method a feed-forward network is used, which connects every neuron in a given layer to the neurons of the subsequent layer.
Figure~\ref{FIG6:NeuralNetwork} shows the diagram of the \ac{nn} used in the proposed method.
The network contains an input and output layer, as well as, a single hidden layer.
The input layer receives the six inputs, as described in the previous sub-section. These inputs are correlated with the optimisation variable being maximised, i.e., the output video quality.
Subsequently, the network transforms the input data through a series of neurons in the hidden layer.
The output layer comprises four neurons corresponding to each available video stream from \SMEC to \SMEXPELTWEN (different error resilience techniques).
Finally, the output with the highest value is selected and the corresponding stream is used for transmission, since it is the one expected to achieve the highest video quality for a particular scenario.
\begin{figure}[tb]
\centering
%\includegraphics[width=.9\columnwidth]{\ConcealmentAwareDir/Figures/BlockDiagram}
@@ -282,6 +268,13 @@ Finally, the output with the highest value is selected and the corresponding str
\FIGSVSPACE
\end{figure}
Figure~\ref{FIG6:NeuralNetwork} shows the diagram of the \ac{nn} used in the proposed method.
The network contains an input and output layer, as well as, a single hidden layer.
The input layer receives the six inputs, as described in the previous sub-section. These inputs are correlated with the optimisation variable being maximised, i.e., the output video quality.
Subsequently, the network transforms the input data through a series of neurons in the hidden layer.
The output layer comprises four neurons corresponding to each available video stream from \SMEC to \SMEXPELTWEN (different error resilience techniques).
Finally, the output with the highest value is selected and the corresponding stream is used for transmission, since it is the one expected to achieve the highest video quality for a particular scenario.
In the proposed approach a shallow \ac{nn} is used for simplicity and to avoid over-fitting, due to the limited test cases.
By avoiding over-fitting, it is guaranteed that the proposed method would continue to work in future test conditions, not used for the network training.
However, different network topologies with different number of neurons in each layer could also be used to predict between different error resilience techniques, as in the proposed method.
@@ -349,15 +342,6 @@ The results from the validation set, shown in Table~\ref{Table6:ConfusionVal}, r
\end{tabular}
\end{table}
To further evaluate the performance of the \ac{nn}, the confusion matrix of the data set used for testing (test set) is shown in the Table~\ref{Table6:ConfusionTest}.
This table shows the same results as Table~\ref{Table6:ConfusionTraining}, but for different test cases, which were not used in the training phase.
% Therefore, the trained network is not aware of the test cases used for validation.
Results show that the proposed \ac{nn} is able to achieve an accuracy of at least 50\% with a maximum value of 84.9\% when the \SMEXPELTWEN is the target stream.
Although a lower accuracy is obtained using the test set, the overall accuracy is still acceptable, since it allows to select the optimal robust coding technique (i.e., \SMEC to \SMEXPELTWEN) for the majority of the conditions.
In summary, the results presented in this sub-section reveal that the input parameters used in the proposed \ac{nn} are correlated with the decoded video quality, as they can effectively be used to perform data categorisation used in the selection of the optimal error robustness coding technique for the \ac{hevc} standard.
\begin{table}[tb]
\centering
\caption{Confusion matrix of the test set.}
@@ -378,6 +362,15 @@ In summary, the results presented in this sub-section reveal that the input para
\end{tabular}
\end{table}
To further evaluate the performance of the \ac{nn}, the confusion matrix of the data set used for testing (test set) is shown in the Table~\ref{Table6:ConfusionTest}.
This table shows the same results as Table~\ref{Table6:ConfusionTraining}, but for different test cases, which were not used in the training phase.
% Therefore, the trained network is not aware of the test cases used for validation.
Results show that the proposed \ac{nn} is able to achieve an accuracy of at least 50\% with a maximum value of 84.9\% when the \SMEXPELTWEN is the target stream.
Although a lower accuracy is obtained using the test set, the overall accuracy is still acceptable, since it allows to select the optimal robust coding technique (i.e., \SMEC to \SMEXPELTWEN) for the majority of the conditions.
In summary, the results presented in this sub-section reveal that the input parameters used in the proposed \ac{nn} are correlated with the decoded video quality, as they can effectively be used to perform data categorisation used in the selection of the optimal error robustness coding technique for the \ac{hevc} standard.
\section{Performance evaluation}
\label{Section6:ExperimentalResults}
@@ -406,7 +399,7 @@ In case of data loss, the results show that the proposed method is able to match
\begin{table}[tb]
\centering
\caption{Average quality gains ($\Delta$PSNR) in comparison with the reference \ac{hevc} across different bitrates and loss ratios.}
\caption{Average quality gains ($\Delta$PSNR) in comparison with the reference \ac{hevc} across different bitrates and packet loss ratios.}
\label{Table6:AverageGains}
\TABLESVSPACE
\newcommand{\colwidth}{2cm}
@@ -439,13 +432,14 @@ Tennis & +1.18 & +2.11 & +2.43 & +3.28 & +3.28 \\ \hline
Subsequently, a comparison between the different methods is performed, in order to evaluate the quality gains obtained with the proposed method in comparison with each of the different techniques.
%for every test case.
Table~\ref{Table6:AverageGains} shows the average quality gains in comparison with the reference \ac{hevc} for the test set.
These were obtained with the proposed scheme and the four different techniques used in selection method, i.e., methods used in the streams \SMEC to \SMEXPELTWEN. The average quality gains are measured across all test cases, i.e., test set with different bitrates and different packet loss ratios.
These were obtained with the proposed scheme and the four different techniques used in selection method, i.e., methods used in the streams \SMEC to \SMEXPELTWEN.
The average quality gains are measured across all test cases, i.e., test set with different bitrates and different packet loss ratios.
Results in Table~\ref{Table6:AverageGains} show that the robust coded streams \SMEC to \SMEXPELTWEN are able to achieve acceptable average quality gains up to 1.50 dB (see results for \SMEXPELTEN).
However, since the implementation of robust coding techniques decreases the coding efficiency, it also leads to lower quality than the reference \ac{hevc}.
This is noticeable in the results obtained for the Four People sequence in the streams \SMEXP to \SMEXPELTWEN and for the Kristen and Sara sequence in the streams \SMEXPELTEN and \SMEXPELTWEN (see red ellipses).
The overall performance of such methods reveals that they are not suitable for transmission conditions with low \acs{plr}, e.g., lower than 3\% (higher number of conditions have \acs{plr} lower than 3\%).
On the contrary, the proposed method is always able to outperform the reference \ac{hevc} and achieve quality gains of at least 0.28 dB for the Four People sequence.
The overall performance of such methods reveals that they are not the best options for transmission conditions with low \acs{plr} (higher number of conditions have \acs{plr} lower than 3\%).
On the contrary, the proposed method is always capable of outperforming the reference \ac{hevc} and achieve quality gains from 0.28 dB to 3.28 dB.
For the Kimono and Tennis sequences, it is able to achieve the highest quality gains, of up 3.07 dB and 3.28 dB, respectively (see blue ellipses).
Summarising, by adaptively selecting the video stream, and implicitly the robust coding technique, the proposed method is able to achieve a better trade-off between coding efficiency and error robustness, achieving an average quality gain across different bitrates and \acs{plr} of 1.88 dB.