jfmcarreira/doc_phd_thesis
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Final revised version

Browse Source
This commit is contained in:
jfmcarreira
2018-12-27 23:17:31 +00:00
parent 59d007205a
commit 3b63e37cc4
14 changed files with 119 additions and 118 deletions
Download Patch File Download Diff File Expand all files Collapse all files
043_DynamicErrorRobustness/DynamicErrorRobustness.tex
+40 -39
View File
@@ -70,10 +70,9 @@ The remaining configuration was kept as described in Section~\ref{Section3:Error
\subsubsection{Preliminary simulation results and discussion}
\begin{figure}[!t]
\begin{figure}[p]
\centering
\input{043_DynamicErrorRobustness/Results/Quality_PLR_Methods.tex}
\vspace{-4mm}
\caption{Average decoded video quality for the bitstreams encoded with the main bitrate (bps).}
\label{Figure6:AverageQualityMain}
\end{figure}
@@ -83,10 +82,9 @@ Figure~\ref{Figure6:AverageQualityMain} shows the results obtained with the test
Results on Figure~\ref{Figure6:AverageQualityReduction} were obtained by reducing the main bitrate by 40\%. This is used to evaluate the impact of the network bandwidth on the decoded video quality. The results shown in the figures correspond to the same test sequences, in order to allow an accurate and fair comparison.
\begin{figure}[!t]
\begin{figure}[p]
\centering
\input{043_DynamicErrorRobustness/Results/Quality_PLR_Methods_Reduced.tex}
\vspace{-4mm}
\caption{Average decoded video quality using a 40\% reduction of the main bitrate (bps).}
\label{Figure6:AverageQualityReduction}
\end{figure}
@@ -213,49 +211,43 @@ Transmission networks with higher loss ratios require a higher bit allocation fo
\end{enumerate}
\begin{table}[tb]
\centering
\caption{Pearson Correlation coefficient between the different parameters and the optimal robust coding techniques (optimal stream).}
\label{Table6:Correlation}
\TABLESVSPACE
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{p{4cm}x{4cm}}
\toprule
Parameter & Absolute Pearson Correlation \\ \hline
\textit{SI} & 0.3953 \\ \hline
\textit{TI} & 0.6079 \\ \hline
\textit{Inter (percentage)} & 0.6487 \\ \hline
\textit{OPT\_FLOW} & 0.1553 \\ \hline
\textit{AVG\_MV} & 0.4647 \\ \hline
\textit{Bitrate} & 0.3580 \\ \hline
\textit{PLR} & 0.5755 \\ \hline
\end{tabular}
\end{table}
These parameters were compared against the optimal stream, selected from \SMEC to \SMEXPELTWEN.
This aims at finding the correlation between the input parameters and the output variable, i.e., target stream.
In this study the streams are characterised by a number between 1 and 4.
Table~\ref{Table6:Correlation} shows the absolute Person Correlation coefficient \cite{PersonCorrelation_2015} obtained across all sequences, bitrates and \acs{plr}.
The table shows the results for the parameters previously described, as well as, a new parameter obtained from the average magnitude of the optical flow measured using the method in~\cite{DeepFlow2013} (\textit{OPT\_FLOW}).
The table shows the results for the parameters previously described, as well as, a new parameter obtained from the average magnitude of the optical flow measured using the method in~\cite{DeepFlow2013} \mbox{(\textit{OPT\_FLOW})}.
Results in Table~\ref{Table6:Correlation} show that the selected parameters are all related with the optimal streams, with the ratio of inter-coded blocks having the highest correlation. Moreover, one can also notice that the parameters related with the motion activity, i.e., \ac{ti} and \textit{AVG\_MV}, are also correlated with the output variable.
Since both \textit{AVG\_MV} and \textit{OPT\_FLOW} represent similar information (motion insensitive), only \textit{AVG\_MV} was used in the proposed method, due to its higher Person Correlation coefficient.
\begin{table}[tb]
\centering
\caption{Pearson Correlation coefficient between the different parameters and the optimal robust coding techniques (optimal stream).}
\label{Table6:Correlation}
\TABLESVSPACE
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{p{4cm}x{4cm}}
\toprule
Parameter & Absolute Pearson Correlation \\ \hline
\textit{SI} & 0.3953 \\ \hline
\textit{TI} & 0.6079 \\ \hline
\textit{Inter (percentage)} & 0.6487 \\ \hline
\textit{OPT\_FLOW} & 0.1553 \\ \hline
\textit{AVG\_MV} & 0.4647 \\ \hline
\textit{Bitrate} & 0.3580 \\ \hline
\textit{PLR} & 0.5755 \\ \hline
\end{tabular}
\end{table}
\subsection{Neural network}
A \ac{nn} is a computational model based on the structure and functions of biological neural networks.
Mainly, a \ac{nn} consists of three different layers: the input layer which receives data from an external environment, the hidden layer which processes the inputs and an output layer~\cite{NeuralNetworks1994}.
In the past \acs{nn} have been widely used in video processing applications such as, quality evaluation~\cite{VideoQualityNN_TNN2006}, video analysis~\cite{VideoAnalysisNN_2018} and video coding~\cite{RateDistortionNN_VCIP2017,HEVCComplexityNN_TIP2018}.
Neural networks are used in the proposed method since they are universal approximators, which can fit arbitrary complex data that could not be approximated using analytical models~\cite{NeuralNetworks_2016}.
In the past \acs{nn} have been widely used in video processing applications such as, quality evaluation~\cite{VideoQualityNN_TNN2006}, video analysis~\cite{VideoAnalysisNN_2018} and video coding~\cite{RateDistortionNN_VCIP2017,HEVCComplexityNN_TIP2018}.
Neural networks are used in the proposed method since they are universal estimators, which can fit arbitrary complex data not easily approximated by analytical models~\cite{NeuralNetworks_2016}.
Therefore, the complex relationship that describes the dependency of video quality from the input video, compressed stream and network parameters is adequate to be modelled by a \ac{nn}, which can capture multi-dimensional statistical nature of the problem through the training process.
In this work a feed-forward network is used, which connects every neuron in a given layer to the neurons of the subsequent layer.
\begin{figure}[tb]
\centering
\input{043_DynamicErrorRobustness/Figures/NeuralNetwork.tikz}
\caption{Representation of the neural network used in the proposed method.}
\label{FIG6:NeuralNetwork}
\FIGSVSPACE
\end{figure}
Figure~\ref{FIG6:NeuralNetwork} shows the diagram of the \ac{nn} used in the proposed method.
The network contains an input and output layer, as well as, a single hidden layer.
@@ -264,8 +256,16 @@ Subsequently, the network transforms the input data through a series of neurons
The output layer comprises four neurons corresponding to each available video stream from \SMEC to \SMEXPELTWEN (different error resilience techniques).
Finally, the output with the highest value is selected and the corresponding stream is used for transmission, since it is the one expected to achieve the highest video quality for a particular scenario.
In the proposed approach a shallow \ac{nn} is used for simplicity and to avoid over-fitting, due to the limited test cases~\cite{NeuralNetworkDesign_2014}. As shown in Figure~\ref{FIG6:NeuralNetwork}, the \ac{nn} using in this method has three layer: input layer with 6 nodes, an hidden layer with 32 nodes and an output layer with 4 nodes.
By avoiding over-fitting, it is guaranteed that the proposed method would continue to work in future test conditions, not used for the network training.
\begin{figure}[tb]
\centering
\input{043_DynamicErrorRobustness/Figures/NeuralNetwork.tikz}
\caption{Representation of the neural network used in the proposed method.}
\label{FIG6:NeuralNetwork}
\FIGSVSPACE
\end{figure}
In the proposed approach a shallow \ac{nn} is used for simplicity and to avoid over-fitting, due to the limited test cases~\cite{NeuralNetworkDesign_2014}. As shown in Figure~\ref{FIG6:NeuralNetwork}, the \ac{nn} used in this method has three layer: an input layer with 6 nodes, an hidden layer with 32 nodes and an output layer with 4 nodes.
By avoiding over-fitting, it is guaranteed that the proposed method would continue to work in future test conditions, not used for the \ac{nn} training.
The number of nodes in the hidden layer was determined by comparing the performance obtained for different cases using the cross-entropy performance metric~\cite{CrossEntropy_2005}.
%However, different network topologies with different number of neurons in each layer could also be used to predict between different error resilience techniques, as in the proposed method.
@@ -273,10 +273,11 @@ The number of nodes in the hidden layer was determined by comparing the performa
To effectively train and validate the \ac{nn}, firstly the cases corresponding to three video sequence were left out, to be used in the performance evaluation of the proposed method.
The remaining data set was randomly partitioned into three different groups: (i) 65\% of the test cases are used for training, (ii) 15\% for validation and (iii) 20\% for testing the \ac{nn} performance.
\HIGHLIGHT{These ratios were chosen to use a relevant number of samples for testing \ac{nn} performance (i.e., 20\%), and the remaining were used mainly for training, leaving a 15\% samples for validation and stop criteria of the supervised training.
The \ac{nn} supervised training uses 557 test conditions, a scaled conjugate gradient algorithm~\cite{NNScaledConjugateTraining_1993} and stopped when the cross-entropy obtained for the validation samples increases, revealing that the training starts to over-fit the training samples and is not being able to generalise to new data.
Therefore, 129 test conditions were used for \ac{nn} validation.}
Finally, 172 conditions are used to evaluate the accuracy of the proposed method after training with different conditions.
The choice of these percentages follow the commonly used approach where about 70\% to 80\% of the dataset is used for training and validation while the remaining 30\% to 20\% is used for testing~\cite{haykin1994neural}.
% to use a relevant number of samples for testing \ac{nn} performance (i.e., 20\%), and the remaining were used mainly for training, leaving a 15\% samples for validation and stop criteria of the supervised training.
In this work the \ac{nn} supervised training uses 557 test conditions and a scaled conjugate gradient algorithm~\cite{NNScaledConjugateTraining_1993} to calculate the backpropagation.
Then, 129 test conditions were used for \ac{nn} validation and 172 conditions o to evaluate the accuracy of the \ac{nn} after training with different conditions. The 11 test sequences, encoded at different bitrates and subject to a wide range of \acs{plr}, were used to generate the three groups of the dataset. Since these sequences have quiet different characteristics (e.g, motion, texture and resolution) each group is considered representative of any type of video content.
The training was stopped when the cross-entropy obtained for the validation samples stop decreasing, which reveals that the \ac{nn} starts to over-fit the training samples and it is not able to generalise to new data.
Tables~\ref{Table6:ConfusionTraining}, \ref{Table6:ConfusionVal} and \ref{Table6:ConfusionTest} show the confusion matrices obtained for training, validation and test sets, respectively.
In these matrix each column corresponds to the target result, i.e., the cases where the coded streams, \SMEC to \SMEXPELTWEN, would lead to the highest decoded video quality.