Final revised version

021_VideoCoding/VideoCodingReview.tex

@@ -51,7 +51,7 @@ This is the reason why the coding scheme is called hybrid, as it combines quiet
 \begin{figure}[t]
 \centering
 \input{021_VideoCoding/Figures/HybridCodingBlockDiagramPrePost.tikz}
-\caption{Block diagram of a generic video coding system, including the inner blocks of a hybrid encoding and decoding scheme (T and Q corresponds to the transform and quatisation operations, respectively).}
+\caption{Block diagram of a generic video coding system, including the inner blocks of a hybrid encoding and decoding scheme (T: transform; Q: quantisation).}
 \label{Figure2:HybridCodec}
 \end{figure}
 
@@ -242,7 +242,7 @@ Due to the introduction of more flexible random-access points using \ac{cra} pic
 \begin{figure}[tb]
     \centering
     \input{021_VideoCoding/Figures/PictureTypes_Leading_Trailing.tikz}
-    \caption{Leading and trailing pictures associated with the random-access point $I_{1}$ (the letters corresponds to the coding type and the numerical indexes to the coding order).}
+    \caption{Leading and trailing pictures associated with the random-access point $I_{1}$ (the letters and indexes identify the coding type and coding order, respectively).}
     \label{Figure2:LeadingTralingPics}
 \end{figure}
 
@@ -265,7 +265,7 @@ Although these frames allow higher flexibility in terms of layer switching, they
     \centering
     % \includegraphics[width=0.75\columnwidth]{021_VideoCoding/Figures/HEVC/Pictures_TSA}
     \input{021_VideoCoding/Figures/PictureTypes_TSA.tikz}
-    \caption{Temporal layers access pictures example with three layers, $T_0$ to $T_2$ (the letters corresponds to the coding type and the numerical indexes to the coding order).}
+    \caption{Temporal layers access pictures example with three layers, $T_0$ to $T_2$ (the letters and indexes identify the coding type and coding order, respectively).}
     \label{Figure2:TSAExample}
 \end{figure}
 
@@ -335,7 +335,7 @@ Each square corresponds to a \ac{ctu} and the numbers correspond to the time ins
 \begin{figure}[tb]
     \centering
     \input{021_VideoCoding/Figures/WPP.tikz}
-    \caption{Illustration of the wavefront parallel processing (numbers corresponds to the time instant each \ac{ctu} is processed).}
+    \caption{Wavefront parallel processing (the numbers indicate the processing instant of each \ac{ctu}).}
     \label{Figure2:WPP}
 \end{figure}
 
@@ -405,7 +405,7 @@ In order to overcome this issue and to allow all possible modes, \ac{hevc} uses
 \input{021_VideoCoding/Figures/IntraPredictionsDirectional.tikz} \\
 \end{tabular}
 }
-\caption{Directional intra prediction modes in HEVC (left) and an example of the intra prediction mode 32 for a $4\times4$ block.}
+\caption{Directional intra prediction modes in HEVC (left) and example of intra prediction mode 32 (right).}
 \label{Figure2:IntraModesHEVC}
 \end{figure}
 
@@ -508,7 +508,7 @@ For the remaining frames, two types of information are encoded: frame number and
 
 \begin{table}[tb]
     \centering
-    \caption{Reference picture set example.}
+    \caption{Example of a reference picture set.}
     \label{Table2:ReferencePictureListExample}
     \small
     \TABLESVSPACE
@@ -552,7 +552,7 @@ These partitions types are only allowed for \acs{cu} with $16\times16$ pixels or
         \input{021_VideoCoding/Figures/PUSymetric.tikz} & (a)\\
         \input{021_VideoCoding/Figures/PUAsymetric.tikz} & (b)\\
     \end{tabular}
-    \caption{Partition types of an inter-coded CU into PUs; in (b) the size (and position between parentheses) corresponds to the smaller partition~\cite{HEVCBlock2012}.}
+    \caption{Partition types of an inter-coded CU (size $2N\times 2N$) into either (a) symmetric or (b) asymmetric PUs - the size and relative position of the smallest PU are indicated~\cite{HEVCBlock2012}.}
     \label{Figure2:InterPuPartitions}
 \end{figure}
 

022_ErrorResilience/RobustTransmissionReview.tex

@@ -7,14 +7,12 @@
 \vspace*{\fill}
 \section{Introduction}
 
-
+In general, digital communications through networks are prone to transmission errors and packet loss.
-
-In general, digital communications through networks are prone to errors or data loss.
 These errors can be handled in different ways, such as retransmission, error correction, and \ac{ec} \cite{ErrorResilience2000,PixelInterlacingIntraConcealment2011}.
 This chapter reviews the existing techniques for robust video coding that can be applied at the \ac{vcl} to reduce the error propagation and increase the decoded video quality.
 Several techniques have been proposed in the past to deal with the problem of robust video transmission, especially for video coding standards with high temporal dependencies, such as \ac{h264} and more recently \ac{hevc}.
 These techniques are normally focused on the spatio-temporal dependencies in the coded streams, which are the basis of high coding gains but lead to low error robustness~\cite{ROPE2007}.
-In this chapter the error resilience and error concealment schemes are divided in different categories, depending on their functionality and algorithms.
+In this chapter the error resilience and error concealment schemes are classified in different categories, depending on their functionality and algorithms.
 Although each approach is based on different techniques they all aim at striking the best trade-off between coding efficiency and decoded video quality under lossy conditions.
 
 The chapter is organised as follows.
@@ -91,7 +89,7 @@ In both examples, the regions affected by errors are clearly reduced.
 
 
 Common localization techniques deal with the error resilience problem within the scope of the  \ac{rd} optimisation process used in video coding.
-Normally, such optimisation is equivalent to the minimisation of the Lagrangian optimisation cost ($J$) \cite{RateDistortion_SPM1998} given by:
+Normally, such optimisation is equivalent to the minimisation of the Lagrangian cost ($J$) \cite{RateDistortion_SPM1998} given by:
 \begin{equation}\label{EQ2:LagragianCost}
  J = D + \lambda R,
 \end{equation}
@@ -175,7 +173,7 @@ When compared with traditional intra-refresh, bitrate savings of up to 12\% are
 
 A different approach is to introduce refresh points based on the video content.
 In~\cite{IntraRefreshAttentionCM2007} the error resilience of video streams is increased by combining rate control and attention-based models to define \acs{roi}.
-A weighting factor is used in the original Lagrangian optimization~\cite{RateDistortion_SPM1998}, as given by (\ref{EQ2:LagragianCost}), to increase the number of intra-coded blocks in the image regions determined by the attention models.
+A weighting factor is introduced in the conventional Lagrangian cost~\cite{RateDistortion_SPM1998}, to increase the number of intra-coded blocks in the image regions defined by the attention models.
 Figure~\ref{FIG2:IntraRefresh} (b) shows an example with a \ac{roi} marked with a black line, which varies for every frame. In this case, the intra-coded blocks are only introduced inside the marked region, in contrast to what is shown in Figure~\ref{FIG2:IntraRefresh} (a).
 Results show that by using the aforementioned method, less refresh points are required to improve the error robustness of the \ac{roi}. When comparing with \cite{DistortionTrackingH264_ICIP2004}, approximately 4 dB of \ac{psnr} gains are achieved in the \ac{roi} without compromising the overall video quality.
 A more recent approach was proposed using a perceptual noticeable difference model to estimate the noticeable distortion, and then using a visual attention model \cite{SaliencyItti1998} to decide which blocks should be used as intra refresh in the bitstream~\cite{IntraRefreshPerceptual2010}.
@@ -198,7 +196,7 @@ In such approach, the reconstructed reference frames are scaled down by applying
 \end{equation}
 where $\hat f_t^i$  is a reference frame, $\alpha$ is the leaky factor used to control the amount of information from the reconstructed frame present in the new frame $\check f_t^i$, and $C$ is a constant value.
 Leaky prediction has been used in the past for error control, especially in scalable video coding \cite{LeakyPredScalable2002}, where $C$ in~(\ref{EQ2:LeakyPredConstant}) is replaced by a lower layer error-free reconstructed frame. In case of single layer coding, the use of a constant value has significant impact on the coding efficiency leading to higher bitrate, as shown in \cite{LeakyPredConstant1973}. The  $\alpha$ parameter allows to control the effect of the constant value, and thus the coding efficiency.
-By decreasing $\alpha$, the coding efficiency increases, but the exponential decay in error propagation becomes less significant. For the determination of the $\alpha$ parameter, one can use network information such as packet loss probability $p$ leading to $\alpha=(1-p)$. However, as can be seen in previous studies, empirical approaches using exhaustive can also be used to optimise the $\alpha$ value for each sequence~\cite{ROPERefGenJan2010}.
+\HIGHLIGHT{By decreasing $\alpha$, the coding efficiency increases, but the exponential decay in error propagation becomes less significant. To determine $\alpha$, one can use network feedback information, such as packet loss probability $p$, resulting in $\alpha=(1-p)$. Alternatively, empirical approaches using exhaustive search can also be used to find optimal values of $\alpha$ for each sequence~\cite{ROPERefGenJan2010}.}
 
 % MHMCP_TCSVT2006
 
@@ -210,9 +208,7 @@ Alternatively, the two previously encoded frames can be used to generate an inte
 \end{equation}
 Therefore, it is possible to decrease the error propagation using $\check f_t^i$ as reference frame for prediction, if only one of the original encoded frames is lost (i.e., $\hat f_{t}^i$ or $\hat f_{t-1}^i$). This is due to the fact that  $\check f_t^i$ contains information from both $\hat f_{t}^i$ and $\hat f_{t-1}^i$.
 In~(\ref{EQ2:LeakyPredDual}) $\hat f_{t}^i$ and $\hat f_{t-1}^i$ always correspond to the encoded frames and the generated frames ($\check f_t^i $) are not used to interpolate subsequent frames. This corresponds to a finite impulse response filter.
-Although the use of two reference frames for leaky prediction is an effective way to reduce the error propagation with a small penalty in coding efficiency, the use of an interpolated frame in (\ref{EQ2:LeakyPredDual}) still leads to a severe error propagation in case of burst frame loss.
+In the case of burst losses where both $ \hat f_t^i $ and $ \hat f_{t-1}^i $ are missing at the decoder, the interpolated frame ($\check f_t^i$) suffers from severe drift, which significantly decreases its quality, because $ \hat f_t^i $ and $ \hat f_{t-1}^i $ are recovered using error concealment algorithms, i.e., they are not correctly decoded.
-In this case both frames would be affect by packet loss, which would reduce the effectiveness of the method.
-
 
 \begin{figure}[tb]
     \centering
@@ -222,7 +218,7 @@ In this case both frames would be affect by packet loss, which would reduce the
 %  \end{tabular}
 %  \caption{Example of error propagation with a ROI (shaded area indicates error, and the arrows indicate temporal predictions to previous frame) \cite{ROILeakyPred2009}.}
     \input{./022_ErrorResilience/Figures/ROILeakyPrediction.tikz}
-    \caption{Example of error propagation with two different regions; as shown errors can propagate from the background to the ROI.}
+    \caption{Example of error propagation across two different regions: errors from background affect the ROI.}
     \label{fig2:ErrorPropagationROI}
 \end{figure}
 
@@ -252,8 +248,8 @@ In \cite{ResilienceDualFrameBuffer2004}, an optimisation algorithm was devised t
 In the decoder-side, an \ac{ec} method was later used to take advantage of such prediction structure \cite{ConcealmentDualFrameBuffer2005}.
 
 A network feedback mechanism was proposed in \cite{ErrorResilienceRPSFeb2009} to reduce error propagation at end-user decoders, by dynamically selecting the reference frame only among those correctly received.
-Figure~\ref{fig2:ReferencePictureSelection} shows an example of an error resilience transcoding scheme, where the original temporal predictions (represented by the solid arrows) are transcoded to use only correctly received frames as reference, i.e., prediction represented by the dashed arrow. This is done based on network feedback.
+Figure~\ref{fig2:ReferencePictureSelection} shows an example of an error resilience transcoding scheme, where the original temporal predictions (represented by the solid arrows) are transcoded to only use correctly received frames as reference, i.e., prediction represented by the dashed arrow. This is done based on network feedback.
-\HIGHLIGHT{As this technique requires transcoding operations, the complexity increase with the \ac{gop} length, as reported in~\cite{ErrorResilienceRPSFeb2009}. Typically, the \ac{gop} length is defined by taking into consideration the video application (i.e., synchronisation, random-access or maximum refresh period allowed) and not by the robust coding scheme.}
+The length of the \ac{gop} has several implications on the performance of this transcoding scheme. For instance the complexity increases with the \ac{gop} length, because the number of frames that must be decoded also increases~\cite{ErrorResilienceRPSFeb2009}. An absolute optimal \ac{gop} length was not found in~\cite{ErrorResilienceRPSFeb2009}, since it mostly depends on the application requirements (i.e., random-access, maximum refresh period, etc).
 A more advanced scheme may be considered, where only the regions affected by mismatch predictions are processed in the compressed domain, and subsequently transcoded to limit the dependencies from lost frames.
 To achieve low complexity, \ac{mv} concatenation can be used, in order to obtain new inter-predictions for error-free frames \cite{MVConcatenationAug2011}.
 
@@ -290,8 +286,8 @@ In compressed video streams different symbols have different relevance in terms
 Therefore, there is an unequal contribution of different syntax elements to the decoding distortion and error propagation, depending on the symbols that are lost in each case.
 Data partition techniques have been developed to provide a mechanism to cope with the unequal importance of coded information by grouping different parts of a compressed bitstream, according to their importance to the decoded quality.
 Thus, some of these groups can be protected more efficiently than others using \ac{uep} schemes.
-These are normally associated with \ac{fec} algorithms, which are typically applied at the channel level which is not aware of the video content, leading to a sub-optimal error protection~\cite{UEPMotion_TM2006}.
+\HIGHLIGHT{These are normally associated with \ac{fec} mechanisms, which are typically applied at the channel level without taking into consideration that not all video data is equally important. This leads to a sub-optimal error protection~\cite{UEPMotion_TM2006}.
-Using \ac{uep}, a source-channel optimisation is applied, whereby multi-levels of \ac{fec} are used to achieve different levels of error protection according to the importance of the coded information.
+In general, when using \ac{uep}, source-channel optimisation is applied, whereby multi-levels of \ac{fec} are used to achieve different levels of error protection according to the importance of coded video data.}
 This is presented and discussed with more detail in Sub-section~\ref{SEC22:UEP}
 
 In single channel transmission, the most important data or syntax elements (e.g., headers) can be protected with stronger channel codes than the remaining ones.
@@ -327,7 +323,7 @@ In the \ac{fmo} type 0 each slice group is defined by a maximum number of \acs{m
 In the \ac{fmo} type 2, slice groups are defined by rectangular shapes bounded by the top-left and bottom-right corners.
 Although the slice groups are allowed to overlap each other, each \ac{mb} may only belong to one slice group.
 Similar to \ac{fmo} type 0, in the types 3, 4 and 5 in Figure~\ref{fig2:FMOTypes}, slice groups are defined by a maximum number of \acs{mb}.
-However, contrary of \ac{fmo} type 0, the configuration is allowed to change every frame for these types
+However, contrary to \ac{fmo} type 0, the slice groups are allowed to change every frame for types 3, 4 and 5.
 In order to keep the signalling overhead affordable, only two slice groups are allowed in these types.
 The \ac{fmo} type 6 is the most generic case, where the slice groups are explicitly defined using high level packets (i.e., signalling \ac{nal} units), allowing more flexibility at the cost of higher signalling overhead.
 
@@ -486,7 +482,8 @@ A typical approach to recover missing regions in an image is to estimate the los
 Figure~\ref{FIG3:pixel_interpolation} shows the estimation process of a lost pixel ($lp$), within the lost region ($R_L$), using the four closest pixels ($p_i$) as reference, and the distance between the lost pixel and the reference ones ($d_i$). The pixel value ($lp$) is recovered using a weighted average of $n$ neighbouring pixels, defined as follows:
 \begin{equation}
 \label{eq:pixel_interpolation}
-lp = \left. \sum_{i=1}^n{ \frac{p_i}{d_i} } \middle/ \sum_{i=1}^n{ \frac{1}{d_i} } \right. ,
+%lp = \left. \sum_{i=1}^n{ \frac{p_i}{d_i} } \middle/ \sum_{i=1}^n{ \frac{1}{d_i} } \right. ,
+lp = \frac{ \sum_{i=1}^n{ \frac{p_i}{d_i} } }{ \sum_{i=1}^n{ \frac{1}{d_i} } } ,
 \end{equation}
 %\begin{equation}
 %\label{eq:pixel_interpolation}
@@ -524,7 +521,7 @@ The \ac{mv} differences between the vectors of successive frame pairs are recurs
 This approach has shown to improve the performance of the \ac{mc} method, with gains up to $0.47\ dB$ (\ac{psnr}) for a \ac{plr} of $20\%$.
 
 
-\subsubsection{Boundary match algorithms}
+\subsubsection{Boundary matching algorithms}
 
 
 \Gls{bma} are used to recover a missing region based on the available neighbouring regions.
@@ -538,7 +535,7 @@ Then, the optimal reconstruction is selected according to the sum of differences
     \centering
     %\includegraphics[width=0.45\columnwidth]{./022_ErrorResilience/Figures/BMAAlgorithm}
     \input{./022_ErrorResilience/Figures/BMAAlgorithm}
-    \caption{Illustration of the lost block and the neighbouring information used in boundary match algorithm.}
+    \caption{Illustration of the lost block and the neighbouring information used in boundary matching algorithm.}
     \label{FIG3:BMAPixelInformation}
 \end{figure}
 
@@ -574,14 +571,14 @@ An extension to the previous approach was considered in \cite{PelBasedMVE_TCE200
 \begin{figure}[t]
     \centering
     \includegraphics[width=0.9\columnwidth]{./022_ErrorResilience/Figures/MVE}
-    \caption{Motion vector extrapolation with the corresponding extrapolated macroblock, based on the method presented in \cite{BlockBasedMVE2002}.}
+    \caption{Motion vector extrapolation with the corresponding extrapolated block, based on the method presented in \cite{BlockBasedMVE2002}.}
     \label{FIG3:MVEExtrapolation}
 \end{figure}
 
 In \cite{2DConcHybridMotion2010} a \ac{hmve} algorithm was proposed, to improve the accuracy of the \acs{mv} of the \ac{pmve} method, by using the extrapolated vectors at the pixel and block level. Moreover, the \acs{mv} that were wrongly extrapolated are discarded in order to obtain a more accurate MV.
 In this \ac{ec} method, firstly a pixel-based motion extrapolation is performed and the pixels are organised into three groups.
 Then, through block-based \ac{mve} two new \acs{mv} are determined: the first one corresponds to the \ac{mv} of the most overlapped block with the current one, and second one is obtained by the weighted average value of the \acs{mv} corresponding to different overlapped blocks.
-To discard the wrongly extrapolated \acs{mv} and finding the more  accurate ones from the MV set described above, further refinement is applied based on the distance between each pair of \acs{mv}.
+\HIGHLIGHT{To discard the wrongly extrapolated \acs{mv} and finding the most accurate ones, further refinement is applied based on the distance between each pair of \acs{mv}.}
 This algorithm is able to outperform the \ac{pmve} for different packet loss probabilities, as shown by the results in \cite{2DConcHybridMotion2010}.
 The method was used to recover both intra and inter frames in video sequences compressed with the \ac{h264}, achieving \ac{psnr} gains above $1.09$ dB in the erroneous frames.
 

031_ErrorResilienceEvaluation/ErrorResilienceEvaluation.tex

@@ -44,7 +44,7 @@ There are some studies presented in the past regarding the error robustness of t
 A subjective evaluation study is presented in~\cite{SubjectiveEvalSep2013,QoEHEVCMay2014} revealing that packet loss ratios higher than $3\%$ have significant effect on the perceived video quality.
 The vulnerability of \ac{mv} prediction is investigated and its impact on the error robustness is evaluated in \cite{DynamicTMVP2011}.
 Moreover, in the same study, a method to introduce \ac{mv} refresh points is proposed, achieving average \ac{psnr} gains of approximately 3~dB.
-Summarising, the decrease in error robustness is mostly due to the strong data dependencies imposed by the highly complex prediction modes of \ac{hevc}. Since these are selected according to a rate distortion optimisation criterion that assumes \ac{plf} transmission, the decoder faces  different problems whenever errors or data loss occur in the transmission network \cite{ROPE2007}.
+Summarising, the decrease in error robustness is mostly due to the strong data dependencies imposed by the highly complex prediction modes of \ac{hevc}. Since these are selected according to a rate distortion optimisation criterion that assumes packet loss-free transmission, the decoder faces  different problems whenever errors or data loss occur in the transmission network \cite{ROPE2007}.
 
 The following sub-sections present an error robustness evaluation of \ac{hevc} focused on the impact of the novel coding techniques introduced in the standard, extending the generic studies presented in previous works. This study provides relevant insights for research directions and motivation to investigate solutions for the error robustness problem in \ac{hevc}.
 The following case-studies are covered in this experimental evaluation:
@@ -63,7 +63,7 @@ The experimental setup used in this study to evaluate the error robustness of th
 Table~\ref{TableTestSequences} presents a summary of the main characteristics of the test sequences. Appendix~\ref{Appen:Sequences} shows a single frame of each test sequence.
 Considering a test sequence with $N$ frames $f(t)$ the pixel values (Luma component) of the frame at instant $t$, the spatial and temporal complexity of the video sequences was measured by using the following metrics \cite{ITUTP9102008}:
 \begin{itemize}
- 	\item {\ac{si}} based on Sobel filter. Each video frame is firstly filtered, then the standard deviation ($std$) of the pixel values is computed. Finally, the maximum value over time ($\maxop_{t\in [1..N]}$), is chosen to represent the \ac{si} metric. This process can be represented by the following equation:
+ 	\item {\ac{si}} based on Sobel filter. Each video frame is firstly filtered, then the standard deviation ($std$) of the pixel values is computed. Finally, the maximum value over time ($\maxop_{t\in [1..N]}$\{ \}), is chosen to represent the \ac{si} metric. This process can be represented by the following equation:
 	\begin{equation}
 %		SI = max_{time}\Big\{ std \big( Sobel\left(f_t\right) \big) \Big\}.
   		SI = \maxop_{t\in [1..N]} \Big\{ std \big( sobel\left(f\left(t\right)\right) \big) \Big\}.
@@ -102,12 +102,12 @@ Moreover, the ratio of inter-coded \acs{cu} of each coded sequence using the def
 	\end{tabular}
 \end{table}
 
-To simulate the lossy transmission environment, each frame is divided into several slices and each slice encoded and packetized into one \ac{nal} unit, which are then transmitted as the payload of independent packets.
+To simulate the lossy transmission environment, each frame was divided into several slices and each slice encoded and packetized into one \ac{nal} unit, which were then transmitted as the payload of independent packets.
 In these simulations a whole packet is lost whenever an error occurs, originating a lost slice.
 \HIGHLIGHT{Random packet loss was simulated using a two-state Markov model, derived from the classical Gilbert-Elliot \cite{GilbertModel_1960, ElliotModel_1963}, which can be used to describe error patterns in real time video applications~\cite{GilbertElliotModelPlr_2008}. In this study a simplified version of the Gilbert-Elliot model was used by only defining the loss probability without control over burst loss events.
 The application of this model resulted in loss patterns with approximately the same average burst loss as the ones provided in \cite{JCTVCLossSim_2012}.
-For each test condition, 50 runs are tested and the average quality is measured. 
+For each test condition, 50 runs were tested and the average quality was measured. 
-Four different \acs{plr} with the characteristics presented in Table~\ref{TAB3:LossPatterns} are used.}
+Four different \acs{plr} with the characteristics presented in Table~\ref{TAB3:LossPatterns} were used.}
 The remaining configurations have been kept at their default values, as given in the common test conditions~\cite{HEVCCommonTests2012}.
 
 
@@ -125,14 +125,14 @@ In the first set of experiments the impact of the maximum \ac{cu} size in the de
 \end{table}
 
 
-Table~\ref{Table3:Quality_BlockSize} shows the average \ac{psnr} (dB) obtained for the three test conditions and for three different packet loss ratios. Results for the \ac{plf} scenario shows a consistent quality increase as the maximum \ac{cu} size increases,
+Table~\ref{Table3:Quality_BlockSize} shows the average \ac{psnr} (dB) obtained for the three test conditions and for three different packet loss ratios. Results for the \NOLOSS case shows a consistent quality increase as the maximum \ac{cu} size increases,
 % up to a maximum of 1.41~dB for the Tennis sequence
 when comparing the maximum \ac{cu} size of $64\times64$ with $16\times16$.
 This confirms the principle used by the \ac{hevc} standard, whereby larger coding units lead to higher coding efficiency.
 For the Tennis sequence, which has the highest \ac{ti} parameter (see Table~\ref{TableTestSequences}), a maximum of 1.41~dB is achieved.
 Moreover, it is also noticeable that sequences with higher amount of inter-coded blocks take less advantage of the larger \acs{cu}.
-This is noticeable in the results of the BQSquare sequence (i.e., highest ratio of inter-coded blocks and lowest quality gain in the \ac{plf} case).
+This is noticeable in the results of the BQSquare sequence (i.e., highest ratio of inter-coded blocks and lowest quality gain in the \NOLOSS case).
-Overall, comparing the results for the \ac{plf} and data loss cases, it is noticeable that similar quality gains are obtained, mostly due to the superior coding efficiency of larger \acs{cu}.
+Overall, comparing the results with and without packet loss, it is noticeable that similar quality gains are obtained, mostly due to the superior coding efficiency of larger \acs{cu}.
 In most cases the quality gains slightly decrease for increasing \acs{plr}.
 Therefore, one may conclude that the error robustness of \ac{hevc} streams is not affected by limiting the maximum \ac{cu} size.
 
@@ -177,7 +177,7 @@ Figure~\ref{Figure3:QualityvsIntra} shows the average decoded video quality (\ac
 Results in the figure indicate that as the distance between intra frames increases, the decoded video quality significantly decreases.
 The quality decreasing is especially noticeable for intra periods higher than 32 frames, indicating that temporal distances greater than approximately one second are not suitable for video streaming under error-prone conditions.
 When a high temporal distance between intra frames is used, the quality decreasing is more noticeable in the People On Street and BQSquare sequences, which have higher spatial complexity (see values for \ac{si}), leading to a reduction of 13 dB and 14 dB, respectively.
-As spatial complexity increases, the prediction residue energy also increases, which leads to more erroneous information in case of data loss, reducing the video quality under errors. \HIGHLIGHT{Finally, based on the confidence interval one may conclude that for different error patterns the average \ac{psnr} of all frames does not significantly changes (maximum interval of 1 dB approximately).}
+As spatial complexity increases, the prediction residue energy also increases, which leads to more erroneous information in case of data loss, reducing the video quality under errors. \HIGHLIGHT{Finally, based on the confidence interval one may conclude that for different error patterns the average \ac{psnr} of all frames does not significantly change. A maximum variation of 1 dB, approximately, is observed in Figure~\ref{Figure3:QualityvsIntra}.}
 
 One should also note from the results of Figure~\ref{Figure3:QualityvsIntra} that, using a small distance between intra frames is not always more efficient.
 Since intra frames require higher amount of bits, the overall coding efficiency is reduced, resulting in lower overall quality in case of data loss, when comparing with other cases.
@@ -197,7 +197,7 @@ A comparison of the quality obtained for different partition modes is also carri
 \begin{figure}[!t]
     \centering
     \input{./031_ErrorResilienceEvaluation/QualityEvaluation/PSNRvsSlices.tex}
-    \caption{Decoded video quality for different slice partitioning modes.}
+    \caption{Decoded video quality with a 95\% confidence interval for different slice partitioning modes.}
     \label{Figure3:SlicePartitioning}
 \end{figure}
 
@@ -211,7 +211,7 @@ Although the use of smaller slices increases the overhead required for slice hea
 This is due to the lack of refresh points within a frame, resulting in errors that affect larger regions of the frame, which are propagated to the subsequent inter predicted frames.
 
 The results for of \textit{SS} (slice partitioning disabled) reveals that this approach also leads to lower error robustness when comparing with those cases where the frames are divided into multiple slices. 
-\HIGHLIGHT{Moreover, in this case the 95\% confidence interval increases, due to the fact that each packet corresponds to a whole frame, which vary in size. Therefore, the amount of lost data varies depending on the affected frame, leading to a different impact of the recovered quality.}
+\HIGHLIGHT{In this case the 95\% confidence interval is greater than the others shown in Figure~\ref{Figure3:SlicePartitioning}. This is because the packet size is variable, which implies higher variability in the amount of lost information.}
 Although higher coding efficiency is achieved in this mode, the lower error robustness leads to an overall poor performance under frame or slice loss conditions.
 It is worthwhile to notice that the results for Kendo sequence do not follow the main trend.
 This can be explained by the low bitrate required to compress the video signal, which is the lowest when compared with the other test sequences (see Table~\ref{TableTestSequences}).
@@ -280,7 +280,7 @@ Moreover, the slice partitioning mode used only 1200 bytes per slice, which intr
 
 The second set of experiments aimed to find the influence of losing temporal motion predictors on the error resilience of \ac{hevc}.
 In order to perform this test, a single frame loss in enforced, and the error propagation is evaluated for two configurations: (i) with the temporal \ac{mv} candidate enabled and (ii) disabled.
-Figure~\ref{Figure3:PSNRvsFrameErrorFreeConc} illustrates the error propagation, when a single frame is lost. For this particular experiment an intra-refreshing period of 32 frames was used to evaluate the impact on long error propagation periods. This is commonly used in quality evaluation of erroneous video decoding and follows the reference conditions~\cite{HEVCCommonTests2012}.
+Figure~\ref{Figure3:PSNRvsFrameErrorFreeConc} illustrates the error propagation, when a single frame is lost. \HIGHLIGHT{For this particular experiment an intra-refreshing period of 32 frames was used to evaluate the impact on long error propagation periods. This is commonly used in quality evaluation of erroneous video decoding and follows the reference conditions~\cite{HEVCCommonTests2012}.}
 The results show that when the \ac{tmvp} mode is enabled, the loss of Frame \#6 leads to significant reduction in the reconstruction quality of the subsequent frames.
 On the contrary, when the temporal \ac{mv} candidate is disabled, the motion information on subsequent frames is not affected by the loss of a single frame and significantly higher quality is obtained.
 This is the effect of breaking the temporal dependencies between \acs{mv}, which prevents erroneous \acs{mv} to propagate throughout the set of dependent frames.
@@ -295,7 +295,7 @@ This is the effect of breaking the temporal dependencies between \acs{mv}, which
 
 In order to further evaluate the impact of the temporal \ac{mv} dependencies on the error propagation, the \ac{hevc} streams were subject to random packet losses.
 Table~\ref{Table3:AvgPSNRWithoutTMVP} presents the average quality results, measured using the \ac{psnr}.
-As expected, the results indicate that in the \ac{plf} case higher quality is achieved by enabling the \ac{tmvp} candidate, since higher compression efficiency is obtained  because more \ac{mv} predictors are used.
+As expected, the results indicate that in the those cases without packet loss (\NOLOSS) higher quality is achieved by enabling the \ac{tmvp} candidate, since higher compression efficiency is obtained  because more \ac{mv} predictors are used.
 This is true for both test conditions, \emph{i.e.}, \ac{ld} and \ac{ra}.
 However, in the presence of errors, the video quality  significantly decreases due to low error robustness when temporal \ac{mv} candidates are used.
 Alternatively, average quality up to 5.17 dB of \ac{psnr} is achieved when the temporal \ac{mv} predictors are disabled in the \ac{ld} configuration.
@@ -348,7 +348,7 @@ In the proposed approach, a temporal \ac{mv} candidate is marked as unsuitable,
     \centering
     % \includegraphics[width=0.6\columnwidth]{./031_ErrorResilienceEvaluation/DynamicTmvp/DynamicTMVP}
     \input{./031_ErrorResilienceEvaluation/DynamicTmvp/DynamicTMVP.tikz}
-    \caption{Proposed approach to constrain the temporal \ac{mv} dependencies (arrow point to the MV predictor).}
+    \caption{Proposed approach to constrain the temporal \ac{mv} dependencies (the arrows point to the MV predictors).}
     \label{ProposedDynamicMethod}
     \FIGSVSPACE
 \end{figure}
@@ -416,7 +416,7 @@ Therefore, since the proposed method achieves the lowest  \ac{psnr} standard dev
 Table~\ref{Table3:AverageResultsProposed} shows the Bjontegaard's average \ac{psnr} increase \cite{BJResults2001}.
 The results are obtained taking the standard \ac{hevc} encoder as reference.
 The results presented in Table~\ref{Table3:AverageResultsProposed} show that the proposed method presents practically the same rate-distortion performance when compared to the \TIDR method.
-In general, the average quality, in the \ac{plf} case, only decreases 0.028 dB in comparison to the \TIDR method and 0.034 dB in comparison to the reference encoder.
+In general, the average quality, in the \NOLOSS case, only decreases 0.028 dB in comparison to the \TIDR method and 0.034 dB in comparison to the reference encoder.
 However, the proposed method widely outperforms the reference \TIDR technique in terms of error resiliency performance, achieving in average higher overall quality under error-prone conditions.
 The effectiveness of the proposed method increases at higher packet loss rates (\emph{e.g.},~$5\%$), achieving an average gain of 0.64 dB ($1.34 - 0.70$), and a maximum gain of 1 dB ($1.44-0.43$) for the Book Arrival sequence , when compared to the \TIDR technique. Higher gains are achieved when compared with the reference \ac{hevc} encoder, as shown in Table~\ref{Table3:AverageResultsProposed}.
 

031_ErrorResilienceEvaluation/QualityEvaluation/TableMVPs.tex

@@ -125,7 +125,7 @@
 %    \resizebox{\columnwidth}{!}{
 \newcommand{\nlines}{2}
 \newcommand{\colwidth}{1.5cm}
-\begin{tabular}{p{2.8cm}|x{2.3cm}|x{\colwidth}|x{\colwidth}|x{\colwidth}|x{\colwidth}}
+\begin{tabular}{p{2.8cm}|x{2.2cm}|x{1.8cm}|x{\colwidth}|x{\colwidth}|x{\colwidth}}
     \toprule
     \multirow{2}{*}{\centering\textbf{Sequence}} & \multirow{2}{*}{\vbox{\centering\textbf{TMVP}}} & \multirow{2}{*}{\vbox{\centering\textbf{\NOLOSS}}} & \multicolumn{3}{c}{\textbf{\TABLEPLRLABEL}} \\
     & & & \textbf{1\%} & \textbf{5\%} & \textbf{10\%} \\

041_ImprovedErrorResilience/Figures/RefPicSelection.tikz

@@ -1,4 +1,4 @@
-\newcommand{\scale}{1}
+\newcommand{\scale}{1.2}
 \newcommand{\xfshift}{4cm}
 \begin{tikzpicture}
 \begin{scope}[scale=\scale]

041_ImprovedErrorResilience/ImprovedErrorResilience.tex

@@ -104,7 +104,7 @@ However, such solution would not take into consideration coding efficiency, sinc
 \begin{figure}[tb]
 	\centering
 	\input{./041_ImprovedErrorResilience/Figures/RefPicSelection.tikz}
-	\caption{Example of the reference picture selection applied to the frame $C$ based on the checkerboard at \ac{ctu} level (\acs{pu} are represented by small partitions).}
+	\caption{Example of the reference picture selection applied to the current frame based on the checkerboard at \ac{ctu} level (\acs{pu} are represented by small partitions).}
 	\label{fig:checkerboardRefSelection}
 \end{figure}
 
@@ -114,7 +114,7 @@ This leads to the overall goal of finding the optimum R-D coding mode for each \
 \sigma_C=std\Big(\mathbf{N}_C(r_i) \Big), i=1, 2,\cdots S,
 \label{eq:StdX}
 \end{equation}
-where  $\sigma_C$  measures the deviation from the checkerboard solution, which corresponds to $\sigma_C = 0$, since all possible reference frames $r_i\in F$ are uniformly used throughout the \acs{pu} of $C$.
+\HIGHLIGHT{where  $\sigma_C$  measures the deviation from the checkerboard solution, which corresponds to $\sigma_C = 0$, since all possible reference frames $r_i\in F$ are uniformly used throughout the \acs{pu} of $C$.}
 
 To attain the goal defined above, the best coding mode for each \ac{pu} is chosen through a Lagrangian \ac{rd} optimisation that tends to minimise $\sigma_C$.
 This is done by penalising (or benefiting) the \ac{rd} cost of those coding modes which intend to use reference frames that have already been used more (or less) frequently than their average use in previous \acs{pu} of the current frame.
@@ -125,7 +125,7 @@ Such approach allows to select the reference frames, by independently processing
 J(\phi) &= \Big( D(\phi) + \lambda \times R(\phi) \Big) \times e^{W(r_{\phi})},
 \label{eq:Lagrangian}
 \end{align}
-where $D(\phi)$ and $R(\phi)$ are the distortion and rate, respectively, associated with the coding mode $\phi$ and $J(\phi)$ is the trade-off between distortion and rate.
+\HIGHLIGHT{where $D(\phi)$ and $R(\phi)$ are the distortion and rate, respectively, associated with the coding mode $\phi$. $J(\phi)$ is Lagrangian cost that measures the the trade-off between distortion and rate.}
 The exponential weighting factor $e^{W(r_{\phi})}$ is used to penalise (or benefit) the Lagrangian cost of a given coding mode $\phi$ associated to reference frame $r_\phi$.
 $W(r_{\phi})$ is obtained according to the global and local deviation between the number of times that $r_\phi$ was used and the average use of all $r_i \in F$ across the \acs{pu} of current frame $C$, as follows,
 \begin{equation}\label{eq:KFactor}
@@ -170,7 +170,7 @@ For instance, for $\gamma=5$, when a reference frame $r_\phi$ is used to predict
             & \footnotesize $r_{1}$: $61.8\%$ | $r_{2}$: $25.7\%$ & \footnotesize $r_{1}$: $42.2\%$ | $r_{2}$: $39.3\%$ & \footnotesize $r_{1}$: $45.22\%$ | $r_{2}$: $43.2\%$ \\
         \end{tabular}
     }
-    \caption{Example of the partitions and references pictures selected using the different encoding methods (the dark gray squares indicate that the \ac{mv} is using the reference $r_{1}$, light~gray squares corresponds to the reference $r_{2}$ and unfilled regions are intra-coded blocks).}
+    \caption{Example of the partitions and references pictures selected using the different encoding methods (the dark gray squares indicate that the \ac{mv} is using the reference $r_{1}$, light~gray squares correspond to the reference $r_{2}$ and unfilled regions are intra-coded blocks).}
     \label{fig:RefPicSelectionExample}
     \FIGSVSPACE
 \end{figure*}
@@ -232,10 +232,9 @@ where $R_U$ is the amount of bits required to encode the $n$ \acs{mv} of $U$ and
 A_{D}(i) = \sum_{j=1}^{DV_i} ( w_j\times h_j ),\hspace{2mm} i = 1, 2 \cdots s,
 \end{equation}
 where $DV_i$ is the number of \acs{mv} dependent from $MV_i$, and $w_j$ and $h_j$ are the width and height of the corresponding \ac{pu}.
-A weighting factor is used to control the amount of bitrate overhead ($R_U$), by increasing the cost of the number of encoded bits based on the duration of the error propagation.
+\HIGHLIGHT{A weighting factor is used to control the amount of bitrate overhead ($R_U$), by increasing the cost of the number of encoded bits based on the duration of the error propagation.
-This duration is given by the ratio between the distance from the current frame ($t_C$) to the previous refresh frame ($t_I$) and the \ac{idr} period ($T_{I}$).
+This duration ($t_C-t_I / T_{I}$) is given by the ratio between the distance from the current frame ($t_C$) to the previous refresh frame ($t_I$) and the \ac{idr} period ($T_{I}$) and varies in the range $\left[0;1 \right]$, achieving the lowest value for  intra-coded frames and the maximum for the frame located immediately before an intra-coded one.
-This weight varies in the range $\left[0;1 \right]$, achieving the lowest value for  intra-coded frames and the maximum  for the frame located immediately before an intra-coded one.
+The parameter $\alpha$ is used as a global control for redundancy of the entire stream, where higher values of $\alpha$ correspond to less redundant \acs{mv}.}
-Moreover, in the proposed approach, the parameter $\alpha$ is used to globally control the redundancy for the entire stream, resulting in higher values of $\alpha$, less redundant \acs{mv} are selected.
 As pointed in Section~\ref{Section4:Architecture}, this streaming stage operates on coded data and may be implemented at any point of the network, to increase robustness without fully decoding the stream. Note that the parsing operation represented in Figure~\ref{fig:proposedMethod} does not include decoding of the video frames. Furthermore, only one frame delay is introduced because the \ac{mv} selection process is performed on a frame-by-frame basis.
 
 To maintain compatibility with the \ac{hevc} standard, the redundant motion information is transmitted using the \ac{sei} \ac{nal} units \cite{HEVCHighLevelReferenceDez2012}. Thus, this information can be multiplexed into the coded bitstream without affecting the compressed video.
@@ -285,7 +284,7 @@ This value was chosen  to allow increasing the Lagrangian cost by about $50\%$,
             PSNR = 31.93 dB & PSNR = 33.83 dB & PSNR = 24.12 dB & PSNR = 31.93 dB \\
         \end{tabular}
     }
-    \caption{Zoom of recovered Frame \#10 after a loss event at Frame \#9 for Basketball Drill and Race Horses sequence.}
+    \caption{Zoom of recovered Frame \#10 after a loss event at Frame \#9 for Basketball Drill and Race Horses sequences.}
     \label{fig:RecoveredExample}
 \end{figure}
 
@@ -303,9 +302,8 @@ Summarising, the reconstructed frames illustrated in Figure~\ref{fig:RecoveredEx
 \begin{figure}[t]
     \renewcommand{\FIGSIZE}{0.8\columnwidth}
     \centering
-%       \includegraphics[width=\FIGSIZE]{./041_ImprovedErrorResilience/Results/PSNRversusFrameKendo}
     \input{041_ImprovedErrorResilience/Results/PSNRversusFrameKendo.tex}
-    \caption{Error propagation when errors affect Frame \#7 in Kendo sequence with an \ac{idr} period of 32 frames for the proposed reference frame selection algorithm in comparison with existing methods.}
+    \caption{Error propagation when errors affect Frame \#7 in Kendo sequence (\ac{idr} period of 32 frames) for the proposed reference frame selection algorithm in comparison with existing methods.}
     \label{fig:PSNRvsFrameForRefPicSel}
 \end{figure}
 
@@ -342,20 +340,21 @@ Taking into account the above results and the spatio-temporal information shown
     %\includegraphics[width=\FIGSIZE,page=1]{./041_ImprovedErrorResilience/Results/PSNRversusGama}
     %\includegraphics[width=\FIGSIZE,page=2]{./041_ImprovedErrorResilience/Results/PSNRversusGama}
     \input{./041_ImprovedErrorResilience/Results/PSNRversusGama.tex}
-    \caption{Influence of the $\gamma$ parameter in the average quality gains, comparing the proposed method with \REF,  for different \acs{plr}.}
+    \caption{Influence of the $\gamma$ parameter in the average quality gains: comparison between the proposed method and \REF, for different \acs{plr}.}
     \label{fig:QualityVsGamma}
 \end{figure}
 
 The influence of $\gamma$ parameter used in (\ref{eq:KFactor}) on the video quality was also evaluated, in order to check how the cost constrains affect the performance of the proposed method.
 Figure~\ref{fig:QualityVsGamma} shows the average quality gains of the proposed method in comparison to the reference \ac{hevc} (\REF) across all test sequences, for different values of $\gamma$.
-For the \ac{plf} case, the results in this figure show that increasing $\gamma$ leads to a decrease in the coding efficiency, especially in the \ac{ld} configuration.
+When no packets are lost, the results in this figure show that increasing $\gamma$ leads to a decrease in the coding efficiency, especially in the \ac{ld} configuration.
 The reason is that higher values of $\gamma$ lead to higher values of $W(r_{\phi})$, therefore the reference picture selection cost is severely constrained by the exponential weights.
 Since $T_{ID}$ is constant for all frames in the \ac{ld}, higher weights are used for all frames ($T_{ID}$ is used to reduce the weights in case of hierarchical coding) and the encoder is not able to select the best reference frame in the \ac{rd} sense.
 In the case where packet loss occurs, higher values of $\gamma$ lead to improved error resilience.
 Since higher weights are used in the \ac{rd} cost, the encoder is constrained to distribute the use of the reference frames of different \acs{pu}, reducing the error propagation.
 For very high values (e.g., $\gamma=1000$) the encoder is forced to  uniformly select the reference frames, resulting in a fixed checkerboard pattern.
-\HIGHLIGHT{Overall, the $\gamma$ parameter is able to control the reference frame selection constrains,
+Overall, the $\gamma$ parameter is able to control the reference frame selection constrains,
-thus achieving better trade-off between improved error resilience and loss of coding efficiency. The value of this parameter should be selected accordingly with the requirements and condition of the streaming application, e.g., network bandwidth or packet loss ratio.}
+which allows to achieve a better trade-off between improved error resilience and loss of coding efficiency. 
+The value of $\gamma$ is a design parameter that can be selected according to the application requirements and networking conditions, e.g., available bandwidth or packet loss probability. A possible objective criterion is to choose $\gamma$ based on the predefined increase percentage of the Lagrangian cost and balancing constrains of reference frame usage. In this work 50\% increase of the Lagrangian cost was defined as adequate whenever the unbalance of the reference frame usage is greater than 3 CTUs (see Figure~\ref{fig:weightfct}).
 
 
 \subsubsection{BD-PSNR under packet loss free conditions}
@@ -366,7 +365,7 @@ Table~\ref{tab:bd_error_free} shows the Bjontegaard's Delta PSNR (BD-PSNR)  valu
 \begin{table}[tb]
     \centering
     \newcommand{\resultswidth}{1.5cm}
-    \caption{BD-PSNR (dB) in comparison with the reference HEVC for the proposed reference frame selection algorithm (\PROP) in comparison with existing methods, for the case without errors.}
+    \caption{BD-PSNR (dB) of the proposed reference frame selection algorithm (\PROP) in comparison with existing methods for the case where no packets are lost.}
     \TABLESVSPACE
     \footnotesize
     \label{tab:bd_error_free}

041_ImprovedErrorResilience/Results/TableResultsRandomLossRefPicSelection.tex

@@ -2,7 +2,7 @@
 \newcommand\nlines{4}
 \newcommand{\resultswidth}{1.3cm}
 \centering
-\caption{Average PSNR (dB) under random loss events obtained with the proposed reference frame selection and three reference methods using the Low-delay configuration (LD).}
+\caption{Average PSNR (dB) under random loss events obtained with the proposed reference frame selection and three reference methods for the Low-delay configuration (LD).}
 \TABLESVSPACE
 \small
 \footnotesize
@@ -94,7 +94,7 @@
 \newcommand{\resultswidth}{1.3cm}
 \newcommand\nlines{4}
 \centering
-\caption{Average PSNR (dB) under random loss events obtained with the proposed reference frame selection and three reference methods using the Random-access configuration (RA).}
+\caption{Average PSNR (dB) under random loss events obtained with the proposed reference frame selection and three reference methods for the Random-access configuration (RA).}
 \TABLESVSPACE
 \small
 \footnotesize

042_ConcealmentAwareEncoding/ConcealmentAwareEncoding.tex

@@ -10,7 +10,7 @@
 \vspace*{\fill}
 \section{Introduction}
 
-In this chapter an error concealment-aware encoding scheme is proposed to improve the quality of decoded video in networks prone to errors and data loss.
+In this chapter an error concealment-aware encoding scheme is proposed to improve the quality of video  delivered over networks prone to errors and data loss.
 The method aims to enhance the reconstruction of recovered lost frames by optimising the efficiency of the \ac{ec}.
 The proposed scheme is based on a scalable coding approach where the best \ac{ec} methods to be used at the decoder are optimally determined at the encoder and signalled through \ac{sei} messages. Such optimal \ac{ec} modes are found by simulating transmission losses followed by a Lagrangian optimisation of the \textit{signalling rate - \ac{ec} distortion} cost. A generalised saliency-weighted distortion is used and the residue between coded frames and their EC substitutes is encoded using a rate-controlled enhancement layer. When data loss occurs the decoder uses the signalling information to  improve the reconstruction quality.
 
@@ -133,7 +133,8 @@ In this method only quadratic polynomials are considered, expressed as:
 \begin{equation}\label{EQ5:PolyFlow}
 f(\mathbf{p}) \sim \mathbf{p}\mathbf{A}_1\mathbf{p}^T+\mathbf{b}_1\mathbf{p}^T+C_1
 \end{equation}
-where $\mathbf{p} = [x\ y]$, $\mathbf{A_1}$ is a symmetric matrix, $\mathbf{b_1}$ is a row vector and $C_1$ is a scalar. The coefficients are estimated using weighted least squares fit to the signal values in the neighbourhood. The use of quadratic polynomials was chosen primarily due to simplicity and to keep the computational complexity as low as possible, accepting a small penalty in the estimation accuracy. However, as stated in \cite{farneback2002polynomial} the extension to higher degree is straightforward.
+where $\mathbf{p} = [x\ y]$, $\mathbf{A_1}$ is a symmetric matrix, $\mathbf{b_1}$ is a row vector and $C_1$ is a scalar. The coefficients are estimated using weighted least squares fit to the signal values in the neighbourhood. 
+\HIGHLIGHT{ The use of quadratic polynomials was chosen primarily due to their low computational complexity and acceptable estimation accuracy. Nevertheless, as stated in \cite{farneback2002polynomial} the extension to higher degrees is straightforward.}
 
 Since the result of a polynomial expansion is such that each neighbourhood is approximated by a polynomial, it is necessary to analyse what happens if a polynomial undergoes an ideal translation.
 Let's consider the signal $f_1(\mathbf{p})$ represented by~(\ref{EQ5:PolyFlow}) and a signal $f_2$ which corresponds to a displacement of $f_1$, given by $\mathbf{d}=[v_x \ v_y]$. This can be expressed by the following:
@@ -424,8 +425,8 @@ In this section the performance of the proposed method is evaluated only for int
 Since intra \acs{cu} result in higher amount of bits, they also require a higher number of packets, making them more prone to errors and data loss.
 Results indicate that for an intra-period of 16 frames, approximately 42\% of the packets correspond to intra coded slices, which is a non-negligible probability of one of them being hit by transmission errors which would lead to packet loss.
 Moreover, by limiting the proposed method to intra frames, the amount of required overhead is reduced.
-In this case  a constant $\lambda_0=10$  was used, without taking into consideration saliency estimation. 
+In this case  a constant $\lambda_0=10$  was used for all test cases, without taking into consideration saliency estimation. 
-This value results from an exhaustive search targetting acceptable quality gains with a low overhead (i.e., average overhead under 5\%).
+This value results from an experimental study using several sequences and different packet loss ratios with the objective of obtaining the maximum quality gain with low overhead (i.e., average overhead under 1\%). This is not a critical value for the performance of the proposed method. It was found that small variations do not lead to significant quality changes.
 
 Since a relevant performance indicator is the error propagation over temporally predicted frames, this was evaluated using a single intra-frame loss.
 Figures~\ref{FIG5:singlelosslowdelay} and~\ref{FIG5:singlelossramdon} show the \ac{psnr} results when frame Intra-frame $\#16$ is missing, for \ac{ld} and \ac{ra} encoding configurations, respectively.
@@ -632,7 +633,9 @@ These results reveal that the proposed method is able to improve the reconstruct
 These results also indicate that the proposed method outperforms both reference methods right after the time instant when the loss occurs. The lost frame itself is recovered with much higher quality than using the reference \ac{ec} methods.
 This is due to the use of the optimal \ac{ec} mode to recover each \ac{ecu}, benefiting from the \ac{sei} information received from the encoder.
 In this case, the proposed method is able to achieve quality gains up to 5~dB, decreasing the error propagation and increasing the \ac{psnr} of affected frames by up to 6~dB.
-Comparing the results of Figures~\ref{FIG5:singlelossInter}~and~~\ref{FIG5:singlelossIntra} one may conclude that the loss of intra-frame leads to higher error propagation. For those cases where an intra-coded frame is missing, not only the reconstructions quality is lower for all methods, but also a longer (in terms of number of frames) error propagation occurs.
+Comparing the results of Figures~\ref{FIG5:singlelossInter}~and~~\ref{FIG5:singlelossIntra} one may conclude that the loss of intra-frame leads to higher error propagation. 
+%For those cases where an intra-coded frame is missing, not only the reconstructions quality is lower for all methods, but also a longer (in terms of number of frames) error propagation occurs. 
+Furthermore, as the \ac{idr} period increases, the error propagation also lasts longer with greater impact on video quality.
 
 The visual impact of using the proposed \ac{ec}-aware encoding can be evaluated by observing the recovered frame (i.e., frame \#6) of Basketball Drill, as an example. Figure~\ref{FIG5:RecoveredExample} shows that higher visual quality and less reconstruction artefacts are obtained in comparison with the \MC method. It is worthwhile to note that the proposed method is able to reconstruct the object edges with higher accuracy than the reference \MC.
 
@@ -697,9 +700,9 @@ Since the proposed \ac{ec}-aware scheme is able to achieve higher quality withou
 An advantage of using a feedback channel is the low extra complexity required to implement robust coding because the encoder receives specific information about transmission errors. This allows straightforward spatio-temporal localization of the lost data without computational effort.
 
 
-Table~\ref{tab:overall_gain_ratio} shows the average quality differences (\textbf{$\mathbf{\Delta}$ PSNR})between the reference method (\REF) and the proposed one plus three other previous works. Two different test conditions are shown in the table: \ac{plf} and PLR=10\%. Moreover, an efficiency measure  $R$ is defined, as the  ratio between the quality gains obtained under packet loss and the quality decrease in the \ac{plf} case, i.e.,
+Table~\ref{tab:overall_gain_ratio} shows the average quality differences (\textbf{$\Delta$ PSNR}) between the reference method (\REF) and the proposed one plus three other previous works. Two different test conditions are shown in the table: no packet loss (\NOLOSS) and PLR=10\%. Moreover, an efficiency measure  $R$ is defined, as the  ratio between the quality differences with and without packet loss, i.e.,
 \begin{equation}
-R = \frac{\Delta PSNR(\ac{plr}=10\%)}{-\Delta PSNR(Error\text{-}free)}.
+R = \abs{\frac{\Delta PSNR(\ac{plr}=10\%)}{\Delta PSNR(No\text{-}Loss)}}.
 \end{equation}
 
 

043_DynamicErrorRobustness/DynamicErrorRobustness.tex

@@ -70,10 +70,9 @@ The remaining configuration was kept as described in Section~\ref{Section3:Error
 
 \subsubsection{Preliminary simulation results and discussion}
 
-\begin{figure}[!t]
+\begin{figure}[p]
 	\centering
 	\input{043_DynamicErrorRobustness/Results/Quality_PLR_Methods.tex}
-	\vspace{-4mm}
 	\caption{Average decoded video quality for the bitstreams encoded with the main bitrate (bps).}
 	\label{Figure6:AverageQualityMain}
 \end{figure}
@@ -83,10 +82,9 @@ Figure~\ref{Figure6:AverageQualityMain} shows the results obtained with the test
 Results on Figure~\ref{Figure6:AverageQualityReduction} were obtained by reducing the main bitrate by 40\%. This is used to evaluate the impact of the network bandwidth on the decoded video quality. The results shown in the figures correspond to the same test sequences, in order to allow an accurate and fair comparison.
 
 
-\begin{figure}[!t]
+\begin{figure}[p]
 \centering
 \input{043_DynamicErrorRobustness/Results/Quality_PLR_Methods_Reduced.tex}
-\vspace{-4mm}
 \caption{Average decoded video quality using a 40\% reduction of the main bitrate (bps).}
 \label{Figure6:AverageQualityReduction}
 \end{figure}
@@ -213,6 +211,14 @@ Transmission networks with higher loss ratios require a higher bit allocation fo
 \end{enumerate}
 
 
+These parameters were compared against the optimal stream, selected from \SMEC to \SMEXPELTWEN.
+This aims at finding the correlation between the input parameters and the output variable, i.e., target stream.
+In this study the streams are characterised by a number between 1 and 4.
+Table~\ref{Table6:Correlation} shows the absolute Person Correlation coefficient \cite{PersonCorrelation_2015} obtained across all sequences, bitrates and \acs{plr}.
+The table shows the results for the parameters previously described, as well as, a new parameter obtained from the average magnitude of the optical flow measured using the method in~\cite{DeepFlow2013} \mbox{(\textit{OPT\_FLOW})}.
+Results in Table~\ref{Table6:Correlation} show that the selected parameters are all related with the optimal streams, with the ratio of inter-coded blocks having the highest correlation. Moreover, one can also notice that the parameters related with the motion activity, i.e., \ac{ti} and \textit{AVG\_MV}, are also correlated with the output variable.
+Since both  \textit{AVG\_MV} and \textit{OPT\_FLOW} represent similar information (motion insensitive), only  \textit{AVG\_MV} was used in the proposed method, due to its higher Person Correlation coefficient.
+
 \begin{table}[tb]
 	\centering
 	\caption{Pearson Correlation coefficient between the different parameters and the optimal robust coding techniques (optimal stream).}
@@ -232,23 +238,24 @@ Transmission networks with higher loss ratios require a higher bit allocation fo
 	\end{tabular}
 \end{table}
 
-These parameters were compared against the optimal stream, selected from \SMEC to \SMEXPELTWEN.
-This aims at finding the correlation between the input parameters and the output variable, i.e., target stream.
-In this study the streams are characterised by a number between 1 and 4.
-Table~\ref{Table6:Correlation} shows the absolute Person Correlation coefficient \cite{PersonCorrelation_2015} obtained across all sequences, bitrates and \acs{plr}.
-The table shows the results for the parameters previously described, as well as, a new parameter obtained from the average magnitude of the optical flow measured using the method in~\cite{DeepFlow2013} (\textit{OPT\_FLOW}).
-Results in Table~\ref{Table6:Correlation} show that the selected parameters are all related with the optimal streams, with the ratio of inter-coded blocks having the highest correlation. Moreover, one can also notice that the parameters related with the motion activity, i.e., \ac{ti} and \textit{AVG\_MV}, are also correlated with the output variable.
-Since both  \textit{AVG\_MV} and \textit{OPT\_FLOW} represent similar information (motion insensitive), only  \textit{AVG\_MV} was used in the proposed method, due to its higher Person Correlation coefficient.
-
-
 \subsection{Neural network}
 
 A \ac{nn} is a computational model based on the structure and functions of biological neural networks.
 Mainly, a \ac{nn} consists of three different layers: the input layer which receives data from an external environment, the hidden layer which processes the inputs and an output layer~\cite{NeuralNetworks1994}.
 In the past \acs{nn} have been widely used in video processing applications such as, quality evaluation~\cite{VideoQualityNN_TNN2006}, video analysis~\cite{VideoAnalysisNN_2018} and video coding~\cite{RateDistortionNN_VCIP2017,HEVCComplexityNN_TIP2018}. 
-Neural networks are used in the proposed method since they are universal approximators, which can fit arbitrary complex data that could not be approximated using analytical models~\cite{NeuralNetworks_2016}.
+Neural networks are used in the proposed method since they are universal estimators, which can fit arbitrary complex data not easily approximated by analytical models~\cite{NeuralNetworks_2016}.
+Therefore, the complex relationship that describes the dependency of video quality from the input video, compressed stream and network parameters is adequate to be modelled by a \ac{nn}, which can capture multi-dimensional statistical nature of the problem through the training process.
 In this work a feed-forward network is used, which connects every neuron in a given layer to the neurons of the subsequent layer.
 
+
+
+Figure~\ref{FIG6:NeuralNetwork} shows the diagram of the \ac{nn} used in the proposed method.
+The network contains an input and output layer, as well as, a single hidden layer.
+The input layer receives the six inputs, as described in the previous sub-section. These inputs are correlated with the optimisation variable being maximised, i.e., the output video quality.
+Subsequently, the network transforms the input data through a series of neurons in the hidden layer.
+The output layer comprises four neurons corresponding to each available video stream from \SMEC to \SMEXPELTWEN (different error resilience techniques).
+Finally, the output with the highest value is selected and the corresponding stream is used for transmission, since it is the one expected to achieve the highest video quality for a particular scenario.
+
 \begin{figure}[tb]
 	\centering
 	\input{043_DynamicErrorRobustness/Figures/NeuralNetwork.tikz}
@@ -257,15 +264,8 @@ In this work a feed-forward network is used, which connects every neuron in a gi
 	\FIGSVSPACE
 \end{figure}
 
-Figure~\ref{FIG6:NeuralNetwork} shows the diagram of the \ac{nn} used in the proposed method.
+In the proposed approach a shallow \ac{nn} is used for simplicity and to avoid over-fitting, due to the limited test cases~\cite{NeuralNetworkDesign_2014}. As shown in Figure~\ref{FIG6:NeuralNetwork}, the \ac{nn} used in this method has three layer: an input layer with 6 nodes, an hidden layer with 32 nodes and an output layer with 4 nodes. 
-The network contains an input and output layer, as well as, a single hidden layer.
+By avoiding over-fitting, it is guaranteed that the proposed method would continue to  work in future test conditions, not used for the \ac{nn} training.
-The input layer receives the six inputs, as described in the previous sub-section. These inputs are correlated with the optimisation variable being maximised, i.e., the output video quality.
-Subsequently, the network transforms the input data through a series of neurons in the hidden layer.
-The output layer comprises four neurons corresponding to each available video stream from \SMEC to \SMEXPELTWEN (different error resilience techniques).
-Finally, the output with the highest value is selected and the corresponding stream is used for transmission, since it is the one expected to achieve the highest video quality for a particular scenario.
-
-In the proposed approach a shallow \ac{nn} is used for simplicity and to avoid over-fitting, due to the limited test cases~\cite{NeuralNetworkDesign_2014}. As shown in Figure~\ref{FIG6:NeuralNetwork}, the \ac{nn} using in this method has three layer: input layer with 6 nodes, an hidden layer with 32 nodes and an output layer with 4 nodes. 
-By avoiding over-fitting, it is guaranteed that the proposed method would continue to  work in future test conditions, not used for the network training.
 The number of nodes in the hidden layer was determined by comparing the performance obtained for different cases using the cross-entropy performance metric~\cite{CrossEntropy_2005}.
 %However, different network topologies with different number of neurons in each layer could also be used to predict between different error resilience techniques, as in the proposed method.
 
@@ -273,10 +273,11 @@ The number of nodes in the hidden layer was determined by comparing the performa
 
 To effectively train and validate the \ac{nn}, firstly the cases corresponding to three video sequence were left out, to be used in the performance evaluation of the proposed method.
 The remaining data set was randomly partitioned into three different groups: (i) 65\% of the test cases are used for training, (ii) 15\% for validation and (iii) 20\% for testing the \ac{nn} performance. 
-\HIGHLIGHT{These ratios were chosen to use a relevant number of samples for testing \ac{nn} performance (i.e., 20\%), and the remaining were used mainly for training, leaving a 15\% samples for validation and stop criteria of the supervised training.
+The choice of these percentages follow the commonly used approach where about 70\% to 80\%  of the dataset is used for training and validation while the remaining 30\% to 20\% is used for testing~\cite{haykin1994neural}.
-The \ac{nn} supervised training uses 557 test conditions, a scaled conjugate gradient algorithm~\cite{NNScaledConjugateTraining_1993} and stopped when the cross-entropy obtained for the validation samples increases, revealing that the training starts to over-fit the training samples and is not being able to generalise to new data.
+%	 to use a relevant number of samples for testing \ac{nn} performance (i.e., 20\%), and the remaining were used mainly for training, leaving a 15\% samples for validation and stop criteria of the supervised training.
-Therefore, 129 test conditions were used for \ac{nn} validation.}
+In this work the \ac{nn} supervised training uses 557 test conditions and a scaled conjugate gradient algorithm~\cite{NNScaledConjugateTraining_1993} to calculate the backpropagation.
-Finally, 172 conditions are used to evaluate the accuracy of the proposed method after training with different conditions.
+Then, 129 test conditions were used for \ac{nn} validation and 172 conditions o to evaluate the accuracy of the \ac{nn} after training with different conditions. The 11 test sequences, encoded at different bitrates and subject to a wide range of \acs{plr}, were used to generate the three groups of the dataset. Since these sequences have quiet different characteristics (e.g, motion, texture and resolution) each group is considered representative of any type of video content.
+The training was stopped when the cross-entropy obtained for the validation samples stop decreasing, which reveals that the \ac{nn} starts to over-fit the training samples and it is not able to generalise to new data.
 
 Tables~\ref{Table6:ConfusionTraining}, \ref{Table6:ConfusionVal} and \ref{Table6:ConfusionTest} show the confusion matrices obtained for training, validation and test sets, respectively.
 In these matrix each column corresponds to the target result, i.e., the cases where the coded streams, \SMEC to \SMEXPELTWEN, would lead to the highest decoded video quality.

043_DynamicErrorRobustness/Results/Quality_PLR_Methods.tex

@@ -1,5 +1,5 @@
 \newcommand{\pltw}{\columnwidth}
-\newcommand{\plth}{0.75\textheight}
+\newcommand{\plth}{0.8\textheight}
 
 \begin{gnuplot}[terminal=epslatex,terminaloptions={monochrome font ",\PlotFont" size \convertlen{\pltw},\convertlen{\plth}}]
 load 'carreira_phd_thesis_gnuplot_template.cfg'

043_DynamicErrorRobustness/Results/Quality_PLR_Methods_Reduced.tex

@@ -1,5 +1,5 @@
 \newcommand{\pltw}{\columnwidth}
-\newcommand{\plth}{0.7\textheight}
+\newcommand{\plth}{0.8\textheight}
 
 \begin{gnuplot}[terminal=epslatex,terminaloptions={monochrome font ",\PlotFont" size \convertlen{\pltw},\convertlen{\plth}}]
 load 'carreira_phd_thesis_gnuplot_template.cfg'

050_Conclusions/Conclusion.tex

@@ -53,7 +53,8 @@ Therefore, an adaptive scheme based on six input parameters and a pre-trained \a
 Results revealed that using the proposed adaptive scheme, an efficient trade-off between coding efficiency and error resilience is achieved for the majority of test conditions.
 Using this architecture, the problem of robust video coding and decoding was addressed using a combination of different methods, which leads to a superior performance in comparison with individual techniques.
 
-As a final remark, one may conclude that the use of the novel methods proposed in this thesis consistently contributes to increase the robustness of video communications using state of the art high efficiency video encoding tools in error prone networks.
+As a final remark, the novel methods proposed in this thesis are also suitable for emerging formats such as, multi-view and 360-degree video. Since not all visual information is equally important at the same time thus, dynamic error resilient methods like the ones investigated in this thesis are particularly suitable for these formats.
+Overall, one may conclude that the use of the novel methods proposed in this thesis consistently contributes to increase the robustness of video communications using state of the art high efficiency video encoding tools in error prone networks.
 The best performance is achieved by combining robust video coding techniques with efficient error concealment algorithms optimised at the encoder, using an optimised dynamic approach.
 
 
@@ -73,7 +74,7 @@ As the complexity of the decoder is also a relevant issue, due to the increasing
 The performance of the adaptive selection of the robust coding technique can also be improved by combining different classification algorithms, based on more complex neural network architectures.
 
 Furthermore, the error robustness schemes proposed in this research work can be extended to cope with robust transmission of multi-view video, where errors propagate in the temporal and inter-layer domains.
-Finally, as a new coding standard, entitled Versatile Video Coding, is currently being developed, targeting the deployment of higher-quality video services and emerging applications such as \HIGHLIGHT{360-degree multimedia, where the relative importance of different view points could be used to restrict the application of the proposed techniques} or high-dynamic-range video. This opens new research directions to the study the impact of errors in such video applications and the development of novel efficient techniques to guarantee high quality visual experiences, to users of such emerging video formats.
+Finally, as a new coding standard, entitled Versatile Video Coding, is currently being developed, targeting the deployment of higher-quality video services and emerging applications such as, 360-degree multimedia. This opens new research directions to the study the impact of errors in such video applications and the development of novel efficient techniques to guarantee high quality visual experiences, to users of such emerging video formats.
 
 
 

carreira_phd_thesis.tex

@@ -16,7 +16,7 @@
 \selectoriginallanguage
 
 % \newcommand{\HIGHLIGHT}[1]{\textcolor{red}{#1}}
-\newcommand{\HIGHLIGHT}[1]{\textcolor{red}{#1}}
+\newcommand{\HIGHLIGHT}[1]{}
 %\newcommand{\HERE}{\textcolor{red}{CHECK HERE!}}
 %\newcommand{\VERSION}[1]{\textcolor{red}{(r#1)}}
 \newcommand{\VERSION}[1]{}

carreira_phd_thesis_abr.sty

@@ -115,7 +115,7 @@
 
 \customnewacronym{fc}{FC}{Frame-Copy}
 \customnewacronym{mc}{MC}{Motion-Copy}
-\customnewacronym{bma}{BMA}{Boundary Match Algorithms}
+\customnewacronym{bma}{BMA}{Boundary Matching Algorithms}
 \customnewacronym{mve}{MVE}{Motion Vector Extrapolation}
 \customnewacronym{pmve}{PMVE}{Pixel-based Motion Vector Extrapolation}
 \customnewacronym{hmve}{HMVE}{Hybrid Motion Vector Extrapolation}
@@ -145,10 +145,10 @@
 \def\LOSTALL{$IDR_{Loss}$\xspace}
 \def\KEEPINTRA{$IDR_{NoLoss}$\xspace}
 
-\def\NOLOSS{PLF}
+\def\NOLOSS{No-Loss}
 
 \customnewacronym{plr}{PLR}{Packet Loss Ratio}
-\customnewacronym{plf}{PLF}{Packet Loss Free}
+% \customnewacronym{plf}{PLF}{Packet Loss Free}
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %% Reference methods