From 61dd330c24726b7977525efe0a3055d0e20e5949 Mon Sep 17 00:00:00 2001 From: Joao Carreira Date: Fri, 14 Dec 2018 02:39:48 +0000 Subject: [PATCH] Rename files and folders --- 010_Introduction/Introduction.tex | 33 ++------ 021_VideoCoding/VideoCodingReview.tex | 78 +++++++++--------- .../ErrorResilienceEvaluation.tex | 63 +++++++------- .../ImprovedErrorResilience.tex | 32 ++++--- .../ConcealmentAwareEncoding.tex | 19 ++--- .../Results/ECMQualitySingleloss.tex | 4 +- .../DynamicErrorRobustness.tex | 14 ++-- .../Figures/JointlyRobustnessDiagram.tikz | 0 .../Figures/NeuralNetwork.tikz | 0 .../Results/AdaptiveSequenceList.tex | 0 .../Results/PlotQualty_vs_PLR_Predicted.bash | 0 .../Results/PlotQualty_vs_PLR_Predicted.tex | 0 .../Results/Quality_PLR_Methods.bash | 0 .../Results/Quality_PLR_Methods.pdf | Bin .../Results/Quality_PLR_Methods.tex | 0 .../Results/Quality_PLR_Methods_Reduced.tex | 0 .../Results/cfgData.py | 0 .../Results/summary_decoding | 0 carreira_phd_thesis.sty | 2 +- carreira_phd_thesis.tex | 14 ++-- 20 files changed, 114 insertions(+), 145 deletions(-) rename 043_EndToEndRobustSystem/EndToEndRobustSystem.tex => 043_DynamicErrorRobustness/DynamicErrorRobustness.tex (98%) rename {043_EndToEndRobustSystem => 043_DynamicErrorRobustness}/Figures/JointlyRobustnessDiagram.tikz (100%) rename {043_EndToEndRobustSystem => 043_DynamicErrorRobustness}/Figures/NeuralNetwork.tikz (100%) rename {043_EndToEndRobustSystem => 043_DynamicErrorRobustness}/Results/AdaptiveSequenceList.tex (100%) rename {043_EndToEndRobustSystem => 043_DynamicErrorRobustness}/Results/PlotQualty_vs_PLR_Predicted.bash (100%) rename {043_EndToEndRobustSystem => 043_DynamicErrorRobustness}/Results/PlotQualty_vs_PLR_Predicted.tex (100%) rename {043_EndToEndRobustSystem => 043_DynamicErrorRobustness}/Results/Quality_PLR_Methods.bash (100%) rename {043_EndToEndRobustSystem => 043_DynamicErrorRobustness}/Results/Quality_PLR_Methods.pdf (100%) rename {043_EndToEndRobustSystem => 043_DynamicErrorRobustness}/Results/Quality_PLR_Methods.tex (100%) rename {043_EndToEndRobustSystem => 043_DynamicErrorRobustness}/Results/Quality_PLR_Methods_Reduced.tex (100%) rename {043_EndToEndRobustSystem => 043_DynamicErrorRobustness}/Results/cfgData.py (100%) rename {043_EndToEndRobustSystem => 043_DynamicErrorRobustness}/Results/summary_decoding (100%) diff --git a/010_Introduction/Introduction.tex b/010_Introduction/Introduction.tex index 732b09b..c095de6 100644 --- a/010_Introduction/Introduction.tex +++ b/010_Introduction/Introduction.tex @@ -15,7 +15,6 @@ Such need has been driven not only by higher resolutions but also by 360-degree, Moreover, the ever increasing traffic generated by mobile multimedia applications \cite{CiscoMobile1621_Feb2017} and different video services with growing popularity (e.g., Netflix and Facebook) impose new challenges to the existing networks. To cope with such demanding requirements a new coding standard was developed by the \ac{itut} \ac{vceg} and the \ac{iso}/\ac{iec} \ac{mpeg} standardisation organisations, referred to as H.265 - \ac{hevc}~\cite{H265StandardFeb2018,HEVCOverviewDez2012}. -\glsunset{h264} The \ac{hevc} is currently the main video coding framework to extend current services towards higher resolution formats delivered with better compression efficiency. Similar to its predecessor, \ac{hevc} is a block-based hybrid video codec using predictive, transform and entropy coding. @@ -27,14 +26,11 @@ Thus, when \ac{hevc} streams incur in transmission losses, this leads to signifi This is mostly due to the strong decoding dependencies imposed by the highly complex prediction modes of \ac{hevc}, which assume \mbox{error-free} transmission~\cite{ROPE2007}. A possible approach to encode robust video streams is to increase the error resilience of \ac{hevc} by including \mbox{error-aware} mechanisms in the coding process. As a result, the effect of error propagation, due to prediction mismatch at the decoding side, will be limited. - - The widespread use of \ac{hevc} in broadcast and multimedia delivery services is based on transport technologies such as \ac{rtp}~\cite{HEVCStreamMay2012}, \ac{ts}~\cite{HEVCLayerIntegrationDez2012}, or more recent standards as \ac{dash}~\cite{HEVCHTTPStreamAug2016}. Recent developments also enable combinations of different technologies, e.g, \mbox{HTTP-based} streaming and \ac{ts}, to increase the flexibility of over-the-top video services\cite{Broadcast2017}. For content delivery over heterogeneous networks, the \ac{mmt}~\cite{ATSCMMT2016} is another possible technology to be used with the \ac{hevc} standard. Besides the transport technologies, -the overall error robustness of video transmission may also benefit from the increasing flexibility of multimedia systems to cope with heterogeneous networks, by using auxiliary streams that allow the video decoder to increase the quality of the recovered lost slices~\cite{ErrorResilienceHeterogenousNetworks2016,HeterogeneousStreaming2016}. +the overall error robustness of video transmission may also benefit from the increasing flexibility of multimedia systems to cope with heterogeneous networks, by using auxiliary streams that allow the video decoder to increase the quality of the recovered lost slices~\cite{ErrorResilienceHeterogenousNetworks2016,HeterogeneousStreaming2016}. Moreover, error resilience techniques may also take advantage of the multi-stream based approaches, where each stream allows for different levels of error robustness~\cite{HEVCHTTPStreamAug2016,PerfMediaTransport2017}. - In summary, increasing the error robustness of the highly compressed video streams is a relevant research challenge, whose solutions will be useful to increase the adaptability for transmission over diverse network conditions and also cope with communication errors, providing increased levels of \ac{qoe} to the end users at all times. In general, increased error robustness in video transmission can be jointly achieved at three different stages: (i) using robust video encoding, by selecting robust coding modes, (ii) after encoding, by adding robust features to the compressed streams, as side-information, and (iii) using robust video decoding techniques, e.g., efficient \ac{ec}. This is exemplified in Figure~\ref{FIG1:RobustVideoTransmission} where at the encoder different layers are used to increase the error robustness, which results in a video stream composed by a \ac{vcl} and \ac{sei} layers. At the decoder two processing layers are used to achieve robust video decoding. The \ac{hevc} reference codec is continuously being improved and extended by multiple proponents from around the world, which also address the problem of error resilience and robust delivered transmission. @@ -54,27 +50,21 @@ Thus, this work investigates novel techniques to increase the video quality of \ The following objectives were pursued in this work: \begin{enumerate} \item -\textbf{Study and evaluation of error robustness in highly compressed video:} -The objective of this research study is to evaluate the robustness against transmission errors of highly efficiently coded video. -The study and its conclusions aim at finding relevant research directions to develop new methods for increasing the error robustness of coded streams. +\textbf{Study and evaluation of error robustness in highly compressed video:} +The objective of this research study is to evaluate the robustness against transmission errors of highly efficiently coded video. +The study and its conclusions aim at finding relevant research directions to develop new methods for increasing the error robustness of coded streams. The results and conclusions of this study were published in conference paper (\ref{Conf:ErrorResilience}) and in journal paper (\ref{Journal:TwoStage}) (see Appendix~\ref{Appen:Papers}). \item -\textbf{Robust video coding:} -At the encoder-side, novel techniques to extend the existing error robustness of highly predictive video coding techniques are investigated in order to increase the received video quality over error-prone networks. -This research topic resulted in a new method to reduce the error propagation in motion vector coding algorithms, which was published in conference paper (\ref{Conf:DynamicTMVP}). +\textbf{Robust video coding:} +At the encoder-side, novel techniques to extend the existing error robustness of highly predictive video coding techniques are investigated in order to increase the received video quality over error-prone networks. +This research topic resulted in a new method to reduce the error propagation in motion vector coding algorithms, which was published in conference paper (\ref{Conf:DynamicTMVP}). Moreover, a novel two-stage approach was devised to improve the error robustness of highly efficiently coded video at the encoder and streaming levels, resulting in two conference publications (\ref{Conf:MotionVectorRedundancies} and~\ref{Conf:Chkb}) and a journal publication (\ref{Journal:TwoStage}). \item \textbf{Robust video decoding:} error concealment techniques are used to increase the error robustness of video decoding and improve the perceived video quality. The paradigm of \ac{ec} is evaluated at the encoder-side, as part of the encoding process, in order to optimise the reconstruction process at the decoder-side. This research resulted in a novel error concealment-aware technique to optimise the error concealment process at the encoder-side, which was proposed for intra-frame only in a conference paper (\ref{Conf:ConcAware}) and for generic robust video transmission in the journal (\ref{Journal:ConcAware}). -%\item \textbf{jointly error resilience and concealment:} the ultimate goal of this research work is to strike a balance between the designed error resilience and error concealment techniques so as to meet higher quality demands for different video contents characteristics and network conditions. \end{enumerate} -%\HIGHLIGHT{Contributions} - -%Finally, the ultimate goal of this research work is to strike a balance between the designed error resilience and error concealment techniques so as to meet a set of \ac{qos}/\ac{qoe} requirements. -%Since the ultimate quality judgement is made by end users, subjective tests will constitute a part of the research work in order to validate the objective gains reported as a result of the deployment of various proposed and implemented methods. Given the importance of the \ac{hevc} standard in the near future, the simulation scenarios will be defined based on practical service requirements, such as, bit rate allocation, transmission delay and required QoS. - \section{Outline of this thesis} This thesis is organised as follows. The current chapter introduces the context, motivation and objectives of this thesis, as well as the original contributions. @@ -91,17 +81,12 @@ Moreover, to reduce the effect of mismatched motion predictions (shown in Chapte This method is able to improve the decoded video quality using a small overhead. Chapter~\ref{Chapter:ConcealmentAwareTechniques} addresses the problem of robust \ac{ec} of high efficiently coded video streams. -%This work was motivated by the requirements for more accurate \ac{ec} algorithms which are able to reduce the error propagation. Different \ac{ec} algorithms were taken into consideration in order to develop a concealment-aware coding scheme. The proposed scheme is based on a scalable coding approach where the best \ac{ec} methods to be used at the decoder are optimally determined at the encoder and signalled to the decoder. -%Such optimal EC\ac{ec} modes are found by simulating transmission losses followed by a \ac{rd} optimisation of the signalling rate and \ac{ec} distortion cost. A generalised saliency-weighted \ac{rd} optimisation is used and the residue between coded frames and their EC substitutes is encoded using a rate-controlled enhancement layer. -%When data loss occurs the decoder uses the signalling information is used at the decoder, in case of data loss, to improve the reconstruction quality. -%The simulation results show that the proposed method achieves consistent quality gains in comparison with other reference methods and previous works. - -Chapter~\ref{Chapter:EndToEndRobustSystem} proposes a novel adaptive error robustness method, which combines the different techniques proposed in this thesis. -At the encoder-side various video streams are generated with different levels of compression efficiency and error robustness. +Chapter~\ref{Chapter:EndToEndRobustSystem} proposes a novel adaptive error robustness method, which combines the different techniques proposed in this thesis. +At the encoder-side various video streams are generated with different levels of compression efficiency and error robustness. Then, a pre-training \ac{nn} is used to predict which video stream is expected to achieve the highest decoded video quality for a particular scenario. The selection is based on different input parameters, which include information from the video signal, the coded stream and the transmission network. Finally, Chapter~\ref{Chapter:Conclusions} concludes this dissertation and presents some suggestions for future work. diff --git a/021_VideoCoding/VideoCodingReview.tex b/021_VideoCoding/VideoCodingReview.tex index cefd0cc..43b436f 100644 --- a/021_VideoCoding/VideoCodingReview.tex +++ b/021_VideoCoding/VideoCodingReview.tex @@ -1,8 +1,4 @@ -% Figure folders -\renewcommand{\FigsPath}{./021_VideoCoding/Figures} - -% \chapter{\HIGHLIGHT{The \ac{hevc} Standard}} \chapter{The High Efficiency Video Coding standard \VERSION{3}} \label{Chapter:HEVCReview} @@ -40,7 +36,7 @@ In the last stages of the chain, the compressed format is decoded and presented % \begin{figure}[tb] % \centering -% \input{\FigsPath/GeneralVideoSystem.tikz} +% \input{021_VideoCoding/Figures/GeneralVideoSystem.tikz} % \caption{Block diagram of a generic video coding system.} % \label{FIG2:GeneralVideoSystem} % \end{figure} @@ -54,7 +50,7 @@ This is the reason why the coding scheme is called hybrid, as it combines quiet \begin{figure}[t] \centering -\input{\FigsPath/HybridCodingBlockDiagramPrePost.tikz} +\input{021_VideoCoding/Figures/HybridCodingBlockDiagramPrePost.tikz} \caption{Block diagram of a generic video coding system, including the inner blocks of a hybrid encoding and decoding scheme (T and Q corresponds to the transform and quatisation operations, respectively).} \label{Figure2:HybridCodec} \end{figure} @@ -63,7 +59,7 @@ This is the reason why the coding scheme is called hybrid, as it combines quiet In general, there are two kinds of redundancy in video signal: spatial and temporal. On the one hand, the spatial redundancy is exploited through intra frame prediction, where the underlying mechanism is to find the best matching block for the current one, by searching the neighbouring region. -This is possible due to the existence of high correlation among neighbouring pixels within a frame. +This is possible due to the existence of high correlation among neighbouring pixels within a frame. As a result, when intra frame prediction is used, a block of pixels is reconstructed from the neighbouring blocks previously processed. On the other hand, inter frame prediction is designed to exploit the temporal redundancy that exists among consecutive frames, such as those temporally adjacent, e.g., the past frame or the future frame in regard to the current one. Temporal redundancy is normally referred to as the correlation between two adjacent frames since they usually have quiet similar visual content, such as foreground objects and background. @@ -90,20 +86,20 @@ coding syntax and semantics are standardised. The constrains and functionalities are defined through profiles and levels, while the semantic meaning of syntax elements define the decoding process, such that every decoder conforming to the standard generates the same output when given a conforming stream. Such delimitation of scope in the standard concede maximum freedom for developers and manufactures to develop proprietary optimisations, allowing to cope with various applications’ requirements (\emph{e.g.}, balancing the compression ratio and the coding complexity). Figure~\ref{fig:HEVCBlockDiagram} illustrates the block diagram of the \ac{hevc} encoder. It follows the same overall hybrid architecture as its predecessors starting from H.261. -The encoder uses both intra- and inter-picture prediction to exploit spatial and temporal redundancies, respectively. +The encoder uses both intra- and inter-picture prediction to exploit spatial and temporal redundancies, respectively. Subsequently, the prediction residue is transformed by linear spatial transform and the resulting coefficients are then quantised and entropy coded, further reducing the bitstream redundancy. %The resulting bitstream is combined with control information, which signals the prediction chosen, motion vectors, filtering operations and bitstream headers. \begin{figure}[tb] \centering \includegraphics[width=\columnwidth]{021_VideoCoding/Figures/HEVCBlockDiagram} - %\input{\FigsPath/HEVCBlockDiagram.tikz} + %\input{021_VideoCoding/Figures/HEVCBlockDiagram.tikz} \caption{Block diagram of an HEVC standard encoder.} \label{fig:HEVCBlockDiagram} \end{figure} However, there are some novelties introduced in the \ac{hevc} standard. As depicted in Figure~\ref{fig:HEVCBlockDiagram}, the encoding process follows a block-based approach, therefore each picture is partitioned into larger blocks, referred to as \acs{ctu}, with a maximum size of $64\times64$ pixels. -Subsequently, a dynamic quad-tree partitioning is used to divide the picture into \acs{cu}. +Subsequently, a dynamic quad-tree partitioning is used to divide the picture into \acs{cu}. These units are introduced as the basic processing units for prediction, while for the transform and quantisation operations, transform units are used. In order to generate the prediction residue, the \ac{hevc} may either use one of 34 intra-prediction modes or take advantage of symmetric and asymmetric partitions for motion estimation and inter-prediction. The motion estimation is performed in the pictures listed in the reference picture set, which was introduced in the \ac{hevc} to increase the robustness to errors and data loss of reference picture management. @@ -129,8 +125,8 @@ Similar to \ac{h264}~\cite{H264AVCStandard2003}, the \ac{hevc} standard defines The \ac{nal} is used for stream partitioning and synchronization, which is useful to deal with lossy network environments~\cite{H264OverIPJul2003}. Using this concept the video stream is composed of different \ac{nal} units. The \ac{nal} concept provides the ability to map the \ac{vcl} data that represents coded video slices (or frames) on various transport layers, such as \ac{rtp}, MPEG-2 Systems and MPEG DASH. The~\ac{nal} units are classified into two categories: (i) \ac{vcl} and (ii) non-\ac{vcl}, as they carry coded video or associated data, respectively. -The \ac{hevc} standard defines a two bytes header to describe the content of each \ac{nal} unit, which are transmitted using a fixed coding format length, i.e., without variable length codes. -This allow extra error robustness but increases the bit overhead. +The \ac{hevc} standard defines a two bytes header to describe the content of each \ac{nal} unit, which are transmitted using a fixed coding format length, i.e., without variable length codes. +This allow extra error robustness but increases the bit overhead. Figure~\ref{Figure2:NalHeader} shows the structure of the \ac{hevc} \ac{nal} unit header. The first bit \textit{F}, referred to as \textit{forbiden\_bit}, is always zero in order to ensure that a start code can be detected across different protocol standards. This is followed by the \ac{nal} type, which identifies its content, enabling the decoder to either use it if required for a particular decoding purpose or discard it. Another syntax element introduced by the \ac{hevc} standard is the \textit{temporal\_id\_plus\_one} (TIDP), which defines the temporal layer where the \ac{nal} unit belongs. This allows implicit temporal scalability (layers ranging from 0 to 6) by immediately discarding the \ac{nal} unit if it belongs to a layer that is higher than the pre-defined one. In the header, six bits are reserved for future extensions, preventing the creation of extra \ac{nal} units for scalable and multiview extensions, as done in \ac{h264}. \begin{figure}[tb] @@ -155,9 +151,9 @@ In this approach, such set of parameters is multiplexed in the bitstream and rep The \ac{hevc} includes three sets of parameters, \ac{sps}, \ac{pps} and the newly introduced \ac{vps}\cite{HEVCHighLevelReferenceDez2012}. % \ac{sps} include sequence related information, that include the decoder operation point, flags to enable or disable certain tools within a given profile. \Gls{pps} include picture related information, that may change for every picture, e.g., \ac{qp} and tilling information. -The new set of parameters \ac{vps} contains information related to the different layers of the video signal, avoiding duplications like in the \ac{h264} and its extensions. \Gls{vps} aims to provide signalling for every layer, guaranteeing an extendible standard, capable of supporting multiple layers. -Moreover, \ac{vps} also carries information for session negotiation (\emph{e.g.} profile, level and tier). -The \ac{sps} contains information related to the whole sequence (\emph{i.e.}, should affect all coded slices) and related to the decoder operation point, as well as flags to control optional tools and scalability. +The new set of parameters \ac{vps} contains information related to the different layers of the video signal, avoiding duplications like in the \ac{h264} and its extensions. \Gls{vps} aims to provide signalling for every layer, guaranteeing an extendible standard, capable of supporting multiple layers. +Moreover, \ac{vps} also carries information for session negotiation (\emph{e.g.} profile, level and tier). +The \ac{sps} contains information related to the whole sequence (\emph{i.e.}, should affect all coded slices) and related to the decoder operation point, as well as flags to control optional tools and scalability. The \ac{pps} is responsible to carry information that may change for every picture, such as, initial \ac{qp}, flags for picture related tools and tilling information. @@ -187,14 +183,14 @@ Moreover, introducing random-access points into the bitstream increases its erro BLA & Broken link access & Leading pictures cannot be decoded \\ - + \midrule \midrule \multicolumn{3}{c}{\textbf{Leading pictures}} \\ \midrule RADL & \multicolumn{2}{l}{Random access decodable picture}\\ \midrule RASL & \multicolumn{2}{l}{Random access skipped picture}\\ - + \midrule\midrule \multicolumn{3}{c}{\textbf{Temporal sub-layer access pictures}} \\ \midrule @@ -245,7 +241,7 @@ Due to the introduction of more flexible random-access points using \ac{cra} pic \begin{figure}[tb] \centering - \input{\FigsPath/PictureTypes_Leading_Trailing.tikz} + \input{021_VideoCoding/Figures/PictureTypes_Leading_Trailing.tikz} \caption{Leading and trailing pictures associated with the random-access point $I_{1}$ (letters corresponds to the coding type and the numbers to the coding order).} \label{Figure2:LeadingTralingPics} \end{figure} @@ -253,7 +249,7 @@ Due to the introduction of more flexible random-access points using \ac{cra} pic There are two types of leading pictures, as shown in Table~\ref{Table2:PictureTypes}. The \ac{radl} pictures, which can be correctly decoded, since they only depend on the associated \ac{rap} picture and do not have any prior dependencies to trailing pictures associated with the previous \ac{rap} frame. On the contrary, the \ac{rasl} pictures may have dependencies to prior trailing pictures. -When a random-access is performed at the associated \ac{rap} frame, these frames cannot be correctly decoded, therefore they have to be skipped. +When a random-access is performed at the associated \ac{rap} frame, these frames cannot be correctly decoded, therefore they have to be skipped. In the example of Figure~\ref{Figure2:LeadingTralingPics} the leading pictures correspond to \ac{rasl} pictures. \subsubsection{Temporal sub-layer switching pictures} @@ -262,19 +258,19 @@ In the example of Figure~\ref{Figure2:LeadingTralingPics} the leading pictures c A \ac{tsa} picture is a trailing picture that marks a temporal layer switching point. When decoding a sub-set of temporal layers, if a \ac{tsa} picture is found in the temporal layer just above the maximum temporal layer currently being decoded, it is possible to start decoding any number of additional temporal layers. For example, frame $P_6$ in Figure~\ref{Figure2:TSAExample} is considered a \ac{tsa} picture since it only depends on a frame belonging to temporal layer 0 for prediction, as well as, any subsequent frame predicted from the \ac{tsa} frame $P_6$. -In this example, if the decoder is only decoding the temporal layer 0, after decoding $P_6$ it can also decode layer 1, or decode all the available three layers. +In this example, if the decoder is only decoding the temporal layer 0, after decoding $P_6$ it can also decode layer 1, or decode all the available three layers. Although these frames allow higher flexibility in terms of layer switching, they severely constrain the prediction of frames following the \ac{tsa} picture. \begin{figure}[tb] \centering - % \includegraphics[width=0.75\columnwidth]{\FigsPath/HEVC/Pictures_TSA} - \input{\FigsPath/PictureTypes_TSA.tikz} + % \includegraphics[width=0.75\columnwidth]{021_VideoCoding/Figures/HEVC/Pictures_TSA} + \input{021_VideoCoding/Figures/PictureTypes_TSA.tikz} \caption{Temporal layers access pictures example with three layers ($T_0$ to $T_2$).} \label{Figure2:TSAExample} \end{figure} To reduce the constrains in frame prediction due to layer switching, \ac{hevc} introduces the \ac{stsa} picture. -These pictures have similar purpose as the \ac{tsa} frames, but they only guarantee that frames in the same temporal layer as the \ac{stsa} frame are able to be correctly reconstructed. +These pictures have similar purpose as the \ac{tsa} frames, but they only guarantee that frames in the same temporal layer as the \ac{stsa} frame are able to be correctly reconstructed. %This is guaranteed by avoiding any dependencies from frames preceding the \ac{stsa} to be used as reference in its temporal layer. This is guaranteed by not using frames preceding the \ac{stsa} as reference in its temporal layer. One example of a \ac{stsa} frame is also shown in Figure~\ref{Figure2:TSAExample} in frame $P_2$. @@ -285,7 +281,7 @@ Summarising, \ac{stsa} frames can be used to switch to a particular layer, as \a \subsection{Picture partitioning} -The high-level segmentation of a picture in \ac{hevc} is achieved by using four different approaches associated to different data structures: regular slices, dependent slices, tiles and \ac{wpp}. +The high-level segmentation of a picture in \ac{hevc} is achieved by using four different approaches associated to different data structures: regular slices, dependent slices, tiles and \ac{wpp}. Picture partitioning normally serves one or more of the following three purposes: \begin{description} \item[Error robustness:] partitioning the picture into smaller self-contained units in @@ -313,8 +309,8 @@ A new data partition has been included in the \ac{hevc} standard, referred to as \begin{figure}[tb] \centering \begin{tabular}{x{.45\columnwidth}x{.45\columnwidth}} - \input{\FigsPath/Tiles_Slices_Example_1.tikz} & - \input{\FigsPath/Tiles_Slices_Example_2.tikz} \\ + \input{021_VideoCoding/Figures/Tiles_Slices_Example_1.tikz} & + \input{021_VideoCoding/Figures/Tiles_Slices_Example_2.tikz} \\ (a) & (b) \\ \end{tabular} \caption{Examples of the interaction between tiles and slices.} @@ -338,7 +334,7 @@ Each square corresponds to a \ac{ctu} and the numbers correspond to the time ins \begin{figure}[tb] \centering - \input{\FigsPath/WPP.tikz} + \input{021_VideoCoding/Figures/WPP.tikz} \caption{Illustration of the wavefront parallel processing (numbers corresponds to the time instant each \ac{ctu} is processed).} \label{Figure2:WPP} \end{figure} @@ -350,7 +346,7 @@ Each square corresponds to a \ac{ctu} and the numbers correspond to the time ins \label{Section2:CodingStructures} Despite the fact that the \ac{hevc} standard follows a traditional hybrid video coding architecture, it introduces significant changes in the data structures when compared to the previous standards (e.g., \ac{h264}). -In \ac{h264} each frame is divided into units so-called \acs{mb}, which are composed by a $16\times16$ block of luma samples and two $8\times8$ blocks of chroma samples, assuming video data in YUV 4:2:0 chroma sub-sampling. +In \ac{h264} each frame is divided into units so-called \acs{mb}, which are composed by a $16\times16$ block of luma samples and two $8\times8$ blocks of chroma samples, assuming video data in YUV 4:2:0 chroma sub-sampling. The \ac{mb} is considered the basic processing unit in the \ac{h264}, and the respective coding mode defines whether the corresponding pixels are coded as intra or intra prediction. Each \ac{mb} could also be partitioned into sub-blocks, which should follow the same prediction mode. @@ -358,18 +354,18 @@ Each \ac{mb} could also be partitioned into sub-blocks, which should follow the The basic data structures in \ac{hevc} are quite different from \ac{h264}. Each picture is divided into blocks named as \ac{ctu}~\cite{BlockPartHEVC2012}. The maximum \ac{ctu} size is an encoder configuration parameter that is signalled to the decoder, and should be one of the following: $64\times64$, $32\times32$ and $16\times16$ pixels. The use of data structures with larger size in \ac{hevc} was adopted to achieve higher coding efficiency especially in high resolution video. Each \ac{ctu} can be recursively partitioned into four smaller units, referred to as \acs{cu}, until it reaches the minimum unit size, which cannot be smaller than $8\times8$ pixels. Similar to the maximum \ac{ctu} size, the minimum size is also defined at the encoder. For instance, in case of homogeneous regions, large \acs{cu} can be used to represent such regions by using a smaller number of symbols than in the case of using several small units. -Figure~\ref{Figure2:HEVCCodingStructure} illustrates an example of a \ac{ctu} partitioning into \acs{cu}. +Figure~\ref{Figure2:HEVCCodingStructure} illustrates an example of a \ac{ctu} partitioning into \acs{cu}. The partitioning illustrated in the figure leads to the quad-tree structure represented on the left side with the corresponding coding order shown on the right side. Data structures in \ac{hevc} provide a significant improvement in comparison with previous standards, by eliminating the distinction between \ac{mb} and sub-\ac{mb} and using only the \ac{cu} concept. This allows for the multilevel hierarchical quadtree structure to be specified in a simple and elegant way, with size-independent syntax representation. \begin{figure}[t] \centering - \input{\FigsPath/BlockPartitionsWithTree.tikz} + \input{021_VideoCoding/Figures/BlockPartitionsWithTree.tikz} \caption{Representation of a CTU ($64\times64$) and its partitioning into CUs and PUs (left); corresponding quad-tree with coding order (right).} \label{Figure2:HEVCCodingStructure} \end{figure} -The \ac{cu} is the basic processing unit in \ac{hevc}. +The \ac{cu} is the basic processing unit in \ac{hevc}. It is further partitioned into two types of units, specifically \ac{pu} and \ac{tu}, which are data blocks used for prediction and transform operations, respectively. The decision whether to use intra-frame or inter-frame prediction modes is taken at the \ac{cu} level. However, depending on the partitioning of the \ac{cu} into \acs{pu}, different intra or inter coding modes can be used for each \ac{pu}. @@ -419,8 +415,8 @@ Directional intra prediction was first introduced in the \ac{h264} standard~\cit Directional prediction projects the reconstructed samples in the block neighbourhood (see reference pixels in Figure~\ref{Figure2:IntraModesHEVC}) along a specific direction. The \ac{hevc} increases the number of possible directions from 8 (in \ac{h264}) to 33~\cite{IntraCodingHEVC2012}. The direction of each projection is illustrated in the left-side of Figure~\ref{Figure2:IntraModesHEVC} by the directional modes 2 to 34. %In the figure it is possible to notice that directional modes 2 to 10 only project the pixels from the left neighbouring border in a horizontal direction, modes 10 to 25 use both the left and top neighbouring pixels in a diagonal direction and modes 26 to 34 project the top neighbours in vertical direction. -As shown in Figure~\ref{Figure2:IntraModesHEVC}, the angular modes are designed to provide a dense coverage near the horizontal (i.e., mode 10) and vertical (i.e., mode 26) directions. -Moreover, the angular directions are coarser as it gets closer to the diagonals. +As shown in Figure~\ref{Figure2:IntraModesHEVC}, the angular modes are designed to provide a dense coverage near the horizontal (i.e., mode 10) and vertical (i.e., mode 26) directions. +Moreover, the angular directions are coarser as it gets closer to the diagonals. This reflects the observed statistical prevalence of the angles and the prediction efficiency. To improve the efficiency of angular prediction, when the reference samples need to be projected, a bilinear interpolation is used from the two closest pixels using a $1/32$ accuracy. The dashed arrows on the right of Figure~\ref{Figure2:IntraModesHEVC} correspond to a projection of interpolated pixels. @@ -498,7 +494,7 @@ Another improvement in \ac{hevc} is related with the reference picture manageme Decoded frames in the \ac{dpb} are grouped in one of the following categories: (i) \emph{short-term reference}, (ii) \emph{long-term reference} and (iii) \emph{unused for reference}. Once a frame is marked as \emph{unused for reference} it is no longer used for motion compensation and it will be discarded from the \ac{dpb} after being displayed. In \ac{h264} a sliding window approach is used to implicitly manage the reference picture list. -This process discards a given frame whenever the \ac{dpb} has stored the maximum number of reference frames (which is defined and controlled by the encoder). +This process discards a given frame whenever the \ac{dpb} has stored the maximum number of reference frames (which is defined and controlled by the encoder). Moreover, \ac{h264} uses an explicitly \ac{mmco} to send commands to the decoder and change the normal reference picture management process. In the \ac{hevc} standard the status of the \ac{dpb} is encoded for every slice, instead of the implicit managing used in the past. @@ -553,8 +549,8 @@ These partitions types are only allowed for \acs{cu} with $16\times16$ pixels or \begin{figure}[tb] \centering \begin{tabular}{x{.8\columnwidth}x{.1\columnwidth}} - \input{\FigsPath/PUSymetric.tikz} & (a)\\ - \input{\FigsPath/PUAsymetric.tikz} & (b)\\ + \input{021_VideoCoding/Figures/PUSymetric.tikz} & (a)\\ + \input{021_VideoCoding/Figures/PUAsymetric.tikz} & (b)\\ \end{tabular} \caption{Partition types of an inter-coded CU into PUs; in (b) the size (and position between parentheses) corresponds to the smaller partition~\cite{HEVCBlock2012}.} \label{Figure2:InterPuPartitions} @@ -576,8 +572,8 @@ For B-slices, additional \ac{mv} candidates are considered by choosing two exist \begin{figure}[tb] \centering - \input{\FigsPath/MergeModeHEVC.tikz} - %\input{\FigsPath/HEVC/MergeModeHEVCTwoFrames.tikz} + \input{021_VideoCoding/Figures/MergeModeHEVC.tikz} + %\input{021_VideoCoding/Figures/HEVC/MergeModeHEVCTwoFrames.tikz} \caption{Illustration of the MV prediction candidates in the AMVP and Merge Mode mode.} \label{Figure2:MVCandidates} \end{figure} @@ -626,7 +622,7 @@ Simulation studies have shown that \ac{dst} provides up to 1\% of bitrate savin The \ac{hevc} follows the same principle as previous standards in terms of quantisation. After applying the transform operation to the prediction residue, a quantisation is applied to the resulting coefficients using a uniform-reconstruction quantiser scheme based on a \ac{qp}. -The standard supports \ac{qp} values ranging from 0 to 51, where an increase of 6 in the \ac{qp} corresponds to doubling the quantisation step. +The standard supports \ac{qp} values ranging from 0 to 51, where an increase of 6 in the \ac{qp} corresponds to doubling the quantisation step. This procedure is the main cause for the coding distortion, as it performs a many-to-one mapping, which cannot be reversed. @@ -659,8 +655,8 @@ Such grid-based restriction reduces the computational complexity and facilitates Three strength levels can be used by the \ac{dbf}, depending on the coded block characteristics. The \ac{sao} is a non-linear filter, that is adaptively applied to all samples of the image, after the deblocking filter. In its operation, the \ac{sao} filter modifies the samples -by adding an offset value, extracted from lookup tables transmitted by the encoder. -For each \ac{ctu}, the encoder decides whether to apply the \ac{sao} filter or not, and if used, one of the two filter types is applied, namely the band offset or the edge offset. +by adding an offset value, extracted from lookup tables transmitted by the encoder. +For each \ac{ctu}, the encoder decides whether to apply the \ac{sao} filter or not, and if used, one of the two filter types is applied, namely the band offset or the edge offset. In the band offset mode, the added offset value depends on the sample amplitude, while the edge offset uses the gradient to classify and derive the offset value. diff --git a/031_ErrorResilienceEvaluation/ErrorResilienceEvaluation.tex b/031_ErrorResilienceEvaluation/ErrorResilienceEvaluation.tex index d139a11..6bdd316 100644 --- a/031_ErrorResilienceEvaluation/ErrorResilienceEvaluation.tex +++ b/031_ErrorResilienceEvaluation/ErrorResilienceEvaluation.tex @@ -1,8 +1,5 @@ % Figure folders -\newcommand{\HEVCRobustnessEvaluation}{./031_ErrorResilienceEvaluation/QualityEvaluation} -\newcommand{\DynamicTMVP}{031_ErrorResilienceEvaluation/DynamicTmvp} - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % CHAPTER %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -16,8 +13,8 @@ \section{Introduction} The use of \ac{hevc} in different type of application and video services over networks that are prone to errors and packet losses, requires mechanisms to minimise the effect of such probabilistic events. This chapter starts by presenting an evaluation study about the error resilience of \ac{hevc} in Section~\ref{Section3:ErrorResilienceEvaluation}, with the aim of defining the weak elements of coded streams and the level of error robustness under different transmission loss conditions. The evaluation study covers different coding configuration to evaluate the individual impact of some new coding tools introduced in this standard, e.g., increased partition size, intra-refreshing, slice partitioning and advanced motion vector prediction. - -Then, in Section~\ref{Section3:DynamicMVRefreshHEVC}, based on the results of the simulation study, a novel method to increase the error robustness of the \ac{mv} coding is presented and compared to existing techniques. + +Then, in Section~\ref{Section3:DynamicMVRefreshHEVC}, based on the results of the simulation study, a novel method to increase the error robustness of the \ac{mv} coding is presented and compared to existing techniques. The proposed method introduces refresh points in the motion prediction used in the \ac{hevc}, which significantly reduces the impact of mismatch \ac{mv} predictions. Finally, Section~\ref{Section3:Summary} concludes the chapter. @@ -45,7 +42,7 @@ This is the main focus of the study presented in this section. There are some studies presented in the past regarding the error robustness of the \ac{hevc} standard, which are worth to be mentioned as they provide relevant insights for the research carried out in this work. In~\cite{StudyHEVCErrorsDez2012,HEVCWireless_2016} the coding efficiency and error robustness of the \ac{hevc} are evaluated and compared against the \ac{h264}. Experimental results confirm that \ac{hevc} has reduced error robustness despite its increased coding efficiency. A subjective evaluation study is presented in~\cite{SubjectiveEvalSep2013,QoEHEVCMay2014} revealing that packet loss ratios higher than $3\%$ have significant effect on the perceived video quality. -The vulnerability of \ac{mv} prediction is investigated and its impact on the error robustness is evaluated in \cite{DynamicTMVP2011}. +The vulnerability of \ac{mv} prediction is investigated and its impact on the error robustness is evaluated in \cite{DynamicTMVP2011}. Moreover, in the same study, a method to introduce \ac{mv} refresh points is proposed, achieving average \ac{psnr} gains of approximately 3~dB. Summarising, the decrease in error robustness is mostly due to the strong data dependencies imposed by the highly complex prediction modes of \ac{hevc}. Since these are selected according to a rate distortion optimisation criterion that assumes error-free transmission, the decoder faces different problems whenever errors or data loss occur in the transmission network \cite{ROPE2007}. @@ -68,7 +65,7 @@ Considering a test sequence with $N$ frames $f(t)$ the pixel values (Luma compon \begin{itemize} \item {\ac{si}} based on Sobel filter. Each video frame is firstly filtered, then the standard deviation ($std$) of the pixel values is computed. Finally, the maximum value over time ($\maxop_{t\in [1..N]}$), is chosen to represent the \ac{si} metric. This process can be represented by the following equation: \begin{equation} -% SI = max_{time}\Big\{ std \big( Sobel\left(f_t\right) \big) \Big\}. +% SI = max_{time}\Big\{ std \big( Sobel\left(f_t\right) \big) \Big\}. SI = \maxop_{t\in [1..N]} \Big\{ std \big( sobel\left(f\left(t\right)\right) \big) \Big\}. \end{equation} \item {\ac{ti}} based upon the motion difference, which is given by the difference between the pixel values at the same spatial location but in consecutive time instances. \ac{ti} is computed as the maximum displacement over time of the standard deviation over space. Larger displacements in adjacent frames will result in higher values of \ac{ti}. This process can be represented by the following: @@ -77,7 +74,7 @@ Considering a test sequence with $N$ frames $f(t)$ the pixel values (Luma compon TI = \maxop_{t\in [2..N]} \Big\{ std \big( f\left(t\right) - f\left(t-1\right) \big) \Big\}. \end{equation} \end{itemize} -The values of \ac{si} and \ac{ti} calculated for the test sequences are shown in the Table~\ref{TableTestSequences} (second column). +The values of \ac{si} and \ac{ti} calculated for the test sequences are shown in the Table~\ref{TableTestSequences} (second column). The percentage of inter-coded blocks and the bitrates are also shown in the table and explained below. \input{031_ErrorResilienceEvaluation/TablesSequences.tex} @@ -107,7 +104,7 @@ Moreover, the ratio of inter-coded \acs{cu} of each coded sequence using the def To simulate the lossy transmission environment, each frame is divided into several slices and each slice encoded and packetized into one \ac{nal} unit, which are then transmitted as the payload of independent packets. In these simulations a whole packet is lost whenever an error occurs, originating a lost slice. -\HIGHLIGHT{Random packet loss was simulated using a two-state Markov model, derived from the classical Gilbert-Elliot \cite{GilbertModel_1960, ElliotModel_1963}, which can be used to describe error patterns in real time video applications~\cite{GilbertElliotModelPlr_2008}. In this study a simplified version of the Gilbert-Elliot model was used by only defining the loss probability without control over burst loss events. +\HIGHLIGHT{Random packet loss was simulated using a two-state Markov model, derived from the classical Gilbert-Elliot \cite{GilbertModel_1960, ElliotModel_1963}, which can be used to describe error patterns in real time video applications~\cite{GilbertElliotModelPlr_2008}. In this study a simplified version of the Gilbert-Elliot model was used by only defining the loss probability without control over burst loss events. The application of this model results in loss patterns with approximately the same average burst loss as the ones provided in \cite{JCTVCLossSim_2012}. For each test condition, 50 runs were performed and the average \ac{psnr} was measured. Four different \acs{plr} are used resulting in the characteristics presented in Table~\ref{TAB3:LossPatterns}.} The remaining configurations have been kept at their default values, as given in the common test conditions~\cite{HEVCCommonTests2012}. @@ -130,11 +127,11 @@ In the first set of experiments the impact of the maximum \ac{cu} size in the de Table~\ref{Table3:Quality_BlockSize} shows the average \ac{psnr} (dB) obtained for the three test conditions and for three different packet loss ratios. Results for the error-free scenario shows a consistent quality increase as the maximum \ac{cu} size increases, % up to a maximum of 1.41~dB for the Tennis sequence when comparing the maximum \ac{cu} size of $64\times64$ with $16\times16$. -This confirms the principle used by the \ac{hevc} standard, whereby larger coding units lead to higher coding efficiency. -For the Tennis sequence, which has the highest \ac{ti} parameter (see Table~\ref{TableTestSequences}), a maximum of 1.41~dB is achieved. -Moreover, it is also noticeable that sequences with higher amount of inter-coded blocks take less advantage of the larger \acs{cu}. +This confirms the principle used by the \ac{hevc} standard, whereby larger coding units lead to higher coding efficiency. +For the Tennis sequence, which has the highest \ac{ti} parameter (see Table~\ref{TableTestSequences}), a maximum of 1.41~dB is achieved. +Moreover, it is also noticeable that sequences with higher amount of inter-coded blocks take less advantage of the larger \acs{cu}. This is noticeable in the results of the BQSquare sequence (i.e., highest ratio of inter-coded blocks and lowest quality gain in the error-free case). -Overall, comparing the results for the error-free and data loss cases, it is noticeable that similar quality gains are obtained, mostly due to the superior coding efficiency of larger \acs{cu}. +Overall, comparing the results for the error-free and data loss cases, it is noticeable that similar quality gains are obtained, mostly due to the superior coding efficiency of larger \acs{cu}. In most cases the quality gains slightly decrease for increasing \acs{plr}. Therefore, one may conclude that the error robustness of \ac{hevc} streams is not affected by limiting the maximum \ac{cu} size. @@ -144,8 +141,8 @@ Therefore, one may conclude that the error robustness of \ac{hevc} streams is no % \resizebox{\columnwidth}{!}{ \begin{tabular}{y{.45\columnwidth}y{.45\columnwidth}} \small - \includegraphics[width=\FIGSIZE]{\HEVCRobustnessEvaluation/Bask_Partitions_16.png} & - \includegraphics[width=\FIGSIZE]{\HEVCRobustnessEvaluation/Bask_Partitions_64.png} \\ + \includegraphics[width=\FIGSIZE]{./031_ErrorResilienceEvaluation/QualityEvaluation/Bask_Partitions_16.png} & + \includegraphics[width=\FIGSIZE]{./031_ErrorResilienceEvaluation/QualityEvaluation/Bask_Partitions_64.png} \\ (a) $16\times16$ & (b) $64\times64$ \\ \end{tabular} % } @@ -154,7 +151,7 @@ Therefore, one may conclude that the error robustness of \ac{hevc} streams is no \end{figure} Figure~\ref{Figure3:BlockPartitionBask} illustrates an example of the frame partitions for a region of the Basketball Drill sequence for two cases: (i) maximum \ac{cu} size of $64\times64$ and (ii) maximum \ac{cu} size of $16\times16$. -As shown, in the moving regions, e.g., moving player, the partitions obtained for both cases have a similar pattern, mainly composed of \acs{cu} with small size. Larger \acs{cu} are only used in the floor, which has a lower spatial complexity and suffers small changes between frames, therefore, easier to compress with $64\times64$ \acs{cu}. +As shown, in the moving regions, e.g., moving player, the partitions obtained for both cases have a similar pattern, mainly composed of \acs{cu} with small size. Larger \acs{cu} are only used in the floor, which has a lower spatial complexity and suffers small changes between frames, therefore, easier to compress with $64\times64$ \acs{cu}. Moreover, those regions can be more accurately recovered, by using neighbouring information correctly decoded (spatial and temporal), thus not incurring in severe error propagation. Therefore, the overall error robustness is not affected by reducing the maximum \ac{cu} size. One may conclude that the use of larger coding units is able to improve the overall \ac{hevc} efficiency in error-free and error prone communications. @@ -178,14 +175,14 @@ This allows the decoder to recover from frame/slice loss and continue error-free Figure~\ref{Figure3:QualityvsIntra} shows the average decoded video quality (\ac{psnr}) for different distances of intra frames, covering \acs{plr} between 3\% and 5\%. Results in the figure indicate that as the distance between intra frames increases, the decoded video quality significantly decreases. The quality decreasing is especially noticeable for intra periods higher than 32 frames, indicating that temporal distances greater than approximately one second are not suitable for video streaming under error-prone conditions. -When a high temporal distance between intra frames is used, the quality decreasing is more noticeable in the People On Street and BQSquare sequences, which have higher spatial complexity (see values for \ac{si}), leading to a reduction of 13 dB and 14 dB, respectively. +When a high temporal distance between intra frames is used, the quality decreasing is more noticeable in the People On Street and BQSquare sequences, which have higher spatial complexity (see values for \ac{si}), leading to a reduction of 13 dB and 14 dB, respectively. As spatial complexity increases, the prediction residue energy also increases, which leads to more erroneous information in case of data loss, reducing the video quality under errors. One should also note from the results of Figure~\ref{Figure3:QualityvsIntra} that, using a small distance between intra frames is not always more efficient. Since intra frames require higher amount of bits, the overall coding efficiency is reduced, resulting in lower overall quality in case of data loss, when comparing with other cases. This is clearly noticeable for the Four People sequence, where replacing the intra period from 8 to 16 frames leads to a quality increase of 1 dB for 5\% of \ac{plr}. -For the remaining sequences it can be concluded that using sixteen frames (not higher than half a second) between refresh points (i.e., intra frames) yields an acceptable efficiency under errors. -This provides a good trade-off between compression efficiency and error robustness in case of data loss. +For the remaining sequences it can be concluded that using sixteen frames (not higher than half a second) between refresh points (i.e., intra frames) yields an acceptable efficiency under errors. +This provides a good trade-off between compression efficiency and error robustness in case of data loss. For this reason for the remaining experimental evaluation an \ac{idr} of 16 frame will be used. @@ -207,16 +204,16 @@ Figure~\ref{Figure3:SlicePartitioning} shows the average \ac{psnr} for four diff The first three modes are characterised by a different amount of bytes per slice (B/S), namely 1200, 2400 and 4800 bytes. In these cases each frame has a variable number of slices. The last mode uses a single slice per frame (referred to as \textit{SS}), corresponding to the case where slice partitioning is disabled. The results shown in Figure~\ref{Figure3:SlicePartitioning} reveal a clear impact of the slice partitioning mode on the error robustness of \ac{hevc}, as the quality degradation for increasing \ac{plr} changes significantly for each mode. -Comparing the cases with fixed amount of bytes per slice, it is clearly noticeable that a higher amount of bytes also corresponds to a higher quality degradation. +Comparing the cases with fixed amount of bytes per slice, it is clearly noticeable that a higher amount of bytes also corresponds to a higher quality degradation. This is true for the results of most test sequences shown in Figure~\ref{Figure3:SlicePartitioning}. Although the use of smaller slices increases the overhead required for slice headers, the superior error robustness achieved with 1200 B/S is able to overcome the inherent degradation on coding efficiency. This is due to the lack of refresh points within a frame, resulting in errors that affect larger regions of the frame, which are propagated to the subsequent inter predicted frames. -The results for of \textit{SS} (slice partitioning disabled) reveals that this approach also leads to lower error robustness when comparing with those cases where the frames are divided into multiple slices. +The results for of \textit{SS} (slice partitioning disabled) reveals that this approach also leads to lower error robustness when comparing with those cases where the frames are divided into multiple slices. Although higher coding efficiency is achieved in this mode, the lower error robustness leads to an overall poor performance under frame or slice loss conditions. -It is worthwhile to notice that the results for Kendo sequence do not follow the main trend. -This can be explained by the low bitrate required to compress the video signal, which is the lowest when compared with the other test sequences (see Table~\ref{TableTestSequences}). -Therefore, most of the inter-coded frames can be transmitted using a single slice, even with 1200 bytes per slice, resulting in a similar partitioning for all tested modes, and thus similar error robustness. +It is worthwhile to notice that the results for Kendo sequence do not follow the main trend. +This can be explained by the low bitrate required to compress the video signal, which is the lowest when compared with the other test sequences (see Table~\ref{TableTestSequences}). +Therefore, most of the inter-coded frames can be transmitted using a single slice, even with 1200 bytes per slice, resulting in a similar partitioning for all tested modes, and thus similar error robustness. In this case, the higher coding efficiency achieved by the \textit{SS} case also leads to higher quality in case of errors. @@ -272,7 +269,7 @@ Moreover, the slice partitioning mode used only 1200 bytes per slice, which intr \begin{figure}[!t] \centering \renewcommand{\FIGSIZE}{0.76\textwidth} - % \includegraphics[width=\FIGSIZE,page=2]{\HEVCRobustnessEvaluation/PSNRvsFrameSingleFrameTMVP} + % \includegraphics[width=\FIGSIZE,page=2]{./031_ErrorResilienceEvaluation/QualityEvaluation/PSNRvsFrameSingleFrameTMVP} \input{./031_ErrorResilienceEvaluation/QualityEvaluation/PSNRvsFrameSingleFrameTMVP} \caption{Error propagation for Kendo sequence when Frame~\#6 is lost for the \ac{ld} configuration using different TMVP configurations.} \label{Figure3:PSNRvsFrameErrorFreeConc} @@ -281,9 +278,9 @@ Moreover, the slice partitioning mode used only 1200 bytes per slice, which intr The second set of experiments aimed to find the influence of losing temporal motion predictors on the error resilience of \ac{hevc}. In order to perform this test, a single frame loss in enforced, and the error propagation is evaluated for two configurations: (i) with the temporal \ac{mv} candidate enabled and (ii) disabled. -Figure~\ref{Figure3:PSNRvsFrameErrorFreeConc} illustrates the error propagation, when a single frame is lost. For this particular experiment an intra-refreshing period of 32 frames was used to evaluate the impact on long error propagation periods. This is commonly used in quality evaluation of erroneous video decoding and follows the reference conditions~\cite{HEVCCommonTests2012}. +Figure~\ref{Figure3:PSNRvsFrameErrorFreeConc} illustrates the error propagation, when a single frame is lost. For this particular experiment an intra-refreshing period of 32 frames was used to evaluate the impact on long error propagation periods. This is commonly used in quality evaluation of erroneous video decoding and follows the reference conditions~\cite{HEVCCommonTests2012}. The results show that when the \ac{tmvp} mode is enabled, the loss of Frame \#6 leads to significant reduction in the reconstruction quality of the subsequent frames. -On the contrary, when the temporal \ac{mv} candidate is disabled, the motion information on subsequent frames is not affected by the loss of a single frame and significantly higher quality is obtained. +On the contrary, when the temporal \ac{mv} candidate is disabled, the motion information on subsequent frames is not affected by the loss of a single frame and significantly higher quality is obtained. This is the effect of breaking the temporal dependencies between \acs{mv}, which prevents erroneous \acs{mv} to propagate throughout the set of dependent frames. \begin{table}[!t] @@ -338,7 +335,7 @@ The objective is to increase the number of \acs{mv} that can be independently de The proposed method also overcomes the limitation of the method presented in \cite{DynamicTMVP2011}, which breaks the motion information dependencies between frames. Instead of disabling the temporal motion candidates at the frame level, the proposed method aims to remove the dependencies at the \ac{cu} level. To achieve this goal, the temporal dependency of all \ac{mv} candidates is firstly analysed for each block, in order to select the suitable candidates to be used as predictors for the current \ac{mv}. -Based on the analysis of the temporal dependency, some temporal candidates are marked as unsuitable predictors of the current \ac{mv}, and for this reason they are not selected for \ac{mv} prediction. +Based on the analysis of the temporal dependency, some temporal candidates are marked as unsuitable predictors of the current \ac{mv}, and for this reason they are not selected for \ac{mv} prediction. In the proposed approach, a temporal \ac{mv} candidate is marked as unsuitable, based on the following criteria: \begin{enumerate} \item if it is encoded based on another temporal \ac{mv} candidate from a previously encoded frame; @@ -347,7 +344,7 @@ In the proposed approach, a temporal \ac{mv} candidate is marked as unsuitable, \begin{figure}[t] \centering - % \includegraphics[width=0.6\columnwidth]{\DynamicTMVP/DynamicTMVP} + % \includegraphics[width=0.6\columnwidth]{./031_ErrorResilienceEvaluation/DynamicTmvp/DynamicTMVP} \input{./031_ErrorResilienceEvaluation/DynamicTmvp/DynamicTMVP.tikz} \caption{Proposed approach to constrain the temporal \ac{mv} dependencies (arrow point to the MV predictor).} \label{ProposedDynamicMethod} @@ -356,8 +353,8 @@ In the proposed approach, a temporal \ac{mv} candidate is marked as unsuitable, Figure~\ref{ProposedDynamicMethod} can be used to explain the proposed method, illustrating different blocks and different \ac{mv} dependencies represented with arrows. Note that the squares illustrate a generic block size; the representation of the temporal dependencies is simplified, and in some cases may not correspond to the exact co-located block in the previous frame. In frame $f_{t-1}$ there are several \acs{mv} encoded using spatial predictions and one \ac{mv} encoded using a temporal prediction (arrow pointing to $f_{t-2}$). In frame $f_{t}$, the \ac{mv} corresponding to the block (1) may use the temporal \ac{mv} candidate, since the co-located block in $f_{t-1}$ is not temporally dependent. However, in the proposed scheme the \ac{mv} prediction identified with the arrow (2) is not allowed, which prevents the propagation of temporal dependencies. -Moreover, the block corresponding to the arrow (3) cannot use the temporal \ac{mv} candidate since it already depends on a \ac{mv} that was previously encoded using the temporal candidate. The proposed method reduces the temporal dependencies of the \glspl{mv} at the block level, providing improved error robustness to the encoded streams. -Using this approach, the dependencies are selectively removed, improving the error recovery accuracy, without fully disabling the temporal \gls{mv} candidates for a given frame. +Moreover, the block corresponding to the arrow (3) cannot use the temporal \ac{mv} candidate since it already depends on a \ac{mv} that was previously encoded using the temporal candidate. The proposed method reduces the temporal dependencies of the \glspl{mv} at the block level, providing improved error robustness to the encoded streams. +Using this approach, the dependencies are selectively removed, improving the error recovery accuracy, without fully disabling the temporal \gls{mv} candidates for a given frame. Therefore, coding efficiency is not significantly penalised. @@ -398,7 +395,7 @@ Secondly, further simulations were performed to evaluate the effectiveness of th %As in previous experiments, each coded frame is transmitted in one packet. Table~\ref{Table3:AverageStdPSNR} illustrates the average standard deviation of the PSNR, for three of the test sequences. The results show that increasing the \ac{plr} leads to higher quality variations, thus the impact of errors in the subjective quality experience is higher. -However, reducing the temporal dependencies of the \acs{mv} using the proposed method decreases such quality variations, achieving the lowest variation value for all tested scenarios. +However, reducing the temporal dependencies of the \acs{mv} using the proposed method decreases such quality variations, achieving the lowest variation value for all tested scenarios. The proposed method is able to reduce the quality variations up to 0.94 dB for Kendo sequence (3$\%$ of packet loss) compared to the~\TIDR~method. Therefore, since the proposed method achieves the lowest \ac{psnr} standard deviation, one may conclude that it is able to reduce the video quality variations leading to a better visual quality. @@ -410,7 +407,7 @@ Therefore, since the proposed method achieves the lowest \ac{psnr} standard dev \input{031_ErrorResilienceEvaluation/DynamicTmvp/TableDynamicTMVPPSNR.tex} \end{table} -Table~\ref{Table3:AverageResultsProposed} shows the Bjontegaard's average \ac{psnr} increase \cite{BJResults2001}. +Table~\ref{Table3:AverageResultsProposed} shows the Bjontegaard's average \ac{psnr} increase \cite{BJResults2001}. The results are obtained taking the standard \ac{hevc} encoder as reference. % The results are obtained taking as reference the standard \ac{hevc} encoder, in order to evaluate the quality difference when temporal \ac{mv} candidates are disabled in both error free and under error-prone conditions. The results presented in Table~\ref{Table3:AverageResultsProposed} show that the proposed method presents practically the same rate-distortion performance when compared to the \TIDR method. diff --git a/041_ImprovedErrorResilience/ImprovedErrorResilience.tex b/041_ImprovedErrorResilience/ImprovedErrorResilience.tex index 897cdbc..b4c14ca 100644 --- a/041_ImprovedErrorResilience/ImprovedErrorResilience.tex +++ b/041_ImprovedErrorResilience/ImprovedErrorResilience.tex @@ -1,11 +1,7 @@ % Figure folders -\newcommand{\ImprovedErrorResilience}{./041_ImprovedErrorResilience} -\graphicspath{{\ImprovedErrorResilience/Results/}} - +\graphicspath{{./041_ImprovedErrorResilience/Results/}} \chapter[{A two-stage approach for robust coding and streaming \VERSION{2}}]{A two-stage approach for robust\\coding and streaming \VERSION{2}} -% \chapter{Error Resilience Techniques for \ac{hevc} Standard} -%\chapter{Two-stage Robust Video Coding} \label{Chapter:ErrorResilienceTechniques} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -107,7 +103,7 @@ However, such solution would not take into consideration coding efficiency, sinc \begin{figure}[tb] \centering - \input{\ImprovedErrorResilience/Figures/RefPicSelection.tikz} + \input{./041_ImprovedErrorResilience/Figures/RefPicSelection.tikz} \caption{Example of the reference picture selection applied to the frame $C$ based on the checkerboard at \ac{ctu} level (\acs{pu} are represented by small partitions).} \label{fig:checkerboardRefSelection} \end{figure} @@ -146,7 +142,7 @@ Note that an exponential cost increase is used in (\ref{eq:Lagrangian}) in order \begin{figure}[t] \centering -\includegraphics[width=0.8\columnwidth]{\ImprovedErrorResilience/Figures/ExponentialWeight} +\includegraphics[width=0.8\columnwidth]{./041_ImprovedErrorResilience/Figures/ExponentialWeight} \caption{Exponential weight as a function of the global and local deviations} \label{fig:weightfct} \end{figure} @@ -211,7 +207,7 @@ Therefore, the image area covered by all dependent \acs{pu} is separately consi \begin{figure}[tb] \centering - \input{\ImprovedErrorResilience/Figures/MVDependencies.tikz} + \input{./041_ImprovedErrorResilience/Figures/MVDependencies.tikz} \caption{Example of temporal and spatial \ac{mv} dependencies in \ac{hevc}.} \label{fig:mvdependencies} \end{figure} @@ -307,7 +303,7 @@ Summarising, the reconstructed frames illustrated in Figure~\ref{fig:RecoveredEx \begin{figure}[t] \renewcommand{\FIGSIZE}{0.8\columnwidth} \centering -% \includegraphics[width=\FIGSIZE]{\ImprovedErrorResilience/Results/PSNRversusFrameKendo} +% \includegraphics[width=\FIGSIZE]{./041_ImprovedErrorResilience/Results/PSNRversusFrameKendo} \input{041_ImprovedErrorResilience/Results/PSNRversusFrameKendo.tex} \caption{Error propagation when errors affect Frame \#7 in Kendo sequence with an \ac{idr} period of 32 frames for the proposed reference frame selection algorithm in comparison with existing methods.} \label{fig:PSNRvsFrameForRefPicSel} @@ -344,8 +340,8 @@ Taking into account the above results and the spatio-temporal information shown \begin{figure}[tb] \renewcommand{\FIGSIZE}{0.8\columnwidth} \centering - %\includegraphics[width=\FIGSIZE,page=1]{\ImprovedErrorResilience/Results/PSNRversusGama} - %\includegraphics[width=\FIGSIZE,page=2]{\ImprovedErrorResilience/Results/PSNRversusGama} + %\includegraphics[width=\FIGSIZE,page=1]{./041_ImprovedErrorResilience/Results/PSNRversusGama} + %\includegraphics[width=\FIGSIZE,page=2]{./041_ImprovedErrorResilience/Results/PSNRversusGama} \input{./041_ImprovedErrorResilience/Results/PSNRversusGama.tex} \caption{Influence of the $\gamma$ parameter in the average quality gains, comparing the proposed method with \REF, for different \acs{plr}.} \label{fig:QualityVsGamma} @@ -448,12 +444,12 @@ Results also show that for both approaches the decoded video quality increases a \begin{figure}[!t] \centering \newcommand{\CurrFigSize}{0.49\columnwidth} - \includegraphics[width=\CurrFigSize,page=1]{\ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf} - \includegraphics[width=\CurrFigSize,page=2]{\ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf} - \includegraphics[width=\CurrFigSize,page=3]{\ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf} - \includegraphics[width=\CurrFigSize,page=4]{\ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf} - \includegraphics[width=\CurrFigSize,page=5]{\ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf} - \includegraphics[width=\CurrFigSize,page=6]{\ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf} + \includegraphics[width=\CurrFigSize,page=1]{./041_ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf} + \includegraphics[width=\CurrFigSize,page=2]{./041_ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf} + \includegraphics[width=\CurrFigSize,page=3]{./041_ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf} + \includegraphics[width=\CurrFigSize,page=4]{./041_ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf} + \includegraphics[width=\CurrFigSize,page=5]{./041_ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf} + \includegraphics[width=\CurrFigSize,page=6]{./041_ImprovedErrorResilience/Results/PSNRvsRedundancy.pdf} \caption{Decoded video quality for different amounts of redundancy, selected by using a fixed approach and the proposed dynamic method (\DYN).} \label{fig:QualityVsRedundancyMVSelection} \end{figure} @@ -472,7 +468,7 @@ For lower \acs{plr}, since fewer frames are affected by data losses, the use of \TABLESVSPACE \label{tab:AverageQualityRedundancies} \TABLESVSPACE - \input{\ImprovedErrorResilience/Results/TableResultsRandomLossMvRedundancies.tex} + \input{./041_ImprovedErrorResilience/Results/TableResultsRandomLossMvRedundancies.tex} \end{table} Table~\ref{tab:AverageQualityRedundancies} presents the average \ac{psnr} of decoded video for different \ac{plr} (columns~4~to~8) and for various percentages of \ac{mv} redundancy (third column). The absolute quality values are shown for the reference case (\NOREDUND) and \ac{psnr} gains are shown for three different levels of redundancy introduced by the proposed method (\DYN). diff --git a/042_ConcealmentAwareEncoding/ConcealmentAwareEncoding.tex b/042_ConcealmentAwareEncoding/ConcealmentAwareEncoding.tex index 7020e5f..0a1a891 100644 --- a/042_ConcealmentAwareEncoding/ConcealmentAwareEncoding.tex +++ b/042_ConcealmentAwareEncoding/ConcealmentAwareEncoding.tex @@ -1,6 +1,3 @@ -% Figure folders -\newcommand{\ConcealmentAwareDir}{./042_ConcealmentAwareEncoding} - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % CHAPTER %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -90,7 +87,7 @@ For the regions without \acs{mv} the average of the three top-left neighbouring \begin{figure}[tb] \centering - \input{\ConcealmentAwareDir/Figures/MotionVectorExtrapolation.tikz} + \input{./042_ConcealmentAwareEncoding/Figures/MotionVectorExtrapolation.tikz} \caption{Representation of the video frame with associated original and extrapolated motion vectors represented using full and dashed arrows, respectively.} \label{FIG5:MotionVectorExtrapolation} \end{figure} @@ -179,7 +176,7 @@ The smoothness term is a robust penalisation of the gradient of the flow, simila \begin{figure}[t] \centering - \includegraphics[width=.95\columnwidth]{\ConcealmentAwareDir/Figures/DeepFlowBlockDiagram} + \includegraphics[width=.95\columnwidth]{./042_ConcealmentAwareEncoding/Figures/DeepFlowBlockDiagram} \caption{Block diagram of the deep matching algorithm \cite{DeepFlow2013}.} \label{FIG5:DeepFlowDiagram} \FIGSVSPACE @@ -203,7 +200,7 @@ This can be explained by the objects' trajectories of objects which are preserve \caption{Average decoded video quality (\ac{psnr} in dB) for seven error concealment algorithms under 10\% of random packet loss.} \label{TAB5:ECPerformaceComparison} \TABLESVSPACE - \input{\ConcealmentAwareDir/Results/PerformanceConcealmentMethods.tex} + \input{./042_ConcealmentAwareEncoding/Results/PerformanceConcealmentMethods.tex} \end{table} @@ -244,7 +241,7 @@ This residual information contains the difference between the compressed ($\hat \begin{figure}[tb] \centering - %\includegraphics[width=.9\columnwidth]{\ConcealmentAwareDir/Figures/BlockDiagram} + %\includegraphics[width=.9\columnwidth]{./042_ConcealmentAwareEncoding/Figures/BlockDiagram} \input{042_ConcealmentAwareEncoding/Figures/BlockDiagram.tikz} \caption{Architecture of the error concealment-aware encoding scheme.} \label{FIG5:BlockDiagram} @@ -281,8 +278,8 @@ A set of four \ac{ec} candidate modes (M1...M4) is used in the optimisation pro \begin{figure}[tb] \centering -%\includegraphics[width=0.7\columnwidth]{\ConcealmentAwareDir/Figures/ConcealmentCandidates} - \input{\ConcealmentAwareDir/Figures/ConcealmentCandidates.tikz} +%\includegraphics[width=0.7\columnwidth]{./042_ConcealmentAwareEncoding/Figures/ConcealmentCandidates} + \input{./042_ConcealmentAwareEncoding/Figures/ConcealmentCandidates.tikz} \caption{\ac{ec} candidates used for optimization.} \label{FIG5:concealmentcandidates} \FIGSVSPACE @@ -376,8 +373,8 @@ Adaptive arithmetic coding~\cite{ArithmeticCodingJun1994} is used to efficiently \begin{figure}[tb] \centering - \includegraphics[width=0.8\columnwidth]{\ConcealmentAwareDir/Figures/ConcealmentSignalling} - % \input{\ConcealmentAwareDir/Figures/ConcealmentSignalling.tikz} + \includegraphics[width=0.8\columnwidth]{./042_ConcealmentAwareEncoding/Figures/ConcealmentSignalling} + % \input{./042_ConcealmentAwareEncoding/Figures/ConcealmentSignalling.tikz} \caption{Payload structure of the \ac{sei} \acs{nal}} % to signal the optimal error concealment method.} % Symbols transmitted through \ac{sei} messages to signal the optimal error concealment method.} \label{FIG5:signalling} diff --git a/042_ConcealmentAwareEncoding/Results/ECMQualitySingleloss.tex b/042_ConcealmentAwareEncoding/Results/ECMQualitySingleloss.tex index a5a5ff8..59bd600 100644 --- a/042_ConcealmentAwareEncoding/Results/ECMQualitySingleloss.tex +++ b/042_ConcealmentAwareEncoding/Results/ECMQualitySingleloss.tex @@ -1036,8 +1036,8 @@ e \newcommand{\figsize}{0.3\columnwidth} %\resizebox{\columnwidth}{!}{ \begin{tabular}{y{.4\columnwidth}y{.4\columnwidth}} -\includegraphics[width=\figsize]{\ConcealmentAwareDir/Figures/DecFrames/Bask_MVE_F6} & -\includegraphics[width=\figsize]{\ConcealmentAwareDir/Figures/DecFrames/Bask_PropSign_F6} \\ +\includegraphics[width=\figsize]{042_ConcealmentAwareEncoding/Figures/DecFrames/Bask_MVE_F6} & +\includegraphics[width=\figsize]{042_ConcealmentAwareEncoding/Figures/DecFrames/Bask_PropSign_F6} \\ \small \MC (28.15~dB) & \small \PROP (30.99~dB) \\ %[10pt] \end{tabular} %} diff --git a/043_EndToEndRobustSystem/EndToEndRobustSystem.tex b/043_DynamicErrorRobustness/DynamicErrorRobustness.tex similarity index 98% rename from 043_EndToEndRobustSystem/EndToEndRobustSystem.tex rename to 043_DynamicErrorRobustness/DynamicErrorRobustness.tex index c67b224..4f89dd0 100644 --- a/043_EndToEndRobustSystem/EndToEndRobustSystem.tex +++ b/043_DynamicErrorRobustness/DynamicErrorRobustness.tex @@ -59,7 +59,7 @@ The following methods were evaluated in the simulation study, namely: \end{itemize} These methods were selected because they are the main techniques proposed within this research work and cover two elements of a robust video communication chain: (i) robust video encoding by dynamically select the reference frames and (ii) robust video decoding by improving the \ac{ec} performance. -\input{043_EndToEndRobustSystem/Results/AdaptiveSequenceList.tex} +\input{043_DynamicErrorRobustness/Results/AdaptiveSequenceList.tex} Table~\ref{Table6:TestMaterialCharacteristics} presents the list of test sequences used. These sequences were selected to cover a wide range of video content, with different levels of spatial and temporal complexity, as demonstrated by the values of \ac{si} and \ac{ti}~\cite{ITUTP9102008}, respectively. @@ -77,7 +77,7 @@ Results on Figure~\ref{Figure6:AverageQualityReduction} were obtained by reducin \begin{figure}[!t] \centering - \input{043_EndToEndRobustSystem/Results/Quality_PLR_Methods.tex} + \input{043_DynamicErrorRobustness/Results/Quality_PLR_Methods.tex} \vspace{-4mm} \caption{Average decoded video quality for the bitstreams encoded with the main bitrate (bps).} \label{Figure6:AverageQualityMain} @@ -86,7 +86,7 @@ Results on Figure~\ref{Figure6:AverageQualityReduction} were obtained by reducin \begin{figure}[!t] \centering -\input{043_EndToEndRobustSystem/Results/Quality_PLR_Methods_Reduced.tex} +\input{043_DynamicErrorRobustness/Results/Quality_PLR_Methods_Reduced.tex} \vspace{-4mm} \caption{Average decoded video quality using a 40\% reduction of the main bitrate (bps).} \label{Figure6:AverageQualityReduction} @@ -116,8 +116,7 @@ Figure~\ref{Figure6:BlockDiagram} illustrates the proposed architecture, whose m \begin{figure}[tb] \centering - %\includegraphics[width=.9\columnwidth]{\ConcealmentAwareDir/Figures/BlockDiagram} - \input{043_EndToEndRobustSystem/Figures/JointlyRobustnessDiagram.tikz} + \input{043_DynamicErrorRobustness/Figures/JointlyRobustnessDiagram.tikz} \caption{Adaptive error robustness architecture for 4 streams (\SMEC...\SMEXPELTWEN).} \label{Figure6:BlockDiagram} \FIGSVSPACE @@ -251,8 +250,7 @@ In the proposed method a feed-forward network is used, which connects every neur \begin{figure}[tb] \centering - %\includegraphics[width=.9\columnwidth]{\ConcealmentAwareDir/Figures/BlockDiagram} - \input{043_EndToEndRobustSystem/Figures/NeuralNetwork.tikz} + \input{043_DynamicErrorRobustness/Figures/NeuralNetwork.tikz} \caption{Representation of the neural network used in the proposed method.} \label{FIG6:NeuralNetwork} \FIGSVSPACE @@ -370,7 +368,7 @@ The quality obtained with the proposed optimal error robustness scheme (\PROP) i \begin{figure}[!t] \centering - \input{043_EndToEndRobustSystem/Results/PlotQualty_vs_PLR_Predicted.tex} + \input{043_DynamicErrorRobustness/Results/PlotQualty_vs_PLR_Predicted.tex} \vspace{-2mm} \caption{Average video quality using different error resilience techniques.} \label{Figure6:ProposedQuality} diff --git a/043_EndToEndRobustSystem/Figures/JointlyRobustnessDiagram.tikz b/043_DynamicErrorRobustness/Figures/JointlyRobustnessDiagram.tikz similarity index 100% rename from 043_EndToEndRobustSystem/Figures/JointlyRobustnessDiagram.tikz rename to 043_DynamicErrorRobustness/Figures/JointlyRobustnessDiagram.tikz diff --git a/043_EndToEndRobustSystem/Figures/NeuralNetwork.tikz b/043_DynamicErrorRobustness/Figures/NeuralNetwork.tikz similarity index 100% rename from 043_EndToEndRobustSystem/Figures/NeuralNetwork.tikz rename to 043_DynamicErrorRobustness/Figures/NeuralNetwork.tikz diff --git a/043_EndToEndRobustSystem/Results/AdaptiveSequenceList.tex b/043_DynamicErrorRobustness/Results/AdaptiveSequenceList.tex similarity index 100% rename from 043_EndToEndRobustSystem/Results/AdaptiveSequenceList.tex rename to 043_DynamicErrorRobustness/Results/AdaptiveSequenceList.tex diff --git a/043_EndToEndRobustSystem/Results/PlotQualty_vs_PLR_Predicted.bash b/043_DynamicErrorRobustness/Results/PlotQualty_vs_PLR_Predicted.bash similarity index 100% rename from 043_EndToEndRobustSystem/Results/PlotQualty_vs_PLR_Predicted.bash rename to 043_DynamicErrorRobustness/Results/PlotQualty_vs_PLR_Predicted.bash diff --git a/043_EndToEndRobustSystem/Results/PlotQualty_vs_PLR_Predicted.tex b/043_DynamicErrorRobustness/Results/PlotQualty_vs_PLR_Predicted.tex similarity index 100% rename from 043_EndToEndRobustSystem/Results/PlotQualty_vs_PLR_Predicted.tex rename to 043_DynamicErrorRobustness/Results/PlotQualty_vs_PLR_Predicted.tex diff --git a/043_EndToEndRobustSystem/Results/Quality_PLR_Methods.bash b/043_DynamicErrorRobustness/Results/Quality_PLR_Methods.bash similarity index 100% rename from 043_EndToEndRobustSystem/Results/Quality_PLR_Methods.bash rename to 043_DynamicErrorRobustness/Results/Quality_PLR_Methods.bash diff --git a/043_EndToEndRobustSystem/Results/Quality_PLR_Methods.pdf b/043_DynamicErrorRobustness/Results/Quality_PLR_Methods.pdf similarity index 100% rename from 043_EndToEndRobustSystem/Results/Quality_PLR_Methods.pdf rename to 043_DynamicErrorRobustness/Results/Quality_PLR_Methods.pdf diff --git a/043_EndToEndRobustSystem/Results/Quality_PLR_Methods.tex b/043_DynamicErrorRobustness/Results/Quality_PLR_Methods.tex similarity index 100% rename from 043_EndToEndRobustSystem/Results/Quality_PLR_Methods.tex rename to 043_DynamicErrorRobustness/Results/Quality_PLR_Methods.tex diff --git a/043_EndToEndRobustSystem/Results/Quality_PLR_Methods_Reduced.tex b/043_DynamicErrorRobustness/Results/Quality_PLR_Methods_Reduced.tex similarity index 100% rename from 043_EndToEndRobustSystem/Results/Quality_PLR_Methods_Reduced.tex rename to 043_DynamicErrorRobustness/Results/Quality_PLR_Methods_Reduced.tex diff --git a/043_EndToEndRobustSystem/Results/cfgData.py b/043_DynamicErrorRobustness/Results/cfgData.py similarity index 100% rename from 043_EndToEndRobustSystem/Results/cfgData.py rename to 043_DynamicErrorRobustness/Results/cfgData.py diff --git a/043_EndToEndRobustSystem/Results/summary_decoding b/043_DynamicErrorRobustness/Results/summary_decoding similarity index 100% rename from 043_EndToEndRobustSystem/Results/summary_decoding rename to 043_DynamicErrorRobustness/Results/summary_decoding diff --git a/carreira_phd_thesis.sty b/carreira_phd_thesis.sty index bf4a49c..a56911d 100644 --- a/carreira_phd_thesis.sty +++ b/carreira_phd_thesis.sty @@ -118,7 +118,7 @@ \usepackage{balance} % *** ABREVIATIONS *** -\usepackage[style=long,toc,xindy,acronym,nomain,indexonlyfirst,nopostdot]{glossaries} +\usepackage[xindy,toc,acronym,nomain,indexonlyfirst,nopostdot]{glossaries} \newcommand{\customnewacronym}[4][]{\newacronym[#1,description=#4]{#2}{\mbox{#3}}{#4}} \setglossarystyle{long} \newcommand{\ac}[1]{\gls{#1}} diff --git a/carreira_phd_thesis.tex b/carreira_phd_thesis.tex index a07cc3b..d0f0a1f 100644 --- a/carreira_phd_thesis.tex +++ b/carreira_phd_thesis.tex @@ -68,12 +68,12 @@ \listoftables \printglossary[title=List of Abbreviations] \glsresetall -\glsunset{itut} -\glsunset{iso} -\glsunset{iec} -\glsunset{rtp} -\glsunset{ts} -\glsunset{dash} +% \glsunset{itut} +% \glsunset{iso} +% \glsunset{iec} +% \glsunset{rtp} +% \glsunset{ts} +% \glsunset{dash} \mainmatter \include{010_Introduction/Introduction} @@ -82,7 +82,7 @@ \include{031_ErrorResilienceEvaluation/ErrorResilienceEvaluation} \include{041_ImprovedErrorResilience/ImprovedErrorResilience} \include{042_ConcealmentAwareEncoding/ConcealmentAwareEncoding} -\include{043_EndToEndRobustSystem/EndToEndRobustSystem} +\include{043_DynamicErrorRobustness/DynamicErrorRobustness} \include{050_Conclusions/Conclusion} \appendix \include{060_Appendix/Sequences}