jfmcarreira/doc_phd_thesis
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Review thesis after PA comments

Browse Source
This commit is contained in:
jfmcarreira
2018-09-17 23:11:37 +01:00
parent 2a97ab6c09
commit 9bdc2ee860
24 changed files with 798 additions and 749 deletions
Download Patch File Download Diff File Expand all files Collapse all files
021_VideoCoding/VideoCodingReview.tex
+86 -75
View File
@@ -3,7 +3,7 @@
\renewcommand{\FigsPath}{./021_VideoCoding/Figures}
% \chapter{\HIGHLIGHT{The \ac{hevc} Standard}}
\chapter{The High Efficiency Video Coding Standard \VERSION{2}}
\chapter{The High Efficiency Video Coding standard \VERSION{3}}
%\chapter{\HIGHLIGHT{The State-of-Art Video Coding}}
\label{Chapter:HEVCReview}
@@ -15,7 +15,7 @@
\glsunset{h264}
\glsreset{hevc}
This chapter describes the H.265~-~\ac{hevc}~\cite{H265StandardFeb2018} standard, focusing on the most relevant features of the encoding algorithm, in the scope of the work developed in this thesis.
This chapter presents an overview of the H.265~-~\ac{hevc}~\cite{H265StandardFeb2018} standard, focusing on the most relevant features of the encoding algorithm, considering the scope of the work developed in this thesis.
In short, the \ac{hevc} standard essentially aims to support all existing applications of \ac{h264}~\cite{H264AVCStandard2003}, while extending two keys aspects: increased video resolution with higher compression ratio and also more efficient use of parallel processing architectures.
In order to increase the coding flexibility and higher video resolutions, the \ac{hevc} standard adopts a new block partition structure, as well as, new prediction modes.
In this chapter the new data structures and coding techniques, specifically introduced to increase the coding efficiency, are described.
@@ -23,7 +23,7 @@ Particular emphasis is given to those which may have a significant impact in the
\glsunset{hevc}
This chapter is organised as follows.
The chapter is organised as follows.
Section~\ref{Section2:GeneraVideoSystem} introduces the functional blocks of a generic hybrid video coding system and Section~\ref{Section2:Overview} provides an overview of the \ac{hevc} standard.
Then, Section~\ref{Section2:HighLevel} describes the high-level syntax elements, which are used to carry relevant information in coded streams.
Section~\ref{Section2:CodingStructures} describes the data structures used to divide the video frames into smaller partitions and Section~\ref{Section2:PredictionModes} describes particular aspects of the main predictions techniques, namely intra and inter coding.
@@ -38,10 +38,10 @@ Section~\ref{Section2:Summary} concludes the chapter.
\label{Section2:GeneraVideoSystem}
A digital video signal is a temporal succession of images, also referred to as pictures or frames, which usually presents a significant amount of temporal and spatial correlations.
In general, video coding and distribution systems follow through a generic chain of processing and delivery stages: pre-processing, encoding, storage or transmission, decoding and post-processing.
In general, video coding and distribution systems follow through a generic chain of processing and delivery stage, comprising pre-processing, encoding, storage or transmission, decoding and post-processing, as shown in Figure~\ref{Figure2:HybridCodec}.
The pre-processing stage is used adapt the raw video signal, either analogue or digital, to an appropriate format, such as RGB or YUV~\cite{DataCompression1997}.
Then, the video data is compressed by the respective encoding module, mostly reducing its redundancy and visual irrelevancy. The encoding process is responsible for representing the raw video in a highly compressed format, while maintaining a good trade-off between video quality and bitrate through high efficient algorithms and coding tools.
In the last stages of the chain, the compressed format is decoded and presented by displaying the signal content after adaptation to the consumer equipment through post-processing.
In the last stages of the chain, the compressed format is decoded and presented by displaying the signal content after adaptation to the consumer equipment through post-processing, when necessary.
% \begin{figure}[tb]
% \centering
@@ -53,8 +53,8 @@ In the last stages of the chain, the compressed format is decoded and presented
Since the early days of video compressing, hybrid video coding has been the basic structure for all video coding standards and recommendations of ITU-T \ac{vceg} and \ac{iso}/\ac{iec} \ac{mpeg}, e.g., from H.261 to \ac{hevc}.
Figure~\ref{Figure2:HybridCodec} shows the block diagram of a generic hybrid video coding system~\cite{BookLucas2017}.
While the algorithmic structure has not been changed, the specific techniques represented by the building blocks have been refined and the applicable coding configuration has become more and more flexible over years, through the use of an increasing variety of coding tools.
Figure~\ref{Figure2:HybridCodec} also shows the block diagram of a generic hybrid video coding system~\cite{BookLucas2017}.
While the generic algorithmic structure has not been changed, the specific techniques represented by the building blocks have been refined and the applicable coding configuration has become more and more flexible over years, through the use of an increasing variety of coding tools.
This is the reason why the coding scheme is called hybrid, as it combines quiet different prediction modes between pictures with transform and entropy coding~\cite{VideoCodingFundamentals2015}.
\begin{figure}[t]
@@ -68,7 +68,8 @@ This is the reason why the coding scheme is called hybrid, as it combines quiet
In general, there are two kinds of redundancy in video signal: spatial and temporal.
On the one hand, the spatial redundancy is exploited through intra frame prediction, where the underlying mechanism is to find the best matching block for the current one, by searching the neighbouring region.
This is possible due to a high correlation between neighbouring pixels within a frame. As result, in intra frame prediction, a block of pixels can be reconstructed from the blocks previously processed.
This is possible due to the existence of high correlation between neighbouring pixels within a frame.
As result, when intra frame prediction is used, a block of pixels is reconstructed from the neighbouring blocks previously processed.
On the other hand, inter frame prediction is designed to exploit the temporal redundancy that exists between consecutive frames, such as those temporally adjacent, e.g., the past frame or the future frame in regard to the current one.
Temporal redundancy is normally referred to as the correlation between two adjacent frames since they usually have quiet similar visual content, such as foreground objects and background.
Thus, the purpose of inter-frame prediction is to search for the best matching block for the current one in a set of reference frames.
@@ -92,16 +93,18 @@ In the next section, the aforementioned blocks are described in the context of t
\glsreset{cu}
\glsreset{pu}
\glsreset{tu}
\glsreset{nalu}
\glsreset{nal}
%\glsreset{nalu}
\glsreset{cabac}
\glsreset{rpset}
Similar to the previous coding standards developed by JCT-VC, in \ac{hevc}~\cite{HEVCOverviewJul2012,HEVCOverviewDez2012,BookHEVC2014} only the
Similar to the previous coding standards developed by \ac{jctvc}, in \ac{hevc}~\cite{HEVCOverviewJul2012,HEVCOverviewDez2012,BookHEVC2014} only the
coding syntax and semantics are standardised.
The constrains and functionalities are defined through profiles and levels, while the semantic meaning of syntax elements define the decoding process, such that every decoder conforming to the standard generates the same output when given a complaint stream.
Such delimitation of scope in the standard allows maximum freedom for developers and manufactures to develop proprietary optimisations to cope with various applications’ requirements (\emph{e.g.}, balancing the compression ratio and the coding complexity).
Such delimitation of scope in the standard concede maximum freedom for developers and manufactures to develop proprietary optimisations, allowing to cope with various applications’ requirements (\emph{e.g.}, balancing the compression ratio and the coding complexity).
Figure~\ref{fig:HEVCBlockDiagram} illustrates the block diagram of the \ac{hevc} encoder. It follows the same overall hybrid architecture as its predecessors starting from H.261.
The encoder uses both intra- and inter-picture prediction to exploit spatial and temporal redundancies, respectively. Subsequently, the prediction residue transformed by linear spatial transform, which is then quantised, and entropy coded, allowing to further reduce the bitstream redundancy.
The encoder uses both intra- and inter-picture prediction to exploit spatial and temporal redundancies, respectively.
Subsequently, the prediction residue is transformed by linear spatial transform and the resulting coefficients are then quantised and entropy coded, further reducing the bitstream redundancy.
%The resulting bitstream is combined with control information, which signals the prediction chosen, motion vectors, filtering operations and bitstream headers.
\begin{figure}[tb]
@@ -113,8 +116,9 @@ The encoder uses both intra- and inter-picture prediction to exploit spatial and
\end{figure}
However, there are some novelties introduced in the \ac{hevc} standard. As depicted in Figure~\ref{fig:HEVCBlockDiagram}, the encoding process follows a block-based approach, therefore each picture is partitioned into larger blocks, referred to as \acs{ctu}, with a maximum size of $64\times64$ pixels.
Subsequently, a dynamic quad-tree partitioning is used to divide the picture into \acs{cu}. These units are introduced as the basic processing units for prediction, and for the transform and quantisation operations, \acs{tu} are used.
In order to generate the prediction residue the \ac{hevc} may use one of 34 intra-prediction modes or take advantage of symmetric and asymmetric partitions for motion estimation and inter-prediction.
Subsequently, a dynamic quad-tree partitioning is used to divide the picture into \acs{cu}.
These units are introduced as the basic processing units for prediction, while for the transform and quantisation operations, \acs{tu} are used.
In order to generate the prediction residue, the \ac{hevc} may use one of 34 intra-prediction modes or take advantage of symmetric and asymmetric partitions for motion estimation and inter-prediction.
The motion estimation is performed in the pictures listed in the \ac{rpset}, which was introduced in the \ac{hevc} to increase the robustness to errors and data loss of reference picture management.
To improve the overall quality of the reconstructed pictures and smooth the coding artefacts, \ac{hevc} not only uses a deblocking filter but also a \ac{sao} filter. Finally, the control data, quantised coefficients, prediction modes, motion vectors and filter information are compressed using the \ac{cabac}. The compressed bitstream is then allocated to various \ac{nal} units.
@@ -131,17 +135,20 @@ To improve the overall quality of the reconstructed pictures and smooth the codi
\glsreset{vcl}
This section describes the high-level components of the \ac{hevc} standard that provide support for signalling and stream partitioning, which are relevant elements in the context of robust video communications.
\ac{hevc} inherits several high-level features from \ac{h264}, e.g., the network abstraction layer, parameter sets, the use of \ac{poc} and \ac{sei} messages for auxiliary data.
\ac{hevc} inherits several high-level features from \ac{h264}, e.g., the \ac{nal}, parameter sets, the use of \ac{poc} and \ac{sei} messages for auxiliary data.
However, some high-level features defined in \ac{h264} were not included, such as \ac{fmo}, redundant slices, arbitrary slice order, data partitioning and switching slices.
However, several new high-level features are introduced, such as the \ac{vps}, tiles and wavefront tools for parallel processing, dependent slices, for reduced delay, and a new reference picture management concept.
Futhermore, several new high-level features are introduced, such as the \ac{vps}, tiles and wavefront tools for parallel processing, dependent slices, for reduced delay, and a new reference picture management concept.
Moreover, new pictures types were introduced to increase the random-access flexibility and temporal sub-layer switching.
\subsection{Network Abstraction Layer}
Similar to \ac{h264}~\cite{H264AVCStandard2003}, the \ac{hevc} streams are based on \glsfirst{nal} for stream partitioning and synchronization, which is useful to deal with lossy network environments~\cite{H264OverIPJul2003}. Using this concept the stream is composed of different \ac{nal} units. The \ac{nal} concept provides the ability to map the \ac{vcl} data that represents coded video slices (or frames) on various transport layers, such as \ac{rtp}, MPEG-2 Systems and MPEG DASH. The~\ac{nal} units are classified into two categories: (i) \ac{vcl} and (ii) non-\ac{vcl}, as they carry coded video or associated data, respectively.
Similar to \ac{h264}~\cite{H264AVCStandard2003}, the \ac{hevc} standard defines an adaptation layer to cope with different transport protocols.
The \ac{nal} is used for stream partitioning and synchronization, which is useful to deal with lossy network environments~\cite{H264OverIPJul2003}.
Using this concept the video stream is composed of different \ac{nal} units. The \ac{nal} concept provides the ability to map the \ac{vcl} data that represents coded video slices (or frames) on various transport layers, such as \ac{rtp}, MPEG-2 Systems and MPEG DASH. The~\ac{nal} units are classified into two categories: (i) \ac{vcl} and (ii) non-\ac{vcl}, as they carry coded video or associated data, respectively.
The \ac{hevc} standard defines two bytes to describe the content of each \ac{nal} unit.
These bytes are used to identify the \ac{nal} type are transmitted using a fixed coding format length, i.e., without variable length codes. This allow extra error robustness but increases the bit overhead. Figure~\ref{Figure2:NalHeader} shows the structure of the \ac{hevc} \ac{nal} unit header. The first bit \textit{F}, referred to as \textit{forbiden\_bit}, is always zero in order to ensure that a start code can be detected across different protocol standards. This is followed by the \ac{nal} type, which identifies its content, enabling the decoder to either use it if required for a particular decoding purpose or discard it. Another syntax element introduced by the \ac{hevc} standard is the \textit{temporal\_id\_plus\_one} (TIDP), which defines the temporal layer where the \ac{nal} unit belongs to. This allows implicit temporal scalability (layers ranging from 0 to 6) by immediately discarding the \ac{nal} unit if it belongs to a layer higher than the desired one. In the header, six bits are reserved for future extensions, preventing the creation of extra \ac{nal} units for scalable and multiview extensions, as done in \ac{h264}.
The \ac{hevc} standard defines a two bytes header to describe the content of each \ac{nal} unit, which are transmitted using a fixed coding format length, i.e., without variable length codes.
This allow extra error robustness but increases the bit overhead.
Figure~\ref{Figure2:NalHeader} shows the structure of the \ac{hevc} \ac{nal} unit header. The first bit \textit{F}, referred to as \textit{forbiden\_bit}, is always zero in order to ensure that a start code can be detected across different protocol standards. This is followed by the \ac{nal} type, which identifies its content, enabling the decoder to either use it if required for a particular decoding purpose or discard it. Another syntax element introduced by the \ac{hevc} standard is the \textit{temporal\_id\_plus\_one} (TIDP), which defines the temporal layer where the \ac{nal} unit belongs. This allows implicit temporal scalability (layers ranging from 0 to 6) by immediately discarding the \ac{nal} unit if it belongs to a layer that is higher than the pre-defined one. In the header, six bits are reserved for future extensions, preventing the creation of extra \ac{nal} units for scalable and multiview extensions, as done in \ac{h264}.
\begin{figure}[tb]
\centering
@@ -151,10 +158,10 @@ These bytes are used to identify the \ac{nal} type are transmitted using a fixed
\rlap{\bitbox{#2}{\color{#1}\rule{\width}{\height}}}%
\bitbox{#2}{#3}}
\begin{bytefield}[bitwidth=20pt]{16}
% \bitheader{0-15} \\
\bitheader{0-15} \\
\colorbitbox{TikZColorFill}{1}{F} & \colorbitbox{gray!60}{6}{NAL Type} & \colorbitbox{TikZColorFill}{6}{Reserved} & \colorbitbox{gray!60}{3}{TIDP}
\end{bytefield}
\caption{Two bytes \ac{nal} unit header in HEVC.}
\caption{\ac{nal} unit header format in HEVC.}
\label{Figure2:NalHeader}
\end{figure}
@@ -163,11 +170,14 @@ These bytes are used to identify the \ac{nal} type are transmitted using a fixed
\glsreset{vps}
The encoder uses a set of signalling packets as a robust method to share data with the decoder, referred to as parameter set units. This information was introduced in previous standards to deal with vulnerabilities in the transport of signalling information between the encoder and decoder, which may compromise the streaming applications.
Within this approach such set of parameters is multiplexed in the bitstream and repeated as many times as required by the application, e.g., streaming, storage among others. In \ac{hevc}, this structure is inherited from \ac{h264}\cite{H264OverIPJul2003} with modifications to cope with new coding tools.
In this approach, such set of parameters is multiplexed in the bitstream and repeated as many times as required by the application, e.g., streaming, storage among others. In \ac{hevc}, this structure is inherited from \ac{h264}\cite{H264OverIPJul2003} with modifications to cope with new coding tools.
The \ac{hevc} includes three sets of parameters, \ac{sps}, \ac{pps} and the newly introduced \ac{vps}\cite{HEVCHighLevelReferenceDez2012}.
% \ac{sps} include sequence related information, that include the decoder operation point, flags to enable or disable certain tools within a given profile. \Gls{pps} include picture related information, that may change for every picture, e.g., \ac{qp} and tilling information.
The new set of parameters \ac{vps} contains information related to the different layers of the video signal, avoiding duplications like in the \ac{h264} and its extensions. \Gls{vps} aims to provide signalling for every layer, guaranteeing an extendible standard, capable of supporting multiple layers. Moreover, \ac{vps} carries information for session negotiation (\emph{e.g.} profile, level and tier). The \ac{sps} contains information also related to the whole sequence (\emph{i.e.}, should affect all coded slices), and also related with the decoder operation point, as well as flags to control optional tools and scalability. The \ac{pps} is responsible to carry information that may change for every picture, such as, initial \ac{qp}, flags for picture related tools and tilling information.
The new set of parameters \ac{vps} contains information related to the different layers of the video signal, avoiding duplications like in the \ac{h264} and its extensions. \Gls{vps} aims to provide signalling for every layer, guaranteeing an extendible standard, capable of supporting multiple layers.
Moreover, \ac{vps} also carries information for session negotiation (\emph{e.g.} profile, level and tier).
The \ac{sps} contains information related to the whole sequence (\emph{i.e.}, should affect all coded slices) and related with the decoder operation point, as well as flags to control optional tools and scalability.
The \ac{pps} is responsible to carry information that may change for every picture, such as, initial \ac{qp}, flags for picture related tools and tilling information.
\subsection{Picture types}
@@ -213,22 +223,20 @@ The new set of parameters \ac{vps} contains information related to the different
\end{tabular}
\end{table}
The \ac{hevc} standard supports several pictures types, as show in Table~\ref{Table2:PictureTypes}, which are divided in the following categories: (i) random-access points, (ii) leading pictures and (iii) temporal sub-layer access pictures~\cite{HEVCHighLevelReferenceDez2012,BookHEVC2014}.
The \ac{hevc} standard supports several pictures types, as show in Table~\ref{Table2:PictureTypes}, which are classified in the following categories: (i) random-access points, (ii) leading pictures and (iii) temporal sub-layer access pictures~\cite{HEVCHighLevelReferenceDez2012,BookHEVC2014}.
The use of different types of pictures allows the \ac{hevc} stream to be more flexible, support a wider range of applications and achieve robust video communications.
They are important to perform random-access, which is a critical feature for channel switching, seek operations and dynamic streaming services.
Moreover, introducing random-access points into the bitstream increases its error robustness as they provide anchor points for a clean decoding after data loss or synchronisation problems.
%\HIGHLIGHT{adicionar texto mencionar robustez}
\subsubsection{Random-Access Point}
The \ac{rap} pictures are used to provide points in the bitstream with no dependencies, where it is possible to start the decoding process. The standard defines that a conforming bitstream must start with a \ac{rap} picture.
The \ac{rap} are pictures used to provide access points in the bitstream with no dependencies, where it is possible to start the decoding process. The standard defines that a conforming bitstream must start with a \ac{rap} picture.
These frames must belong to temporal sub-layer 0 and should be coded using intra coding techniques, i.e., must not use previously coded frames as reference.
One should note that there may be frames compressed with intra prediction only, but not be marked as random-access points.
It is always possible to start decoding from a \ac{rap} frame onwards, and to output any subsequent pictures in the display order, even if all pictures that precede the \ac{rap} in the decoding order are discarded from the bitstream.
The \ac{idr} frame is an intra coded picture that completely refreshes the decoding process, i.e., cleans the decoder picture buffer preventing predictions to previously coded frames, starting a new conforming video stream. This means that neither the \ac{idr} nor any other subsequent frame in the decoding order have prior dependencies. \ac{idr} frames are allowed to have leading pictures, i.e., frames follow the \ac{rap} picture in the decoding order but precede it in the display order. However, they must be decodable, thus creating a true random-access point where all subsequent frames in the decoding order can be decoded and displayed.
The \ac{idr} frame is an intra coded picture that completely refreshes the decoding process, i.e., cleans the decoder picture buffer preventing predictions to previously coded frames, starting a new conforming video stream. This means that neither the \ac{idr} nor any other subsequent frame in the decoding order have prior dependencies. \ac{idr} frames are allowed to have leading pictures, i.e., frames that follow the \ac{rap} picture in the decoding order but precede it in the display order. However, they must be decodable, thus creating a true random-access point where all subsequent frames in the decoding order can be decoded and displayed.
Introducing \ac{rap} reduces the coding efficiency since no prior reference frames can be used.
In order to reduce this impact, the \ac{hevc} standard introduces a new \ac{rap} picture, referred to as \ac{cra} picture.
@@ -236,13 +244,13 @@ These frames do not refresh the decoding process, allowing leading pictures to d
This enables a more efficient prediction structure, while maintaining an access point to begin the decoding process.
However, it results in leading pictures which may not be decodable, which are explained in the next sub-section.
The introduction of \ac{cra} frames in \ac{hevc} was able to improve the coding efficiency by up to 6\% \cite{HEVCCRAPics2011}.
% TODO: Deves saber o porque desta diferença ^
%\HERE
The \ac{hevc} supports bitstream splicing, which means taking a particular bitstream from a \ac{rap} (both \ac{idr} and \ac{cra} frames) and inserting it into another bitstream at a random-access point.
In case of bitstream splicing, starting from an \ac{idr} frame, the leading pictures do not have dependencies from frames prior the \ac{rap}, thus all frames are decodable.
However, whenever this is performed for a bitstream starting from a \ac{cra} frame, the leading associated frames might not be correctly decoded, because some of their reference pictures are not present in the combined bitstream.
Therefore, in order to make splicing operation straightforward, the \ac{nal} units containing the \ac{cra} picture are changed to the type \ac{bla}. This way, the decoder is aware that the leading frames should be discarded, namely the \ac{rasl} pictures.
To make the splicing operation straightforward, the \ac{nal} units containing the \ac{cra} picture are changed to the type \ac{bla}. Therefore, the decoder is aware that the leading frames should be discarded, namely the \ac{rasl} pictures.
\subsubsection{Leading and trailing pictures}
@@ -263,9 +271,10 @@ Due to the introduction of more flexible random-access points using \ac{cra} pic
\end{figure}
There are two types of leading pictures, as shown in Table~\ref{Table2:PictureTypes}.
The \ac{radl} pictures, which can be correctly decoded since they only depend on the associated \ac{rap} picture and do not have any prior dependencies to trailing pictures associated with the previous \ac{rap} frame.
On contrary, the \ac{rasl} pictures may have dependencies to prior trailing pictures.
When a random-access is performed at the associated \ac{rap} frame, these frames cannot be correctly decoded, therefore they should be skipped. In the example of Figure~\ref{Figure2:LeadingTralingPics} the leading pictures represented correspond to \ac{rasl} pictures.
The \ac{radl} pictures, which can be correctly decoded, since they only depend on the associated \ac{rap} picture and do not have any prior dependencies to trailing pictures associated with the previous \ac{rap} frame.
On the contrary, the \ac{rasl} pictures may have dependencies to prior trailing pictures.
When a random-access is performed at the associated \ac{rap} frame, these frames cannot be correctly decoded, therefore they have to be skipped.
In the example of Figure~\ref{Figure2:LeadingTralingPics} the leading pictures correspond to \ac{rasl} pictures.
\subsubsection{Temporal sub-layer switching pictures}
@@ -273,7 +282,8 @@ When a random-access is performed at the associated \ac{rap} frame, these frames
A \ac{tsa} picture is a trailing picture that marks a temporal layer switching point.
When decoding a sub-set of temporal layers, if a \ac{tsa} picture is found in the temporal layer just above the maximum temporal layer currently being decoded, it is possible to start decoding any number of additional temporal layers.
For example, frame $P_6$ in Figure~\ref{Figure2:TSAExample} is considered a \ac{tsa} picture since it only depends on a frame belonging to temporal layer 0 for prediction, as well as, any subsequent frame predicted from the \ac{tsa} frame $P_6$.
In this example, if the decoder was only decoding the temporal layer 0, it can after decoding $P_6$ it can also decode both layers 0 and 1, or decode all the available three layers. Although theses frame allows higher flexibility in terms of layer switching, they severely constrain the prediction of frames following the \ac{tsa} picture.
In this example, if the decoder is only decoding the temporal layer 0, after decoding $P_6$ it can also decode layer 1, or decode all the available three layers.
Although these frames allow higher flexibility in terms of layer switching, they severely constrain the prediction of frames following the \ac{tsa} picture.
\begin{figure}[tb]
\centering
@@ -284,7 +294,9 @@ In this example, if the decoder was only decoding the temporal layer 0, it can a
\end{figure}
To reduce the constrains in frame prediction due to layer switching, \ac{hevc} introduces the \ac{stsa} picture.
These pictures have similar purpose as the \ac{tsa} frames, but they only guarantee that frames in the same temporal layer as the \ac{stsa} frame are able to be correctly reconstructed. This is guaranteed by avoiding any dependency from frames preceding the \ac{stsa} to be used as reference in its temporal layer.
These pictures have similar purpose as the \ac{tsa} frames, but they only guarantee that frames in the same temporal layer as the \ac{stsa} frame are able to be correctly reconstructed.
%This is guaranteed by avoiding any dependencies from frames preceding the \ac{stsa} to be used as reference in its temporal layer.
This is guaranteed by not using frames preceding the \ac{stsa} as reference in its temporal layer.
One example of a \ac{stsa} frame is also shown in Figure~\ref{Figure2:TSAExample} in frame $P_2$.
This frame can be used to switch to layer 1, as it does not have dependencies to any prior frame in the same layer ($P_2$ only depends on $P_0$).
However, it cannot be classified as \ac{tsa} picture because $P_3$ has dependencies to a prior frame in the decoding order ($P_1$).
@@ -293,10 +305,11 @@ Summarising, \ac{stsa} frames can be used to switch to a particular layer, as \a
\subsection{Picture partitioning}
The high-level segmentation of a picture in \ac{hevc} is achieved through four techniques: regular slices, dependent slices, tiles and \glsfirst{wpp}. Picture partitioning normally serves one or more of the following three purposes:
The high-level segmentation of a picture in \ac{hevc} is achieved by using four different approaches associated to different data structures: regular slices, dependent slices, tiles and \ac{wpp}.
Picture partitioning normally serves one or more of the following three purposes:
\begin{description}
\item[Error robustness:] partitioning the picture into smaller self-contained units in
order to gain error robustness, allowing to re-synchronize both the parsing and decoding
order to increase error robustness, allowing to re-synchronize both the parsing and decoding
processes in case of data losses.
\item[Network adaptation:] adapt to the network constraint of \ac{mtu} size, found for example in \ac{ip} networks.
@@ -307,11 +320,11 @@ The high-level segmentation of a picture in \ac{hevc} is achieved through four t
\subsubsection{Slices}
The \ac{hevc} standard preserves the slice structure defined in the \ac{h264} standard. A slice may comprise either the entire frame or a section of it, and all the associated data (\emph{i.e.}, entropy symbols, prediction and residue information) can be independently decoded. Each slice is transported in a different \ac{nal} unit.
The \ac{hevc} standard preserves the slice structure previously defined in the \ac{h264} standard. A slice may comprise either the entire frame or a section of it, and all the associated data (\emph{i.e.}, entropy symbols, prediction and residue information) can be independently decoded. Each slice is transported in a different \ac{nal} unit.
As some dependencies across slice boundaries are disabled, each slice can be independently reconstructed, regardless whether previous slices were lost or incorrectly decoded. Slice partitioning is the only tool that can be used for parallel processing in a virtually identical form as in \ac{h264}.
As each slice is independent from each other, it is straightforward to process multiple slices in parallel, without inter-process communication (except for inter-frame prediction). Although this is a simple approach, it incurs in substantial coding overhead due to the higher number of slice headers, and reduction of causal neighbours for prediction (due to lack of predictions across slice boundaries). Another disadvantage of using slices for parallel processing is that they are also used to meet the \gls{mtu} size constrains. Therefore, it might be impossible to meet both requirements using only one technique.
Another partitioning tool in \ac{hevc} standard is the dependent slices. These slices allow the partitioning of a frame at the coding unit boundaries without breaking intra-frame predictions. Moreover, as they have a smaller slice header they are more efficient in terms of signalling overhead than regular slices. Dependent slices are normally used to reduce the end-to-end delay by allowing part of the slice to transmitted while the rest of its data is still being processed.
Another partitioning tool in \ac{hevc} standard is the dependent slices. These types of slices allow the partitioning of a frame at the coding unit boundaries without breaking intra-frame predictions. Moreover, as they have a smaller slice header they are more efficient in terms of signalling overhead than regular slices. Dependent slices are normally used to reduce the end-to-end delay by allowing part of the slice to be transmitted while the rest of its data is still being processed.
\subsubsection{Tiles}
@@ -337,11 +350,13 @@ The use of tiles provide several advantages. Specifically, they improve frame pa
\subsubsection{Wavefront Parallel Processing}
\glsreset{wpp}
As mentioned before, the use of either slices or tiles to process each frame using parallel threads require the use of different entropy coding contexts for each slice/tile, in order to make them independently decodable. As consequence, the compression efficiency decreases. In order to overcome this issue, the concept of \ac{wpp} was introduced, which enables coding using efficient parallel processing~\cite{HEVCParallelCSVT2016}.
In \ac{wpp}, each slice (or tile) is divided into single rows of \acs{ctu} which can be processed in parallel, but entropy coding and prediction are enabled across \acs{ctu} of different rows, in order to avoid loss in coding efficiency.
To re-use the same context in the whole slice (or tile) each row requires a delay of two \acs{ctu} in order to ensure that all necessary dependencies have already been encoded/decoded. Figure~\ref{Figure2:WPP} illustrates an example of \ac{wpp} using at least five different threads.
Each square corresponds to a \ac{ctu} and the numbers correspond to the time instance when a \ac{ctu} starts being processed. Using this staggered start (which appears like a wavefront, as represented in Figure~\ref{Figure2:WPP}), the parallelisation may use as many threads as the number of \acs{ctu} rows available in the video frame. However, although the number of threads does not affect the coding efficiency, the required inter-process communication substantially increases.
Each square corresponds to a \ac{ctu} and the numbers correspond to the time instance when a \ac{ctu} starts being processed. Using this staggered start, which appears like a wavefront, as represented in Figure~\ref{Figure2:WPP}, the parallelisation may use as many threads as the number of \acs{ctu} rows available in the video frame. However, although the number of threads does not affect the coding efficiency, the required inter-process communication substantially increases.
\begin{figure}[tb]
\centering
@@ -357,8 +372,9 @@ Each square corresponds to a \ac{ctu} and the numbers correspond to the time ins
\label{Section2:CodingStructures}
Despite the fact that the \ac{hevc} standard follows a traditional hybrid video coding architecture, it introduces significant changes in the data structures when compared to the previous standards (e.g., \ac{h264}).
In \ac{h264} each frame is divided into units so-called \acs{mb}, which are composed by a $16\times16$ block of luma samples and two $8\times8$ blocks of chroma samples, assuming video data in YUV 4:2:0 chroma sub-sampling. The \ac{mb} is considered the basic processing unit in the \ac{h264}, and the respective coding mode define whether the corresponding pixels are coded as intra or intra prediction.
Each \ac{mb} could also be partitioned into sub-blocks which should follow the same prediction mode.
In \ac{h264} each frame is divided into units so-called \acs{mb}, which are composed by a $16\times16$ block of luma samples and two $8\times8$ blocks of chroma samples, assuming video data in YUV 4:2:0 chroma sub-sampling.
The \ac{mb} is considered the basic processing unit in the \ac{h264}, and the respective coding mode defines whether the corresponding pixels are coded as intra or intra prediction.
Each \ac{mb} could also be partitioned into sub-blocks, which should follow the same prediction mode.
\glsreset{ctu}
@@ -366,9 +382,9 @@ Each \ac{mb} could also be partitioned into sub-blocks which should follow the s
The basic data structures in \ac{hevc} are quite different from \ac{h264}. Each picture is divided into blocks named as \ac{ctu}~\cite{BlockPartHEVC2012}. The maximum \ac{ctu} size is an encoder configuration parameter that is signalled to the decoder, and should be one of the following: $64\times64$, $32\times32$ and $16\times16$. The use of data structures with larger size in \ac{hevc} was adopted to achieve higher coding efficiency especially in high resolution video. Each \ac{ctu} can be recursively partitioned into four smaller units, referred to as \acs{cu}, until it reaches the minimum unit size, which cannot be smaller than $8\times8$ pixels.
Similar to the maximum \ac{ctu} size, the minimum size is also defined at the encoder.
For instance, in case of homogeneous regions, large \acs{cu} can be used to represent those regions
by using a smaller number of symbols than is the case of using several small units.
Figure~\ref{Figure2:HEVCCodingStructure} illustrates an example of a \ac{ctu} partitioning into \acs{cu}. The partitioning illustrated in the figure leads to the quad-tree structure represented on the left side with the corresponding coding order shown on the right side.
For instance, in case of homogeneous regions, large \acs{cu} can be used to represent such regions by using a smaller number of symbols than in the case of using several small units.
Figure~\ref{Figure2:HEVCCodingStructure} illustrates an example of a \ac{ctu} partitioning into \acs{cu}.
The partitioning illustrated in the figure leads to the quad-tree structure represented on the left side with the corresponding coding order shown on the right side.
Data structures in \ac{hevc} provide a significant improvement in comparison with previous standards, by eliminating the distinction between \ac{mb} and sub-\ac{mb} and using only the \ac{cu} concept. This allows for the multilevel hierarchical quadtree structure to be specified in a simple and elegant way, with size-independent syntax representation.
\begin{figure}[t]
@@ -385,12 +401,6 @@ The \ac{cu} is the basic processing unit in \ac{hevc}. It is further partitioned
The decision whether to use intra-frame or inter-frame prediction modes is taken at the \ac{cu} level. However, depending on the partitioning of the \ac{cu} into \acs{pu}, different intra or inter coding modes can be used for each \ac{pu}.
%The main key feature of \ac{hevc} is the new block partition structure, which achieves more flexibility and efficiency. Four different block concepts have been introduced in the standard: \ac{ctu}, \ac{cu}, \ac{pu}, and \ac{tu}, all of which have different roles. Figure~\ref{fig:HEVCBlockStructure} illustrates an example of a CTU partitioning in HEVC, and it is compared to the \ac{h264} \ac{mb} structure. CTUs are defined as the high-level partition within each slice and can be as large as 64$\times$64, which can be recursively partitioned into four smaller blocks referred to as CUs. The CU is the basic units for the intra/inter prediction and can be as small as 8$\times$8. Subsequently, the CU are divided into up to four PUs which are used for inter prediction. For intra coding PU is not used. Finally, the CU is divided into a transform unit tree, which may include several TUs, in a quad-tree structure. Transform units are the basic units for the transform operations applied to the coding residue.
%
%Each CU may be split into two or four Pus resulting in either symmetric or asymmetric partitions. Asymmetric partitioning of the CU into two PUs enables more efficient motion estimation since it allows more flexible partitioning of the picture~\cite{BlockPartHEVC2012}. This scheme is able to improve the coding performance by up to $1\%$ when compared with rectangular symmetric partitioning.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% SECTION %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
@@ -406,21 +416,16 @@ This section describes the prediction techniques used in the \ac{hevc} standard
Intra-frame prediction is used to efficiently reduce the spatial redundancy within video frames.
Intra predicted blocks are obtained using the previously encoded pixels around the current \ac{pu}.
When the intra prediction mode is chosen for a \ac{cu}, the size of the \ac{pu} need to be the same as the \ac{cu}, except when the smaller \ac{cu} is selected and further divided into four \acs{pu} is done.
When the intra prediction mode is chosen for a \ac{cu}, the size of the \ac{pu} needs to be the same as the \ac{cu}, except when the smaller \ac{cu} is selected and further divided into four \acs{pu} is done.
For example, when the smaller \ac{cu} size is $8\times8$, the encoder may decide to divide it into four \acs{pu} of $4\times4$ pixels, each one having its own intra prediction mode. This is signalled using a binary flag.
The use of small \acs{pu} is useful for regions with many details that require fine-granularity prediction.
In \ac{hevc} there are 35 intra-frame prediction modes: Planar, DC and 33 angular modes, as shown on the left of Figure~\ref{Figure2:IntraModesHEVC}~\cite{IntraCodingHEVC2012,HEVCIntraperformance2012}.
%Figure~\ref{Figure2:IntraModesHEVC} illustrates these intra prediction modes.
%As shown, the predictions are applied to square blocks only.
%The border around the square blocks corresponds to the reference pixels from which the prediction is applied (different colours correspond to different pixel values).
%The example in the figure corresponds to a \ac{pu} size of $8\times8$ pixels, but block sizes ranging from $4\times4$ to $32\times32$ are also allowed.
For each \ac{pu}, the encoder chooses among all the available intra prediction modes the best mode to be used by the decoder, based on the previously reconstructed pixels also available for similar prediction at the decoder.
The reconstructed reference pixels used in the prediction process belong to
the neighbouring blocks at left-down, left, top-left, top and top-right positions, as
The reconstructed reference pixels used in the prediction process belong to the neighbouring blocks located at left-down, left, top-left, top and top-right positions, as
can be seen in the example shown on the right in Figure~\ref{Figure2:IntraModesHEVC}.
Given a $N \times N$ \ac{pu}, intra prediction requires the top neighbouring row of $2N$ pixels, the left column of $2N$ pixels and the top-left neighbouring pixel.
Due to data unit boundaries (e.g., slice or tile boundaries) and constrained intra prediction (i.e., reference pixels belonging to inter-predicted \acs{pu} are not used in order to remove error propagation from potentially erroneous reconstructed reference frames) neighbouring pixels might not be all available to use intra prediction, resulting in an incomplete set of neighbouring reference pixels.
Due to data unit boundaries (e.g., slice or tile boundaries) and constrained intra prediction (i.e., reference pixels belonging to inter-predicted \acs{pu} are not used in order to remove error propagation from potentially erroneous reconstructed reference frames), neighbouring pixels might not be all available to be used as reference for intra prediction, resulting in an incomplete set of neighbouring reference pixels.
In order to overcome this issue and to allow all possible modes, \ac{hevc} uses reference sample substitution to replace the unavailable reference pixels with the closest available ones, so that all intra prediction modes can be used.
\begin{figure}[tb]
@@ -441,8 +446,10 @@ Directional intra prediction was first introduced in the \ac{h264} standard~\cit
Directional prediction projects the reconstructed samples in the block neighbourhood (see reference pixels in Figure~\ref{Figure2:IntraModesHEVC}) along a specific direction. The \ac{hevc} increases the number of possible directions from 8 (in \ac{h264}) to 33~\cite{IntraCodingHEVC2012}. The direction of each projection is illustrated in the left-side of Figure~\ref{Figure2:IntraModesHEVC} by the directional modes 2 to 34.
%In the figure it is possible to notice that directional modes 2 to 10 only project the pixels from the left neighbouring border in a horizontal direction, modes 10 to 25 use both the left and top neighbouring pixels in a diagonal direction and modes 26 to 34 project the top neighbours in vertical direction.
As shown in Figure~\ref{Figure2:IntraModesHEVC}, the angular modes are designed to provide a dense coverage near the horizontal (i.e., mode 10) and vertical (i.e., mode 26) directions. Moreover, the angular directions are more coarse as it gets closer to the diagonals. This reflects the observed statistical prevalence of the angles and the prediction efficiency.
To improve the efficiency of angular prediction, when the reference samples need to be projected a bilinear interpolation is used from the two closest pixels using a $1/32$ accuracy. The dashed arrows on the right of Figure~\ref{Figure2:IntraModesHEVC} correspond to a projection of interpolated pixels.
As shown in Figure~\ref{Figure2:IntraModesHEVC}, the angular modes are designed to provide a dense coverage near the horizontal (i.e., mode 10) and vertical (i.e., mode 26) directions.
Moreover, the angular directions are coarser as it gets closer to the diagonals.
This reflects the observed statistical prevalence of the angles and the prediction efficiency.
To improve the efficiency of angular prediction, when the reference samples need to be projected, a bilinear interpolation is used from the two closest pixels using a $1/32$ accuracy. The dashed arrows on the right of Figure~\ref{Figure2:IntraModesHEVC} correspond to a projection of interpolated pixels.
@@ -521,7 +528,8 @@ Another improvement in \ac{hevc} is related with the reference picture manageme
Decoded frames in the \ac{dpb} are grouped in one of the following categories: (i) \emph{short-term reference}, (ii) \emph{long-term reference} and (iii) \emph{unused for reference}.
Once a frame is marked as \emph{unused for reference} it is no longer used for motion compensation and it will be discarded from the \ac{dpb} after being displayed.
In \ac{h264} a sliding window approach is used to implicitly manage the reference picture list.
This process discards a given frame whenever the \ac{dpb} has the number of defined reference frames (which is controlled by the encoder). Moreover, \ac{h264} uses an explicitly \ac{mmco} to send commands to the decoder and change the normal reference picture management process.
This process discards a given frame whenever the \ac{dpb} has stored the maximum number of reference frames (which is defined and controlled by the encoder).
Moreover, \ac{h264} uses an explicitly \ac{mmco} to send commands to the decoder and change the normal reference picture management process.
In the \ac{hevc} standard the status of the \ac{dpb} is encoded for every slice, instead of the implicit managing used in the past.
The most fundamental difference in \ac{rpset} concept compared to \ac{mmco}/sliding window of \ac{h264} is that each particular frame has a complete set of the reference frames that are used or will be used.
@@ -534,7 +542,7 @@ For the remaining frames, two types of information are encoded: frame number and
\begin{table}[tb]
\centering
\caption{Reference Picture Set example.}
\caption{\glsfirst{rpset} example.}
\label{Table2:ReferencePictureListExample}
\small
\TABLESVSPACE
@@ -560,7 +568,7 @@ Using the \ac{rpset} concept, the decoder is always able to detect loss events a
\subsubsection{Asymmetric block partitions}
When inter prediction mode is selected a higher range of partitioning schemes is allowed, thus dividing the \ac{cu} into one, two or four \acs{pu}.
When inter prediction mode is selected, a higher range of partitioning schemes is allowed, thus dividing the \ac{cu} into one, two or four \acs{pu}.
The splitting into four \acs{pu} is only allowed when the \ac{cu} size is equal to the minimum.
%Otherwise, the splitting should be carried out at the \ac{cu} level which leads to the same partition tree.
Figure~\ref{Figure2:InterPuPartitions} illustrates the possible \ac{cu} partitioning into \acs{pu}.
@@ -596,22 +604,24 @@ Moreover, it also uses a temporal \ac{mv} candidate, derived from the co-locate
The Merge Mode uses a $C-1$ spatial \ac{mv} candidates, selected from the aforementioned order, and one \ac{tmvp} to fulfil the total number of \acs{mv} ($C$). The value of $C$ is defined at the slice header and a flag controlling the use of \ac{tmvp} is encoded at the \ac{pps}-level.
For B-slices, additional \ac{mv} candidates are considered by choosing two existing candidates, according to a predefined order for reference picture list 0 and list 1. A total number of twelve combinations are specified in \ac{hevc}, which are listed in~\cite{HEVCOverviewDez2012}. For P-slices, if the number of candidates is less than $C$, then zero \acs{mv} are included to fill the remaining entries.
The Merge Mode defines how to encode the \ac{mv} candidate and the residual information.
In \ac{hevc}, the Skip mode, i.e., transmitting only the \ac{mv} without residual information, is treated as a special case of the Merge Mode.
In this case, only a skip flag is enabled and the corresponding merge index is transmitted.
\begin{figure}[tb]
\centering
\input{\FigsPath/MergeModeHEVC.tikz}
%\input{\FigsPath/HEVC/MergeModeHEVCTwoFrames.tikz}
\caption{Illustration of the MV prediction candidates in the AMVP mode.}
\caption{Illustration of the MV prediction candidates in the AMVP and Merge Mode mode.}
\label{Figure2:MVCandidates}
\end{figure}
The Merge Mode defines how to encode the \ac{mv} candidate and the residual information.
In \ac{hevc}, the Skip mode, i.e., transmitting only the \ac{mv} without residual information, is treated as a special case of the Merge Mode.
In this case, only a skip flag is enabled and the corresponding merge index is transmitted.
\subsubsection{Motion vector prediction}
The \ac{hevc} also improves the performance of the \ac{mv} coding for those cases where the Merge Mode is not used.
\ac{hevc} always use a differential encoding approach for efficiently representing the motion information.
The \ac{hevc} also improves the performance of \ac{mv} coding for those cases where the Merge Mode is not used.
A differential encoding approach is always used for efficiently representing the motion information.
It considers the same \ac{mv} candidates of the Merge mode, illustrated in Figure~\ref{Figure2:MVCandidates}, to derive the \ac{mv} predictors from which the difference vector is obtained.
From these \ac{mv} candidates only the first two available candidates are selected, based on the following order:
\begin{enumerate}
@@ -620,8 +630,8 @@ From these \ac{mv} candidates only the first two available candidates are select
\item Temporal candidate: first available from $T_0$ and $T_1$.
\end{enumerate}
Therefore, the use of the temporal \ac{mv} candidate is limited to those cases where the spatial neighbours are not able to provide two valid candidates.
For \ac{mv} prediction only a much lower number of candidates is allowed, since the encoder can still send the \ac{mv} difference. Moreover, the encoder performs motion estimation for each candidate, which increases the encoding complexity for every possibility tested.
For \ac{mv} prediction only a much lower number of candidates is allowed, since the encoder can still send a \ac{mv} difference. Moreover, the encoder performs motion estimation for each candidate, which increases the encoding complexity for every possibility tested.
%For those cases where the reference index is not the same as the current \ac{pu} and the neighbouring one, a scaling version of the \ac{mv} is used.
%In the \ac{h264} standard, along different motion compensated coding techniques~\cite{H264OverviewJul2003}, the so-called P\_SKIP mode was introduced. In this mode, neither the quantised residual information, nor the \ac{mv} is transmitted. This mode is useful for large areas with no change or just constant motion, and it is only enabled for $16\times16$ blocks, \emph{i.e.}, the whole \ac{mb}. The \ac{mv} for the P\_SKIP mode is derived based on the surrounding neighbour MBs\cite{H264MotionVectorPrediction2003}. When no neighbours are available, or the left or top \acs{mv} are equal to zero, a zero \ac{mv} is used. Otherwise, the \ac{mv} is derived from the median value of the following three neighbours: left, above and above right, as shown in Figure~\ref{SkipMergeMode}~(a) by $MV_A$, $MV_B$ and $MV_C$, respectively. Whenever the $MV_C$ is not available, the $MV_D$ is used instead.
@@ -648,7 +658,7 @@ A 2D transform of the residual information is obtained by applying 1D transforms
A special case is applied for coding the residual of $4\times4$ intra coded block.
In this case an alternative integer transform based on the \ac{dst} is used.
The motivation for \ac{dst} is related to the statistics of the residual signal of these blocks, which tend to present higher residue as the distance to the boundary used for prediction (i.e, top or left boundary) increases.
Experiments have shown that \ac{dst} provides up to 1\% of bitrate savings for the compression of $4\times4$ intra predicted blocks, without significantly increasing the computational complexity \cite{DCTDSTComp2013}.
Simulation studies have shown that \ac{dst} provides up to 1\% of bitrate savings for the compression of $4\times4$ intra predicted blocks, without significantly increasing the computational complexity \cite{DCTDSTComp2013}.
\subsubsection{Quantisation}
@@ -656,7 +666,8 @@ Experiments have shown that \ac{dst} provides up to 1\% of bitrate savings for
The \ac{hevc} follows the same principle as previous standards in terms of quantisation.
After applying the transform operation to the prediction residue, a quantisation is applied to the resulting coefficients using a uniform-reconstruction quantiser scheme based on a \ac{qp}.
The standard supports \ac{qp} values ranging from 0 to 51, where an increase of 6 in the \ac{qp} corresponds to doubling the quantisation step. This procedure is the main responsible for the coding distortion as it performs a many-to-one mapping, which cannot be reversed.
The standard supports \ac{qp} values ranging from 0 to 51, where an increase of 6 in the \ac{qp} corresponds to doubling the quantisation step.
This procedure is the main responsible for the coding distortion, as it performs a many-to-one mapping, which cannot be reversed.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
@@ -704,7 +715,7 @@ by adding an offset value, extracted from lookup tables transmitted by the encod
\section{Summary}
\label{Section2:Summary}
This chapter was devoted to describe the main features of the \ac{hevc} standard, giving a special focus on the high-level features, coding structures and newly introduced prediction modes.
This chapter was devoted to describe the main features of the \ac{hevc} standard with special emphasis on the high-level features, coding structures and newly introduced prediction modes.
At the high-level, \ac{hevc} introduces new \ac{nal} types and also new pictures types.
These allow the implementation of various random-access strategies, more flexibility and higher coding efficiency than its predecessor \ac{h264}.
Another major improvement comes from the inter-frame prediction. On the one hand new high-level reference picture management was introduced, which increase the error robustness, as it explicitly defines the active reference frames and the \ac{dpb} status. On the other hand it introduces the \ac{amvp} and the Merge mode, which efficiently reduce the required bits for coding the motion information.