Files
doc_phd_thesis/050_Conclusions/Conclusion.tex
T
2018-09-12 19:21:51 +01:00

82 lines
10 KiB
TeX
Executable File

\chapter{Conclusion and future work \VERSION{0}}
\label{Chapter:Conclusions}
\glsresetall
\glsunset{h264}
This thesis investigated new algorithms and techniques for efficient error robustness in highly efficiency video coders.
The importance of these techniques is justified by the spread of digital video contents, including high resolution formats, multi-view and 360-degree video formats, which significantly the network bandwidth requirements. Consequently, errors and data loss may occur along the transmission system.
Moreover, in order to meet the network bandwidth constrains video coding standard have been improved, by including advanced predictions techniques, which results in increased bitstream dependencies.
This increase the negative impact of errors, leading to higher quality degradation.
To overcome this problem, in this thesis different methods were proposed, addressing the problem of robust coding and efficient \ac{ec}.
%Subsequently, for each chapter of this thesis, the main conclusions are presented.
Firstly, an introduction of the main motivation for this research work is presented, enumerating its main objectives and contributions.
Then, the main functional blocks of the state-of-the-art \ac{hevc} are described.
This description covered the high-level features, such as the new picture types and the slice partitioning schemes, the new block partitioning scheme, which allow multi-level hierarchical structures with size-independent representation, the main predictions techniques, including the newly introduced advanced motion vector prediction technique.
This aimed at introducing the novel techniques in \ac{hevc}, which may have a relevant impact on the error robustness of the coded streams.
The main algorithms proposed in the past to increase the error robustness of video streams were presented, describing techniques based on error resilience \ac{rd} optimisation, redundant based approaches and \ac{ec}-based methods.
It was concluded that, although the previous works provide efficient techniques to reduce the bitstream dependencies and ease the error recovery, the \ac{hevc} standard introduces new problems that could not be fully addressed with existing techniques.
Such problems range from new \ac{mv} coding techniques to the lack of robust coding techniques, such as \ac{fmo} or data partitioning.
Moreover, there was also open issues regarding robust \ac{ec}-based encoding, as existing method do not fully exploit the use of different methods with different levels of complexity, especially the use of advanced \ac{ec} techniques.
To investigate new techniques for robust video coding an evaluation study was firstly carried out, which aimed at finding the influence of the new coding tools on the error resilience of the \ac{hevc} streams.
Existing studies only performed generic comparisons between \ac{hevc} and previous standards (e.g., \ac{h264}).
This study extended existing ones by considering different coding configurations, which allowed to evaluate the impact of new coding tools. The study revealed that some features, such as increased block size, does not have significant impact of the error robustness performance.
However, slice partitioning and intra-refreshing have a noticeable impact on both, coding efficiency and error robustness.
Therefore, a small compromise in coding efficiency should be reasoned, in order to achieve an acceptable quality in case of errors.
Finally, one can also conclude that the use of temporal predictions in \ac{mv} coding significantly increases temporal dependencies, leading to increased quality degradation compromising the performance of an \ac{hevc}-based video transmission system.
Based on the above evidences, a new \ac{mv} coding scheme was used in \ac{hevc}, aiming at reducing the temporal dependencies.
In this scheme the use of the temporal \ac{mv} candidate is selectively used in some \acs{pu}, preventing temporal dependencies across several frames.
As this method significantly reduces the temporal dependencies, the number of mismatched \acs{mv} is reduced, consequently reducing the error propagation.
Moreover, as the temporal candidate is not fully disabled the impact on coding efficiency is reduced.
Experimental results showed a clear improvement on decoded video quality, across different video content, with a small bitrate increase.
The reference frame distribution in highly efficient video coders was investigated, as it may increase the number of mismatch predictions in case of erroneous reference frames.
Since the reference frame usage is consistently unbalanced, a new method based on constrained \ac{rd} optimisation was proposed to equally distribute the use of multiple reference frames across neighbouring \acs{pu}.
An extension using selective \ac{mv} redundancies was also investigated, in order to recover from erroneous motion predictions.
This resulted in a two-fold approach to reduce error propagation.
As part of the proposed method can be applied at the streaming-stage, using a low complexity parser, this method can be used to adapt existing streams to error-prone transmission channels.
The proposed method was compared against existing techniques, and the simulation results revealed a consistently superior performance of using the two-fold approach, making it suitable for video services where packet loss cannot be negligible.
To address the problem of robust video decoding and increase the quality of the reconstructed frames, \ac{ec}-aware resilience techniques were investigated.
To extend existing techniques, a saliency-based \ac{ec} optimisation was proposed to optimally select the \ac{ec} which should be applied at the decoder in case of frame loss. This information is transmitted to the decoder as side-information.
Besides a saliency estimation, a coefficient based on the temporal duration of the error propagation was also considered, in order to keep the signalling overhead affordable. Finally, a small bit allocation was used to transmit an \ac{el}, which was used to improve the reconstruction quality, in those regions where the optimal \ac{ec} was not accurate.
Experimental results revealed that using different \ac{ec} algorithms increase the performance of the \ac{ec}-aware coding scheme and achieve superior performance when compared with existing methods.
Finally, the problem of adaptive robust coding was investigated, in order to achieve superior quality with the error robustness techniques proposed in this research work.
Preliminary results showed that the robust coding technique which obtains the higher quality depends on different factors, such as video content, bitrate and network conditions.
Therefore, an adaptive scheme based on six input parameters and a pre-trained \ac{nn} was proposed, to predicts the video stream that is able to achieve the highest decoded quality.
Results revealed that using the proposed scheme, an efficient trade-off between coding efficiency and error resilience is achieved for the majority of test conditions.
Using this architecture, the problem of robust video coding and decoding is addressed using a combination of different methods, which leads to a superior performance when compared with individual use of a single technique.
As a final remark, one may conclude that the use of novel methods, proposed in the context of this research work, are able to increase the robustness to transmission errors of the state-of-the-art video coding standard.
This was jointly achieved by achieving implementing robust video coding techniques with efficient error concealment algorithms optimised at the encoder.
\subsection*{Future work}
In this research work different methods were proposed to increase the error robustness of video streams and to enhance the quality achieved by \ac{ec} methods are the decoder.
In the context of this thesis there are also some suggestions for future research directions which can be used to strength the overall performance.
In the context of robust coding techniques, one can combine existing error resilience techniques, e.g., intra-refreshing constrains, in the Lagrangian optimisation used in the reference picture selection proposed in Chapter~\ref{Chapter:HEVCErrorResilience}.
This way it would be possible to further prevent error propagation by introducing extra refresh points.
Moreover, one can also use more advanced constrains, such as, end-to-end based optimisations, to achieve a more efficient distribution of the use of reference frames.
Regarding the \ac{ec}-aware method proposed in Chapter~\ref{Chapter:ConcealmentAwareTechniques} there are also some suggestions of future research directions. Firstly, as the proposed does not constrain the \ac{ec} candidate, one can use more advanced algorithms, in order to improve the reconstruction accuracy. This can be adopted either by replacing the existing methods, but also extending beyond four candidates.
Alternatively, to reduce the computational complexity overhead at the encoder, instead of measuring the \ac{rd} cost for each method, pre-trained classification algorithms could be used to predict the optimal \ac{ec} mode for each block.
As the complexity of the decoder is also a relevant issue, due to the increasing of mobile and battery-based devices, the optimal \ac{ec} selection should also take into account the end devices, thus penalising high complexity algorithms (e.g., algorithms requiring an estimated motion field) in case of constraining devices.
The performance of the adaptive selection of the robust coding technique could also be improved by combining different classification algorithms, based on more complex neural network architectures.
The error robustness schemes proposed in this research work could be extended to cope with robust transmission of multi-view video, where error propagate in the temporal and inter-layer domains.
Finally, as a new coding standard is currently being developed, targeting the deployment of higher-quality video services and emerging applications such as 360-degree omnidirectional immersive multimedia and high-dynamic-range video, it is emergent to study the impact of errors in such video applications and propose efficient techniques to provide high quality video to the end users.