Review all chapters and start conclusions

This commit is contained in:
jfmcarreira committed 2018-09-12 02:44:20 +01:00
1 parent 461b560b17
commit 8d3a4c72ff
20 files changed
+304 -427

No files matched your search

+54 -30
View File
@@ -1,37 +1,61 @@
\chapter{\HIGHLIGHT{Conclusion and future work}}
\chapter{Conclusion and future work \VERSION{0}}
\label{Chapter:Conclusions}
This chapter concludes this thesis, presenting some conclusions about this research work and some future research perspectives in the area of video transmission, error resilience and error concealment.
\glsresetall
\glsunset{h264}
%\vspace{10mm}
\section{Conclusions}
% This section presents a summary of the of the chapters presented before and a discussion of the contributions presented.
%
% Chapter~\ref{CHAP:3DVideo} presented a review on the main technologies for 3D video systems related with the formats for representation, coding methods and proposed storage and transmission systems. This chapter also describes the main features of the MVC extension of the H.264/MVC standard, which is the coding standard for the most 3D applications \mbox{(\emph{e.g.}, Blu-ray)}. Moreover, as this research focuses in transmission errors and error concealment, a review of some concealment methods proposed over the last years was presented in the Chapter~\ref{CHAP:Concealment}.
%
% The research work presented in this thesis leaded
% %different contributions
% to the development of an advanced 3D video transmission system. More specifically, our research focused on the study of the subjective impacts of transmission errors in 3D video, when the auxiliary view is subject to frame loss. These results were presented in Chapter~\ref{CHAP:SubjectiveEval}. The results demonstrated that the disparity is intrinsically related with the quality of the concealed 3D video, and that errors have higher effects in those sequences with higher disparity. Moreover, for longer error bursts, the results showed that it is better to switch into 2D visualisation rather than trying to restore the 3D effect by using frame concealment methods.
%
% The results presented in the Chapter~\ref{CHAP:SubjectiveEval} leaded to further experiments in order to evaluate the objective quality of the transmitted 3D video using different objective metrics. A transmission framework was presented in Chapter~\ref{CHAP:HierarchicalDVBT3D} and the quality evaluation was performed. The quality models, devised from the analysis of the quality results, are a novel contribution to the research community and also for the network planners. Through those models, estimated values of the quality using different metrics can be obtained from the packet loss ratio of the transmission channel. This can be easily measured at the receiver side. The quality models are able to cover different video quality features, as the signal-to-noise relation, the structural similarity and a more specific feature of the 3D video, the stereo perception.
% The tools developed are part of the first contribution of this research work, which resulted in an efficient 3D receiver to cope with transmission errors in 3D video streams, delivered using the MPEG-2 TS encapsulation. Moreover, as part of the receiver side, an efficient 3D video decoder was achieved, that is able to detect and conceal the missing frames using different concealment techniques. This set of tools provides relevant knowledge, for further investigations in quality evaluation and error concealment.
%
% The second contribution of this research work was the development of efficient concealment methods, able to recover the motion field for missing frames in 3D video. Assuming the hierarchical transmission system, as presented in Chapter~\ref{CHAP:HierarchicalDVBT3D}, the proposed methods are an efficient alternative to the existing ones. They are mainly based on the information presented in the error-free view (\emph{i.e.}, the base view), since these information can provide more accurate clues to estimate the loss data in the auxiliary view, as described in Chapter~\ref{CHAP:PropFrameLossConc}. The presented results showed the effectiveness of the proposed techniques, when compared with conventional motion-based algorithms, achieving an average gain of $1.89\ dB$ in random frame loss and up to $6\ dB$ of gain in the objective quality of the reconstructed frame. Moreover, an improvement to the proposed method was devised, and a motion based decision approach was incorporated, combining both methods, thus allowing to improve the quality of the reconstructed frames up $2\ dB$.
%
% One may conclude that the research on alternative frame loss concealment techniques may bring new insights to achieve a more efficiently video decoding in the presence of error prone channels. Moreover, the subjective study presented in this dissertation provided relevant subjective factors that may be used in future developments of error concealment techniques. New representation formats for 3D video and novel coding approaches are being proposed, leading to new error concealment problems, and different ways to recovery the missing information, thus, the research work in this area should continue.
This thesis investigated new algorithms and techniques for efficient error robustness in highly efficiency video coders.
The importance of these techniques is justified by the spread of digital video contents, including high resolution formats, multi-view and 360-degree video formats, which significantly the network bandwidth requirements. Consequently, errors and data loss may occur along the transmission system.
Moreover, in order to meet the network bandwidth constrains video coding standard have been improved, by including advanced predictions techniques, which results in increased bitstream dependencies.
This increase the negative impact of errors, leading to higher quality degradation.
To overcome this problem, in this thesis different methods were proposed, addressing the problem of robust coding and efficient \ac{ec}.
%Subsequently, for each chapter of this thesis, the main conclusions are presented.
\section{Future work}
Firstly, an introduction of the main motivation for this research work is presented, enumerating its main objectives and contributions.
Then, the main functional blocks of the state-of-the-art \ac{hevc} are described.
This description covered the high-level features, such as the new picture types and the slice partitioning schemes, the new block partitioning scheme, which allow multi-level hierarchical structures with size-independent representation, the main predictions techniques, including the newly introduced advanced motion vector prediction technique.
This aimed at introducing the novel techniques in \ac{hevc}, which may have a relevant impact on the error robustness of the coded streams.
The main algorithms proposed in the past to increase the error robustness of video streams were presented, describing techniques based on error resilience \ac{rd} optimisation, redundant based approaches and \ac{ec}-based methods.
It was concluded that, although the previous works provide efficient techniques to reduce the bitstream dependencies and ease the error recovery, the \ac{hevc} standard introduces new problems that could not be fully addressed with existing techniques.
Such problems range from new \ac{mv} coding techniques to the lack of robust coding techniques, such as \ac{fmo} or data partitioning.
Moreover, there was also open issues regarding robust \ac{ec}-based encoding, as existing method do not fully exploit the use of different methods with different levels of complexity, especially the use of advanced \ac{ec} techniques.
% In this research work, the technologies and the subjective quality impact of errors in 3D video were studied. Nevertheless, there are still a number of topics that require further investigation, and may lead to more efficient 3D video decoders, namely:
% \begin{itemize}
% \item \textbf{3D video quality models extension to the channel level.} The 3D video quality models presented in Chapter~\ref{CHAP:HierarchicalDVBT3D} revealed to match with the experimental data. Thus, the combination with the hierarchical DVB-T channel models could extend the proposed models to a new level, where the quality of the 3D video could be estimated from the channel conditions, instead of the packet loss. This would extend the relevance of the proposed models to the network planners and developers.
%
% \item \textbf{Extension of the proposed method to a multi-view scenario.} The proposed concealment methods, described in Chapter~\ref{CHAP:PropFrameLossConc}, revealed good performance to conceal missing frames in 3D video (\emph{i.e.}, stereoscopic video). Therefore, the same techniques are expected to have similar behaviour under a multi-view scenario, where, besides the base view frames, other auxiliary view frames that may provide motion information clues to estimate the lost frames' motion field.
%
% \item \textbf{Extend the concealment algorithm to MVD coding scenario.} Although the proposed concealment methods did not focus in the recovery of the depth information, they could be used to recover texture in multi-view plus depth video signals. In this case, besides disparity information, depth maps are available to extract relevant information, for example, to detect occlusions and errors in the disparity map. The performance of the proposed method could also be investigated to reconstruct the lost depth map.
%
% \item \textbf{Performance evaluation of the concealment methods in a MDC.} The multi-description coding approach is being investigated over the last years, as a novel alternative for the emergent multi-path networks. The algorithms presented in the proposed method have great flexibility and could be applied under different coding scenarios. Moreover, since MDC introduces new possibilities to error concealment by using the redundant information in each description, such characteristic will be used to investigate how different types of redundancy can be jointly used with the proposed concealment techniques to enhance the quality of received views.
% \end{itemize}
To investigate new techniques for robust video coding an evaluation study was firstly carried out, which aimed at finding the influence of the new coding tools on the error resilience of the \ac{hevc} streams.
Existing studies only performed generic comparisons between \ac{hevc} and previous standards (e.g., \ac{h264}).
This study extended existing ones by considering different coding configurations, which allowed to evaluate the impact of new coding tools. The study revealed that some features, such as increased block size, does not have significant impact of the error robustness performance.
However, slice partitioning and intra-refreshing have a noticeable impact on both, coding efficiency and error robustness.
Therefore, a small compromise in coding efficiency should be reasoned, in order to achieve an acceptable quality in case of errors.
Finally, one can also conclude that the use of temporal predictions in \ac{mv} coding significantly increases temporal dependencies, leading to increased quality degradation compromising the performance of an \ac{hevc}-based video transmission system.
Based on the above evidences, a new \ac{mv} coding scheme was used in \ac{hevc}, aiming at reducing the temporal dependencies.
In this scheme the use of the temporal \ac{mv} candidate is selectively used in some \acs{pu}, preventing temporal dependencies across several frames.
As this method significantly reduces the temporal dependencies, the number of mismatched \acs{mv} is reduced, consequently reducing the error propagation.
Moreover, as the temporal candidate is not fully disabled the impact on coding efficiency is reduced.
Experimental results showed a clear improvement on decoded video quality, across different video content, with a small bitrate increase.
The reference frame distribution in highly efficient video coders was investigated, as it may increase the number of mismatch predictions in case of erroneous reference frames.
Since the reference frame usage is consistently unbalanced, a new method based on constrained \ac{rd} optimisation was proposed to equally distribute the use of multiple reference frames across neighbouring \acs{pu}.
An extension using selective \ac{mv} redundancies was also investigated, in order to recover from erroneous motion predictions.
This resulted in a two-fold approach to reduce error propagation.
As part of the proposed method can be applied at the streaming-stage, using a low complexity parser, this method can be used to adapt existing streams to error-prone transmission channels.
The proposed method was compared against existing techniques, and the simulation results revealed a consistently superior performance of using the two-fold approach, making it suitable for video services where packet loss cannot be negligible.
To address the problem of robust video decoding and increase the quality of the reconstructed frames, \ac{ec}-aware resilience techniques were investigated.
To extend existing techniques, a saliency-based \ac{ec} optimisation was proposed to optimally select the \ac{ec} which should be applied at the decoder in case of frame loss. This information is transmitted to the decoder as side-information.
Besides a saliency estimation, a coefficient based on the temporal duration of the error propagation was also considered, in order to keep the signalling overhead affordable. Finally, a small bit allocation was used to transmit an \ac{el}, which was used to improve the reconstruction quality, in those regions where the optimal \ac{ec} was not accurate.
Experimental results revealed that using different \ac{ec} algorithms increase the performance of the \ac{ec}-aware coding scheme and achieve superior performance when compared with existing methods.
Finally, the problem of adaptive robust coding was investigated, in order to achieve superior quality with the error robustness techniques proposed in this research work.
Preliminary results showed that the robust coding technique which obtains the higher quality depends on different factors, such as video content, bitrate and network conditions.
Therefore, an adaptive scheme based on six input parameters and a pre-trained \ac{nn} was proposed, to predicts the video stream that is able to achieve the highest decoded quality.
Results revealed that using the proposed scheme, an efficient trade-off between coding efficiency and error resilience is achieved for the majority of test conditions.
Using this architecture, the problem of robust video coding and decoding is addressed using a combination of different methods, which leads to a superior performance when compared with individual use of a single technique.
As a final remark, one may conclude that the use of novel methods, proposed in the context of this research work, are able to increase the robustness to transmission errors of the state-of-the-art video coding standard.
This was jointly achieved by achieving implementing robust video coding techniques with efficient error concealment algorithms optimised at the encoder.
\subsection*{Future work}