jfmcarreira
4ebe3013c0
INSTRUCTIONS Px stands for paragraph number GENERAL CORRECTIONS Fix the reference of equation "in Equation (5.2)" -> "in (5.2)" ACKNOWLEDGMENTS: Replace Dr. with Prof. where appropriate. Add reference to Loughborough University London. ABSTRACT: Remove abbreviations. Add MPEG-DASH, MOS, HM to the list. Replaced LTE with LTE-A. LIST OF ABBREVIATIONS: Remove trailing dot and add page where it is defined in the text. CHAPTER 1 Section 1.1 P1: internet -> Internet, added mention to 360-degree video and Facebook. P2: Remove sentence that was lacking reference: "However, it is known that higher coding efficiency leads to lower robustness in the presence of transmission errors and data loss." Subsequent sentence expresses the same idea as it includes reference already. P2: Add an objective score to the "poor error resilience". Section 1.2 P1: Capitalise the first letter of each objective. P1: Fix "error robust" spelling error. Section 1.3 P2: Replace "with" by "to" and efficiently with efficient. P7: Replace "material" with "materials". CHAPTER 2 Section 2.1 P1: Replace "in" with "on". Section 2.2 P1: Fix three spelling errors. P3: Replace "between" with "among" and add missing "a". P4: Add missing "a". Figure 2.1: Add description of T and Q. Section 2.3 P1: Replace "complaint" with "conforming". P2: Add missing either. Figure 2.2: Change figure to avoid confusing in arrow direction. Section 2.4 P1: Fix typo in "pictures types". Section 2.4.2 P1: Remove last part of the second sentence. P2: Replace “with” by “to”. Section 2.4.4 P1: Add missing “that”. P7: Fix two spelling errors. Section 2.5 P2: Add missing pixels. Figure 2.8: Correct the coding order Section 2.6 P1: Fix three typos. Section 2.6.1 P1: Fix third sentence to: “When the intra prediction mode is chosen for a CU, the size of the PU needs to be the same as the CU, except when the smallest CU size is selected and further division into four PUs is allowed.” P7: Fix “three-tap”. P8 (page 24): The last sentence of the paragraph was revised to: “Only the mentioned boundaries are filtered, as they are the ones most likely to introduce discontinuities.” Section 2.6.2 P1: Replace “rely” with “relies”. P2: Replace twice “in” to “of”. P2: Replace “different” with “difference”. P3: Fix two typos. P4: Add HM acronym definition. P6: Replace “not” with “no”. Table 2.2: Remove acronym in the title. Figure 2.11: Review figure and caption to become self-contained. Section 2.7 P1: remove comma after Similarly. P3: Replace “intra coded” with “intra-coded”. P4: Replace last sentence with “This procedure is the main cause for the coding distortion, as it performs a many-to-one mapping, which cannot be reversed.” Section 2.9 P1: Revise the third sentence to: “Such grid-based restriction reduces the computational complexity and facilitates the parallel-processing, which is one of the main goals of HEVC design.” P2: Revise the third sentence: “For each CTU, the encoder decides whether to apply the SAO filter or not, and if used, one of the two filter types is applied, namely the band offset or the edge offset.”
82 lines
10 KiB
TeX
82 lines
10 KiB
TeX
|
|
\chapter{Conclusion and future work \VERSION{3}}
|
|
\label{Chapter:Conclusions}
|
|
|
|
This thesis investigated new algorithms and techniques for efficient error robustness in highly efficiency video coders.
|
|
The importance of this research work is justified by the widespread of digital video content, including high resolution formats, multi-view and 360-degree video formats, which significantly increase the network bandwidth requirements.
|
|
Consequently, errors and data loss may occur along the transmission system, despite the fact that video coding standards have been improved, in order to meet the network bandwidth constrains.
|
|
The use of advanced predictions techniques, results in increased bitstream dependencies with negative impact in the effect of errors, leading to higher quality degradation.
|
|
In this thesis different methods were proposed to address this problem, by investigating robust coding and efficient \ac{ec}.
|
|
%Subsequently, for each chapter of this thesis, the main conclusions are presented.
|
|
|
|
|
|
Firstly, an introduction of the main motivation for this research work was presented, enumerating its main objectives and contributions.
|
|
Then, the main coding tools of the state-of-the-art \ac{hevc} were described.
|
|
This description covered the high-level features, such as the new picture types and the slice partitioning schemes, the new block partitioning scheme, which allow multi-level hierarchical structures with size-independent representation, the main predictions techniques, including the newly introduced advanced motion vector prediction technique.
|
|
This provided the necessary knowledge about the novel techniques in \ac{hevc}, which may have a relevant impact on the error robustness of the coded streams.
|
|
The main algorithms proposed in the past to increase the error robustness of video streams were also presented, describing techniques based on error resilience \ac{rd} optimisation, redundant based approaches and \ac{ec}-based methods.
|
|
After a simulation study it was concluded that, although the previous works provide efficient techniques to reduce the bitstream dependencies and ease the error recovery, vulnerabilities of the \ac{hevc} standard could not be fully addressed with existing techniques.
|
|
The main problems range from new \ac{mv} coding techniques to the lack of robust coding mechanisms, such as \ac{fmo} or data partitioning.
|
|
Moreover, there were also open issues regarding robust \ac{ec}-based encoding, as existing methods do not fully exploit the use of different techniques with different levels of complexity, especially the use of advanced \ac{ec}-based approaches.
|
|
|
|
To investigate new techniques for robust video coding an evaluation study was firstly carried out, with the aim of finding the influence of the new coding tools on the error resilience of the \ac{hevc} streams.
|
|
Existing studies were limited to generic comparisons between \ac{hevc} and previous standards (e.g., \ac{h264}).
|
|
This study extended existing ones by considering different coding configurations, which allowed to evaluate the impact of new coding tools.
|
|
The study revealed that some features, such as increased block size, do not have significant impact on the error robustness performance.
|
|
However, slice partitioning and intra-refreshing have a noticeable impact on both, coding efficiency and error robustness.
|
|
This led to the insight that a small compromise in coding efficiency should be reasoned, in order to achieve an acceptable quality in case of errors.
|
|
Finally, one could also conclude that the use of temporal predictions in \ac{mv} coding significantly increases temporal dependencies, leading to higher quality degradation and compromising the performance of \ac{hevc}-based video transmission systems.
|
|
|
|
Based on the above evidences, a new \ac{mv} coding scheme was devised, aiming at reducing the temporal dependencies.
|
|
In this scheme the use of the temporal \ac{mv} candidate is selectively used in some \acs{pu}, preventing temporal dependencies across several frames.
|
|
As this method significantly reduces the temporal dependencies, the number of mismatched \acs{mv} is decreases.
|
|
Moreover, as the temporal candidate is not fully disabled the impact on coding efficiency was shown to be low.
|
|
Experimental results demonstrated that a clear improvement on decoded video quality is achieved, across different video content, with a small bitrate increase.
|
|
|
|
Another investigation carried out in this thesis was focused on the reference frame distribution in highly efficient video coders.
|
|
Since in general, the reference frame usage is unbalanced, a new method based on constrained \ac{rd} optimisation was proposed to equally distribute the use of multiple reference frames across neighbouring \acs{pu}.
|
|
Moreover, the use of selective \ac{mv} redundancies was also investigated, in order to recover from erroneous motion predictions.
|
|
This resulted in a two-fold approach to reduce error propagation.
|
|
As part of the proposed method can be applied at the streaming-stage, using a low complexity parser, it can be used to adapt pre-encoded streams to error-prone transmission channels.
|
|
The proposed method was compared against existing techniques, and the simulation results revealed a consistently superior performance of using the two-fold approach for different packet loss ratios.
|
|
|
|
To address the problem of robust video decoding and increase the quality of the reconstructed frames, \ac{ec}-aware resilience techniques were also investigated.
|
|
A saliency-based \ac{ec} optimisation was proposed to optimally select the \ac{ec} method that should be applied at the decoder in case of frame loss.
|
|
This information is transmitted to the decoder as side-information.
|
|
To keep the signalling overhead affordable, saliency estimation and a parameter based on the temporal duration of the error propagation
|
|
Then, a limited amount of overhead was used to encode a residual information in an \ac{el}, which is used to improve the reconstruction quality, in those regions where the optimal \ac{ec} was not accurate.
|
|
Experimental results revealed that using different \ac{ec} algorithms contributes to increase the performance of the \ac{ec}-aware coding scheme, achieving superior performance when compared with existing methods.
|
|
|
|
Finally, the problem of adaptive robust coding was investigated, in order to achieve superior quality when different error robustness techniques are available and dynamic conditions are taken into account.
|
|
Simulation results showed that the robust coding technique which obtains the higher quality depends on different factors, such as video content, bitrate and network conditions.
|
|
Therefore, an adaptive scheme based on six input parameters and a pre-trained \ac{nn} was proposed, to predicts the video stream that is able to achieve the highest decoded quality, under transmission errors.
|
|
Results revealed that using the proposed adaptive scheme, an efficient trade-off between coding efficiency and error resilience is achieved for the majority of test conditions.
|
|
Using this architecture, the problem of robust video coding and decoding was addressed using a combination of different methods, which leads to a superior performance in comparison with individual techniques.
|
|
|
|
As a final remark, one may conclude that the use of the novel methods proposed in this thesis consistently contribute to increase the robustness of video communications using state of the art high efficiency video encoding tools are error prone networks.
|
|
The best performance is achieved by combining robust video coding techniques with efficient error concealment algorithms optimised at the encoder, using an optimised dynamic approach.
|
|
|
|
|
|
\subsection*{Future work}
|
|
|
|
In the context of this thesis there are some open issues for future research directions which can be investigated to further strength the overall performance.
|
|
|
|
In the context of robust coding, one can further combine existing error resilience techniques, e.g., intra-refreshing constrains, in the Lagrangian optimisation used in the reference picture selection proposed in Chapter~\ref{Chapter:HEVCErrorResilience}.
|
|
This would possibly allow stronger limitation of error propagation by introducing extra refresh points.
|
|
Moreover, one can also use more advanced constrains, such as, end-to-end based optimisations, to devise more efficient methods for distribution of the use of reference frames.
|
|
|
|
Regarding the \ac{ec}-aware method proposed in Chapter~\ref{Chapter:ConcealmentAwareTechniques} there are also some possible improvements to be addressed in future research.
|
|
Firstly, more advanced algorithms ca be developed on a \ac{ctu} basis, in order to improve the reconstruction accuracy.
|
|
This can be implemented either by replacing the existing methods, or by extending beyond the four candidates that were used in this work.
|
|
Alternatively, to reduce the computational complexity overhead at the encoder, instead of measuring the \ac{rd} cost for each method, pre-trained classification algorithms can be investigated to predict the optimal \ac{ec} mode for each block.
|
|
As the complexity of the decoder is also a relevant issue, due to the increasing of mobile and battery-based devices, the optimal \ac{ec} selection should also take into account the end devices, thus penalising high complexity algorithms (e.g., algorithms requiring an estimated motion field) in case of constrained devices.
|
|
The performance of the adaptive selection of the robust coding technique can also be improved by combining different classification algorithms, based on more complex neural network architectures.
|
|
|
|
Furthermore, the error robustness schemes proposed in this research work can be extended to cope with robust transmission of multi-view video, where errors propagate in the temporal and inter-layer domains.
|
|
Finally, as a new coding standard is currently being developed, targeting the deployment of higher-quality video services and emerging applications such as 360-degree multimedia and high-dynamic-range video, it is also an open direction for research the study of the impact of errors in such video applications and the development of novel efficient techniques to guarantee high quality visual experiences, to users of such emerging video formats.
|
|
|
|
|
|
|
|
|
|
|