jfmcarreira/doc_phd_thesis
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Review chapter 8

Browse Source
This commit is contained in:
jfmcarreira
2018-12-15 01:42:08 +00:00
parent 15f95a314b
commit 77cfe2f79d
1 changed files with 28 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files
050_Conclusions/Conclusion.tex
+28 -28
View File
@@ -3,45 +3,45 @@
\label{Chapter:Conclusions}
This thesis investigated new algorithms and techniques for efficient error robustness in highly efficiency video coders.
The importance of this research work is justified by the widespread of digital video content, including high resolution formats, multi-view and 360-degree video formats, which significantly increase the network bandwidth requirements.
The importance of this research work is justified by the widespread of digital video content, including high resolution formats, multi-view and 360-degree video formats, which significantly increase the network bandwidth requirements.
Consequently, errors and data loss may occur along the transmission system, despite the fact that video coding standards have been improved, in order to meet the network bandwidth constrains.
The use of advanced predictions techniques, results in increased bitstream dependencies with negative impact in the effect of errors, leading to higher quality degradation.
In this thesis different methods were proposed to address this problem, by investigating robust coding and efficient \ac{ec}.
In this thesis different methods were proposed to address this problem, by investigating robust coding and efficient \ac{ec}.
%Subsequently, for each chapter of this thesis, the main conclusions are presented.
Firstly, an introduction of the main motivation for this research work was presented, enumerating its main objectives and contributions.
Then, the main coding tools of the state-of-the-art \ac{hevc} were described.
This description covered the high-level features, such as the new picture types and the slice partitioning schemes, the new block partitioning scheme, which allow multi-level hierarchical structures with size-independent representation, the main predictions techniques, including the newly introduced advanced motion vector prediction technique.
Then, the main coding tools of the state-of-the-art \ac{hevc} were described.
This description covered the high-level features, such as the new picture types and the slice partitioning schemes, the new block partitioning scheme, which allow multi-level hierarchical structures with size-independent representation, the main predictions techniques, including the newly introduced advanced motion vector prediction technique.
This provided the necessary knowledge about the novel techniques in \ac{hevc}, which may have a relevant impact on the error robustness of the coded streams.
The main algorithms proposed in the past to increase the error robustness of video streams were also presented, describing techniques based on error resilience \ac{rd} optimisation, redundant based approaches and \ac{ec}-based methods.
After a simulation study it was concluded that, although the previous works provide efficient techniques to reduce the bitstream dependencies and ease the error recovery, vulnerabilities of the \ac{hevc} standard could not be fully addressed with existing techniques.
The main problems range from new \ac{mv} coding techniques to the lack of robust coding mechanisms, such as \ac{fmo} or data partitioning.
Moreover, there were also open issues regarding robust \ac{ec}-based encoding, as existing methods do not fully exploit the use of different techniques with different levels of complexity, especially the use of advanced \ac{ec}-based approaches.
The main algorithms proposed in the past to increase the error robustness of video streams were also presented, describing techniques based on error resilience \ac{rd} optimisation, redundant based approaches and \ac{ec}-based methods.
After a simulation study it was concluded that, although previous works provide efficient techniques to reduce the bitstream dependencies and ease the error recovery, vulnerabilities of the \ac{hevc} standard could not be fully addressed with existing techniques.
The main problems range from new \ac{mv} coding techniques to the lack of robust coding mechanisms, such as \ac{fmo} or data partitioning.
Moreover, there were also open issues regarding robust \ac{ec}-based encoding, as existing methods do not fully exploit the use of different techniques with different levels of complexity, especially the use of advanced \ac{ec}-based approaches.
To investigate new techniques for robust video coding an evaluation study was firstly carried out, with the aim of finding the influence of the new coding tools on the error resilience of the \ac{hevc} streams.
To investigate new techniques for robust video coding an evaluation study was firstly carried out, with the aim of finding the influence of the new coding tools on the error resilience of the \ac{hevc} streams.
Existing studies were limited to generic comparisons between \ac{hevc} and previous standards (e.g., \ac{h264}).
This study extended existing ones by considering different coding configurations, which allowed to evaluate the impact of new coding tools.
The study revealed that some features, such as increased block size, do not have significant impact on the error robustness performance.
However, slice partitioning and intra-refreshing have a noticeable impact on both, coding efficiency and error robustness.
This led to the insight that a small compromise in coding efficiency should be reasoned, in order to achieve an acceptable quality in case of errors.
This study extended existing ones by considering different coding configurations, which allowed to evaluate the impact of new coding tools.
The study revealed that some features, such as increased block size, do not have significant impact on the error robustness performance.
However, slice partitioning and intra-refreshing have a noticeable impact on both coding efficiency and error robustness.
This led to the insight that a small compromise in coding efficiency should be accepted, in order to achieve an higher quality in case of errors.
Finally, one could also conclude that the use of temporal predictions in \ac{mv} coding significantly increases temporal dependencies, leading to higher quality degradation and compromising the performance of \ac{hevc}-based video transmission systems.
Based on the above evidences, a new \ac{mv} coding scheme was devised, aiming at reducing the temporal dependencies.
In this scheme the use of the temporal \ac{mv} candidate is selectively used in some \acs{pu}, preventing temporal dependencies across several frames.
As this method significantly reduces the temporal dependencies, the number of mismatched \acs{mv} is decreases.
Moreover, as the temporal candidate is not fully disabled the impact on coding efficiency was shown to be low.
Based on the above findings, a new \ac{mv} coding scheme was devised, aiming at reducing the temporal dependencies.
In this scheme the use of the temporal \ac{mv} candidate is selectively used in some \acs{pu}, preventing temporal dependencies across several frames.
As this method significantly reduces the temporal dependencies, the number of mismatched \acs{mv} is decreases.
Moreover, as the temporal candidate is not fully disabled the impact on coding efficiency was shown to be low.
Experimental results demonstrated that a clear improvement on decoded video quality is achieved, across different video content, with a small bitrate increase.
Another investigation carried out in this thesis was focused on the reference frame distribution in highly efficient video coders.
Since in general, the reference frame usage is unbalanced, a new method based on constrained \ac{rd} optimisation was proposed to equally distribute the use of multiple reference frames across neighbouring \acs{pu}.
Moreover, the use of selective \ac{mv} redundancies was also investigated, in order to recover from erroneous motion predictions.
This resulted in a two-fold approach to reduce error propagation.
Since in general, the reference frame usage is unbalanced, a new method based on constrained \ac{rd} optimisation was proposed to equally distribute the use of multiple reference frames across neighbouring \acs{pu}.
Moreover, the use of selective \ac{mv} redundancies was also investigated, in order to recover from erroneous motion predictions.
This resulted in a two-fold approach to reducing error propagation.
As part of the proposed method can be applied at the streaming-stage, using a low complexity parser, it can be used to adapt pre-encoded streams to error-prone transmission channels.
The proposed method was compared against existing techniques, and the simulation results revealed a consistently superior performance of using the two-fold approach for different packet loss ratios.
To address the problem of robust video decoding and increase the quality of the reconstructed frames, \ac{ec}-aware resilience techniques were also investigated.
A saliency-based \ac{ec} optimisation was proposed to optimally select the \ac{ec} method that should be applied at the decoder in case of frame loss.
A saliency-based \ac{ec} optimisation was proposed to optimally select the \ac{ec} method that should be applied at the decoder in case of frame loss.
This information is transmitted to the decoder as side-information.
To keep the signalling overhead affordable, saliency estimation and a parameter based on the temporal duration of the error propagation
Then, a limited amount of overhead was used to encode a residual information in an \ac{el}, which is used to improve the reconstruction quality, in those regions where the optimal \ac{ec} was not accurate.
@@ -49,11 +49,11 @@ Experimental results revealed that using different \ac{ec} algorithms contribute
Finally, the problem of adaptive robust coding was investigated, in order to achieve superior quality when different error robustness techniques are available and dynamic conditions are taken into account.
Simulation results showed that the robust coding technique which obtains the higher quality depends on different factors, such as video content, bitrate and network conditions.
Therefore, an adaptive scheme based on six input parameters and a pre-trained \ac{nn} was proposed, to predicts the video stream that is able to achieve the highest decoded quality, under transmission errors.
Therefore, an adaptive scheme based on six input parameters and a pre-trained \ac{nn} was proposed, to predicts the video stream that is able to achieve the highest decoded quality, under packet loss environments.
Results revealed that using the proposed adaptive scheme, an efficient trade-off between coding efficiency and error resilience is achieved for the majority of test conditions.
Using this architecture, the problem of robust video coding and decoding was addressed using a combination of different methods, which leads to a superior performance in comparison with individual techniques.
As a final remark, one may conclude that the use of the novel methods proposed in this thesis consistently contribute to increase the robustness of video communications using state of the art high efficiency video encoding tools are error prone networks.
As a final remark, one may conclude that the use of the novel methods proposed in this thesis consistently contributes to increase the robustness of video communications using state of the art high efficiency video encoding tools in error prone networks.
The best performance is achieved by combining robust video coding techniques with efficient error concealment algorithms optimised at the encoder, using an optimised dynamic approach.
@@ -61,19 +61,19 @@ The best performance is achieved by combining robust video coding techniques wit
In the context of this thesis there are some open issues for future research directions which can be investigated to further strength the overall performance.
In the context of robust coding, one can further combine existing error resilience techniques, e.g., intra-refreshing constrains, in the Lagrangian optimisation used in the reference picture selection proposed in Chapter~\ref{Chapter:HEVCErrorResilience}.
This would possibly allow stronger limitation of error propagation by introducing extra refresh points.
In the context of robust coding, one can further combine existing error resilience techniques, e.g., intra-refreshing constrains, in the Lagrangian optimisation used in the reference picture selection proposed in Chapter~\ref{Chapter:HEVCErrorResilience}.
This would possibly allow stronger limitation of error propagation by introducing extra refresh points.
Moreover, one can also use more advanced constrains, such as, end-to-end based optimisations, to devise more efficient methods for distribution of the use of reference frames.
Regarding the \ac{ec}-aware method proposed in Chapter~\ref{Chapter:ConcealmentAwareTechniques} there are also some possible improvements to be addressed in future research.
Firstly, more advanced algorithms ca be developed on a \ac{ctu} basis, in order to improve the reconstruction accuracy.
Firstly, more advanced algorithms can be developed on a \ac{ctu} basis, in order to improve the reconstruction accuracy.
This can be implemented either by replacing the existing methods, or by extending beyond the four candidates that were used in this work.
Alternatively, to reduce the computational complexity overhead at the encoder, instead of measuring the \ac{rd} cost for each method, pre-trained classification algorithms can be investigated to predict the optimal \ac{ec} mode for each block.
Alternatively, to reduce the computational complexity overhead at the encoder, instead of measuring the \ac{rd} cost for each method, pre-trained classification algorithms can be investigated to predict the optimal \ac{ec} mode for each block.
As the complexity of the decoder is also a relevant issue, due to the increasing of mobile and battery-based devices, the optimal \ac{ec} selection should also take into account the end devices, thus penalising high complexity algorithms (e.g., algorithms requiring an estimated motion field) in case of constrained devices.
The performance of the adaptive selection of the robust coding technique can also be improved by combining different classification algorithms, based on more complex neural network architectures.
Furthermore, the error robustness schemes proposed in this research work can be extended to cope with robust transmission of multi-view video, where errors propagate in the temporal and inter-layer domains.
Finally, as a new coding standard is currently being developed, targeting the deployment of higher-quality video services and emerging applications such as 360-degree multimedia and high-dynamic-range video, it is also an open direction for research the study of the impact of errors in such video applications and the development of novel efficient techniques to guarantee high quality visual experiences, to users of such emerging video formats.
Finally, as a new coding standard, entitled Versatile Video Coding, is currently being developed, targeting the deployment of higher-quality video services and emerging applications such as \HIGHLIGHT{360-degree multimedia, where the relative importance of different view points could be used to restrict the application of the proposed techniques} or high-dynamic-range video. This opens new research directions to the study the impact of errors in such video applications and the development of novel efficient techniques to guarantee high quality visual experiences, to users of such emerging video formats.