Content-adaptive Gradient-based Error Concealment Scheme for H.264/AVC

Document Type : Original Article

Authors

Faculty of Electrical and Computer Engineering, Noshirvani University of Technology, Babol, Iran

Abstract

A challenging aspect of video error concealment (VEC) is reducing blockiness around the missing region. In this paper, we have devised a novel method for calculating the boundary matching distortion based on the edge direction surrounding the lost area to deal with this problem. After detecting the corresponding slice’s erroneous Macroblock (MB), it is divided into four 8×8 pixels sub-blocks. Then, the gradient of outer boundary pixels is derived and determines each side’s smoothness, which is further used for drawing the hypothetical line. Furthermore, this paper proposes a novel optimization-based model that can accurately measure boundary matching error for loss recovery. We observe better results for our proposed technique than other related VEC algorithms in terms of PSNR and SSIM. The proposed algorithm improves the average PSNR by 1.09 dB and increases the average SSIM by 0.0135 at the packet loss rate of 10%. In addition, for a PLR of 20%, the PSNR is increased by 1.28 dB, and the SSIM enhancement is 0.019. This algorithm is slightly more computationally complex than the compared methods, but it is still acceptable. Thus, by adding a few computations to the video decoder, the proposed method maintains the quality of the video, especially in the rough, damaged regions of a decoded frame.

Keywords

Main Subjects

References

[1] A. J. Hussain and Z. Ahmed, “A survey on video compression fast block matching algorithms,” Neurocomputing, vol. 335, pp. 215–237, 2019.
[2] V. Tiwari and C. Bhatnagar, “A survey of recent work on video summarization: approaches and techniques,” Multimed Tools Appl, vol. 80, no. 18, pp. 27187–27221, 2021.
[3] International Standard ISO/IEC, “Recommendation ITU-T H.264,” 2017.
[4] Cisco, “Cisco Visual Networking Index: Forecast and Methodology,” Cisco White Paper, 2017. [Online]. Available:https://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/complete-white-paper-c11-481360.html. [Accessed Dec 1, 2022]
[5] J. He, Y. Xu, W. Luo, S. Tang, and J. Huang, “A novel selective encryption scheme for H.264/AVC video with improved visual security,” Signal Process Image Commun, vol. 89, p. 115994, 2020.
[6] R. Ma, T. Li, D. Bo, Q. Wu, and P. An, “Error sensitivity model based on spatial and temporal features,” Multimed Tools Appl, vol. 79, no. 43–44, pp. 31913–31930, 2020.
[7] R. Chellappa and S. Theodoridis, Academic Press Library in Signal Processing, Volume 6: Image and Video Processing and Analysis and Computer Vision. Academic Press, 2017.
[8] K. Go, M. Kim, S. Kang, and Y. Yoon, “A systematic reallocation and prioritization scheme for error-resilient transmission of video packets,” Multimed Tools Appl, vol. 76, no. 5, pp. 6755–6783, 2017.
[9] M. Usman, X. He, M. Xu, and K. M. Lam, “Survey of error concealment techniques: Research directions and open issues,” in 31st Picture Coding Symposium (PCS), pp. 233–238, 2015.
[10] C. Nam, C. Chu, T. Kim, and S. Han, “A novel motion recovery using temporal and spatial correlation for a fast temporal error concealment over H.264 video sequences,” Multimed Tools Appl, vol. 79, no. 1, pp. 1221–1240, 2020.
[11] H. R. Wu and K. R. Rao, Digital video image quality and perceptual coding. CRC press, 2017.
[12] D. R. Bull, Communicating pictures: A course in image and video coding. Academic Press, 2014.
[۱۳] محمود دی پیر و فرزاد گویا، "به کارگیری ارتباطات چندکاربره ی دستگاه به دستگاه برای تحویل بیسیم محتوای ویدئویی در شبکه های سلولی،" مجله مهندسی برق، دانشگاه تبریز، دوره پنجاهم شماره چهارم، صفحات ۱۵۸۱-۱۵۹۱، سال ۱۳۹۹.
[14] H. Byongsu, J. Jonghyon, and R. Cholsu, “An improved multi-directional interpolation for spatial error concealment,” Multimed Tools Appl, vol. 78, no. 2, pp. 2587–2598, 2019.
[15] H. Asheri, H. Rabiee, N. Pourdamghani, and M. Ghanbari, “Multi-directional spatial error concealment using adaptive edge thresholding,” IEEE Transactions on Consumer Electronics, vol. 58, no. 3, pp. 880–885, 2012
[16] H. Byongsu, J. Jonghyon, and R. Cholsu, “An improved exemplar-based image inpainting algorithm for error concealment,” ICTACT Journal on Image and Video Processing (IJIVP), vol. 8, no. 1, pp. 1583–1587, 2017.
[17] J. Koloda, J. Seiler, A. M. Peinado, and A. Kaup, “Scalable Kernel-Based Minimum Mean Square Error Estimator for Accelerated Image Error Concealment,” IEEE Transactions on Broadcasting, vol. 63, no. 1, pp. 59–70, 2017.
[18] H. C. Shih, C. T. Wang, and C.-L. Huang, “Spiral-Like Pixel Reconstruction Algorithm for Spatiotemporal Video Error Concealment,” IEEE Access, vol. 6, pp. 6370–6381, 2018.
[19] Mengyao Ma, O. C. Au, S.-H. G. Chan, and Ming-Ting Sun, “Edge-Directed Error Concealment,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 3, pp. 382–395, 2010.
[20] Jing Liu, Guangtao Zhai, Xiaokang Yang, Bing Yang, and Li Chen, “Spatial Error Concealment With an Adaptive Linear Predictor,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 3, pp. 353–366, 2015.
[21] S.-C. Hsia and C. H. Hsiao, “Fast-efficient shape error concealment technique based on block classification,” IET Image Process, vol. 10, no. 10, pp. 693–700, 2016.
[۲۲] مه لقا افراسیابی،حسن ختن لو و محرم منصوری زاده، "شبکه عصبی عمیق برای پیشبینی تعامل انسان در ویدئو با استفاده از روابط فازی و شار نوری"، مجله مهندسی برق، دانشگاه تبریز ، دوره پنجاهم، شماره سوم، صفحات ۱۰۳۵-۱۰۴۶، سال ۱۳۹۹.
[23] G. Choe, C. Nam, and C. Chu, “An effective temporal error concealment in H.264 video sequences based on scene change detection-PCA model,” Multimed Tools Appl, vol. 77, no. 24, pp. 31953–31967, 2018.
[24] D. Kim, Y. Kwon, and K. Choi, “Motion‐Vector Refinement for Video Error Concealment Using Downhill Simplex Approach,” ETRI Journal, vol. 40, no. 2, pp. 266–274, 2018.
[25] Y. Li and R. Chen, “Motion vector recovery for video error concealment based on the plane fitting,” Multimed Tools Appl, vol. 76, no. 13, pp. 14993–15006, Jul. 2017.
[26] A. Radmehr and A. Ghasemi, “Error concealment via particle filter by Gaussian mixture modeling of motion vectors for H.264/AVC,” Signal Image Video Process, vol. 10, no. 2, pp. 311-318, 2016.
[27] T. L. Lin, T. L. Ding, C. Y. Fan, and W. C. Chen, “Error concealment algorithm based on sparse optimization,” Multimed Tools Appl, vol. 76, no. 1, pp. 397–413, 2017.
[28] M. Kazemi, M. Ghanbari, and S. Shirmohammadi, “A review of temporal video error concealment techniques and their suitability for HEVC and VVC,” Multimedia Tools and Applications, vol. 80, no. 8, pp. 12685–12730, 2021.
[29] M. Kazemi, M. Ghanbari, and S. Shirmohammadi, “The Performance of Quality Metrics in Assessing Error-Concealed Video Quality,” IEEE Transactions on Image Processing, vol. 29, pp. 5937–5952, 2020.
[30] B. Chung and C. Yim, “Bi-Sequential Video Error Concealment Method Using Adaptive Homography-Based Registration,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 6, pp. 1535–1549, 2020.
[31] Y. Zhang, X. Xiang, D. Zhao, S. Ma, and W. Gao, “Packet video error concealment with auto regressive model,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 12, no. 1, pp. 12–27, 2012.
[32] C. Xiang, J. Xu, C. Yan, Q. Peng, and X. Wu, “Generative adversarial networks based error concealment for low resolution video,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1827–1831, 2019.
[33] A. Sankisa, A. Punjabi, and A. K. Katsaggelos, “Video Error Concealment Using Deep Neural Networks,” in 25th IEEE International Conference on Image Processing (ICIP), pp. 380–384, 2018.
[34] S. Chapaneri, S. Mistry, and S. Dixit, “Performance Evaluation of Edge-based Video Error Concealment using H. 264 Flexible Macroblock Ordering,” International Journal of Computer Science & Engineering Technology (IJCSET), vol. 3, no. 12, pp. 605–615, 2015.
[35] J. Korhonen, “Study of the subjective visibility of packet loss artifacts in decoded video sequences,” IEEE Transactions on Broadcasting, vol. 64, no. 2, pp. 354–366, 2018.
[36] T.-H. Tsai, S.-S. Su, and T.-Y. Lee, “Fast mode decision method based on edge feature for HEVC inter-prediction,” IET Image Process, vol. 12, no. 5, pp. 644–651, 2018.
[37] T.-L. Lin, X. Wei, X. Wei, T.-H. Su, and Y.-L. Chiang, “Novel pixel recovery method based on motion vector disparity and compensation difference,” IEEE Access, vol. 6, pp. 44362-44375, 2018.
[38] U. Sara, M. Akter, and M. S. Uddin, “Image Quality Assessment through FSIM, SSIM, MSE and PSNR—A Comparative Study,” Journal of Computer and Communications, vol. 07, no. 03, pp. 8–18, 2019.
[39] F. Tommasi, V. de Luca, and C. Melle, “Packet losses and objective video quality metrics in H.264 video streaming,” J Vis Commun Image Represent, vol. 27, pp. 7–27, 2015.
 
TABRIZ JOURNAL OF ELECTRICAL ENGINEERING
Volume 53, Issue 4 - Serial Number 106
January 2024
Pages 299-308

Files

Share

How to cite

Statistics