A Multi-class Magnitude Classifying Sparse Source Model for Compressible Sources

Document Type : Original Article

Authors

1 Electrical Engineering Department, Engineering Faculty, Shahid Chamran Universiy of Ahvaz, Ahvaz, Iran

2 Electrical and Computer Engineering Department, Engineering Faculty, University of Mohaghegh Ardabili, Ardabil, Iran

Abstract

Source modelling is a gateway to the fascinating world of source coding. Many real-world sources are sparse or have a sparse representation. According to this fact, this work has focused on providing a new model to represent real-world non-strictly sparse (compressible) sources. To this aim, a novel model has been evolved from a simple sparse binary source to reflect the characteristics of compressible sources. The model is capable to represents real-world compressible sources by classifying samples into different classes based on their magnitudes. The model parameters are estimated using an innovative approach, a combination of a clustering technique and the binary genetic algorithm. The ability of the new approach has been assessed in modeling DCT coefficients of still images and video sequences. The proposed model also inspires an efficient coding approach to compress a wide range of sources including compressible sources. Comparison with classical well-known distributions including Laplace, Cauchy, and generalized Gaussian distribution and also with the most recent Noisy BG model reveals the capabilities of the proposed model in describing the characteristics of sparse sources. The numerical results based on the “chi-square goodness of fit” show that the proposed model provides a better fit to reflect the statistical characteristics of compressible sources.

Keywords

References

[1]         H. M. Al-Kadhim and H. S. Al-Raweshidy, “Energy Efficient Data Compression in Cloud Based IoT,” IEEE Sens. J., vol. 21, no. 10, pp. 12212–12219, 2021.
[2]         A. Biason, C. Pielli, A. Zanella, and M. Zorzi, “Access control for IoT nodes with energy and fidelity constraints,” IEEE Trans. Wirel. Commun., vol. 17, no. 5, pp. 3242–3257, 2018.
[3]         L.-M. Ang, K. P. Seng, A. M. Zungeru, and G. K. Ijemaru, “Big sensor data systems for smart cities,” IEEE Internet Things J., vol. 4, no. 5, pp. 1259–1271, 2017.
[4]          ث. عمویی و ک. میرزایی, “«فشرده سازی تصویر توسط چندی سازی برداری مبتنی بر الگوریتم کرم شب تاب بهبودیافته»,” مجله مهندسی برق دانشگاه تبریز, vol. 42, no. 2, pp. 693–707, 1398.
[5]          م. مگری and ه. گرایلو, “فشرده سازی سیگنالهای الکترومایوگرام مبتنی بر هموارسازی به کمک تکنیک پیشتاکید-واتاکید,” مجله مهندسی برق دانشگاه تبریز, vol. 42, no. 4, pp. 1837–1848, 1398.
[6]          ط. محمود and ج. سپیده, “فشرده سازی سیگنالهای ژنوم با کمک حسگری فشرده و کاربرد آن در مقایسه دنباله های ژنی,” مجله مهندسی برق دانشگاه تبریز, vol. 42, no. 1, pp. 307–316, 1398.
[7]         T. M. Cover and J. A. Thomas, “Elements of information theory.” Hoboken, NJ: Wiley-Interscience, pp. 301–346, 2006.
[8]         D. L. Donoho, M. Vetterli, R. A. DeVore, and I. Daubechies, “Data compression and harmonic analysis,” IEEE Trans. Inf. Theory, vol. 44, no. 6, pp. 2435–2476, 1998.
[9]         E. J. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. theory, vol. 52, no. 2, pp. 489–509, 2006.
[10]       D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. theory, vol. 52, no. 4, pp. 1289–1306, 2006.
[11]       E. J. Candes and T. Tao, “Near-optimal signal recovery from random projections: Universal encoding strategies?,” IEEE Trans. Inf. theory, vol. 52, no. 12, pp. 5406–5425, 2006.
[12]       C. Weidmann and M. Vetterli, “Rate distortion behavior of sparse sources,” IEEE Trans. Inf. theory, vol. 58, no. 8, pp. 4969–4992, 2012.
[13]       F. Abramovich, T. Sapatinas, and B. W. Silverman, “Wavelet thresholding via a Bayesian approach,” J. R. Stat. Soc. Ser. B (Statistical Methodol., vol. 60, no. 4, pp. 725–749, 1998.
[14]       H. Rosenthal and J. Binia, “On the epsilon entropy of mixed random variables,” IEEE Trans. Inf. Theory, vol. 34, no. 5, pp. 1110–1114, 1988.
[15]       A. Elzanaty, A. Giorgetti, and M. Chiani, “Lossy compression of noisy sparse sources based on syndrome encoding,” IEEE Trans. Commun., vol. 67, no. 10, pp. 7073–7087, 2019.
[16]       A. Fraysse, B. Pesquet-Popescu, and J.-C. Pesquet, “On the uniform quantization of a class of sparse sources,” IEEE Trans. Inf. Theory, vol. 55, no. 7, pp. 3243–3263, 2009.
[17]       W.-H. Chen and C. Smith, “Adaptive coding of monochrome and color images,” IEEE Trans. Commun., vol. 25, no. 11, pp. 1285–1292, 1977.
[18]       E. Y. Lam and J. W. Goodman, “A mathematical analysis of the DCT coefficient distributions for images,” IEEE Trans. image Process., vol. 9, no. 10, pp. 1661–1666, 2000.
[19]       N. Kamaci, Y. Altunbasak, and R. M. Mersereau, “Frame bit allocation for the H. 264/AVC video coder via Cauchy-density-based rate and distortion models,” IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 8, pp. 994–1006, 2005.
[20]       S. G. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 11, no. 7, pp. 674–693, 1989.
[21]     P. Moulin and J. Liu, “Analysis of multiresolution image denoising schemes using generalized Gaussian and complexity priors,” IEEE Trans. Inf. Theory, vol. 45, no. 3, pp. 909–919, 1999.
[22]       A. Antoniadis, D. Leporini, and J. Pesquet, “Wavelet thresholding for some classes of non–Gaussian noise,” Stat. Neerl., vol. 56, no. 4, pp. 434–453, 2002.
[23]       M. Aramideh, E. Namjoo, and M. Nooshyar, “Non-strictly Sparse Source Modelling Using Heavy-tailed Distributions for DCT Coefficients,” in 2019 27th Iranian Conference on Electrical Engineering (ICEE), 2019, pp. 1570–1575.
[24]       L. Palzer and R. Timo, “Fixed-length compression for letter-based fidelity measures in the finite blocklength regime,” in Information Theory (ISIT), 2016 IEEE International Symposium on, 2016, pp. 2424–2428.
[25]       L. Palzer and R. Timo, “A lower bound for the rate-distortion function of spike sources that is asymptotically tight,” in Information Theory Workshop (ITW), 2016 IEEE, 2016, pp. 101–105.
[26]       M. Leinonen, M. Codreanu, M. Juntti, and G. Kramer, “Rate-distortion performance of lossy compressed sensing of sparse sources,” IEEE Trans. Commun., vol. 66, no. 10, pp. 4498–4512, 2018.
[27]       A. Kipnis, G. Reeves, Y. C. Eldar, and A. J. Goldsmith, “Compressed sensing under optimal quantization,” in 2017 IEEE international symposium on information theory (ISIT), 2017, pp. 2148–2152.
[28]       M. Kaaniche, A. Fraysse, B. Pesquet-Popescu, and J.-C. Pesquet, “Accurate rate-distortion approximation for sparse Bernoulli-Generalized Gaussian models,” in 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014, pp. 2020–2024.
[29]       A. Cohen, N. Shlezinger, S. Salamatian, Y. C. Eldar, and M. Médard, “Serial quantization for sparse time sequences,” IEEE Trans. Signal Process., 2021.
[30]       M. Kaaniche, A. Fraysse, B. Pesquet-Popescu, and J.-C. Pesquet, “A bit allocation method for sparse source coding,” IEEE Trans. Image Process., vol. 23, no. 1, pp. 137–152, 2013.
[31]       S. B. Korada and R. L. Urbanke, “Polar codes are optimal for lossy source coding,” IEEE Trans. Inf. Theory, vol. 56, no. 4, pp. 1751–1768, 2010.
[32]       M. J. Wainwright, E. Maneva, and E. Martinian, “Lossy source compression using low-density generator matrix codes: Analysis and algorithms,” IEEE Trans. Inf. theory, vol. 56, no. 3, pp. 1351–1368, 2010.
[33]       A. Golmohammadi, D. G. M. Mitchell, J. Kliewer, and D. J. Costello, “Encoding of Spatially Coupled LDGM Codes for Lossy Source Compression,” IEEE Trans. Commun., vol. 66, no. 11, pp. 5691–5703, 2018.
[34]       C. Chen, L. Wang, and S. Liu, “The design of protograph LDPC codes as source codes in a JSCC system,” IEEE Commun. Lett., vol. 22, no. 4, pp. 672–675, 2018.
[35]       A. No and T. Weissman, “Rateless lossy compression via the extremes,” IEEE Trans. Inf. theory, vol. 62, no. 10, pp. 5484–5495, 2016.
[36]       S. Eghbalian-Arani and H. Behroozi, “On the performance of polar codes for lossy compression of Gaussian sources,” in 2013 Iran Workshop on Communication and Information Theory, 2013, pp. 1–5.
[37]       F. Yang, K. Niu, K. Chen, Z. He, and B. Tian, “Design and analysis of lossy source coding of Gaussian sources with finite-length polar codes,” in 2015 IEEE Wireless Communications and Networking Conference (WCNC), 2015, pp. 201–206.
[38]       S. G. Mallat, “A theory for multiresolution signal decomposition: the wavelet representation,” in Fundamental Papers in Wavelet Theory, Princeton University Press, 2009, pp. 494–513.
[39]       R. L. Haupt and S. Ellen Haupt, “Practical genetic algorithms,” 2004.
[40]       F. Müller, “Distribution shape of two-dimensional DCT coefficients of natural images,” Electron. Lett., vol. 29, no. 22, pp. 1935–1936, 1993.
[41]       “BIOID-FACEDATABASE.” .
[42]       “Xiph.org Video Test Media.” .
[43]       K. Krishnamoorthy, “Handbook of Statistical Distributions with Applications,” 2006.

Files

Share

How to cite

Statistics