Applying Regression Models on Subsets with High Correlations for a Better Numeric Missing Values Imputation

Authors

Faculty of Computer Engineering, Shahid Rajaee Teacher Training University, Tehran, Iran

Abstract

The presence of missing values in the real world data is a very prevalent and inevitable problem. So, it’s necessary to fill up these missing values accurately, before they are used for knowledge discovery process. This paper proposes three novel methods to fill numeric missing values. All of the proposed methods apply regression models on subsets of data which there are strong correlations among them. These subsets are selected using forward selection based approaches. In the selection of the desired subsets, it is tried to maximize the correlation between missing attribute and other attributes. The correlation coefficient is used to measure the relationships between attributes. The priority of each missing attribute for imputation purpose is also considered in the proposed methods. The performance of proposed methods is evaluated on five real world datasets with different missing ratios. The efficiency of the proposed methods is compared with five different estimation methods, namely, the mean imputation, the k nearest neighbours imputation, a fuzzy c-means based imputation, a decision tree based imputation, and a regression based imputation algorithm, called “Incremental Attribute Regression Imputation” (IARI) method. Two well-known evaluation criteria, namely, Root Mean Squared Error (RMSE) and Coefficient of Determination (CoD) are used to compare the performance of proposed methods with other imputation methods. Experimental results show that the proposed methods perform better than other compared methods, even when the missing ratio is high.

Keywords

References

[1] Y. Qin, S. Zhang, X. Zhu, J. Zhang, and C. Zhang, “Pop algorithm: Kernel-based imputation to treat missing values in knowledge discovery from databases,” Expert Systems with Applications, vol. 36, no. 2, pp. 2794–2804, 2009.
[2] مرتضی خرّم کشکولی و مریم دهقانی، «تشخیص، شناسایی و جداسازی عیب توربین گاز پالایشگاه دوم پارس جنوبی با استفاده از روش‌های ترکیبی داده‌کاوی، k-means، تحلیل مؤلفه‌های اصلی (PCA) و ماشین بردار پشتیبان (SVM)»، مجله علمی پژوهشی مهندسی برق دانشگاه تبریز، جلد 47، شماره 2، صفحات 501-515، 1396.
[3] علیرضا سردار و رمضان هاونگی، «بهبود عملکرد الگوریتم خوشه‌یابی خودکار تصاویر رنگی به کمک پیش‌پردازش با شبکه عصبی خودسامانده (SOM)»، مجله علمی پژوهشی مهندسی برق دانشگاه تبریز، جلد 47، شماره 3، صفحات 1082-1073، 1396.
[4] G. Sun, J. Shao, H. Han, and X. Ding, “Missing value imputation for wireless sensory soil data: A comparative study,” in 2nd International Conference on Big Data Computing and Communications, pp. 172–184, Springer, Shenyang, China, 2016.
[5] M. Lichman, UCI Machine Learning Repository, Available online at: http://archive.ics.uci.edu/ml, Accessed June 2017.
[6] P. J. Garcia-Laencina, J. L. Sancho-Gomez, and AR. Figueiras-Vidal, “Pattern classification with missing data: a review,” Neural Computing and Applications, vol. 19, no. 2, pp. 263–282, 2010.
[7] E. L. Silva-Ramrez, R. Pino-Mejas, and M. Lpez-Coello, “Single imputation with multilayer perceptron and multiple imputation combining multi-layer perceptron and k-nearest neighbours for monotone patterns,” Applied Soft Computing, vol. 29, no. 1, pp. 65 – 74, 2015.
[8] M. G. Rahman and M. Z. Islam, “Missing value imputation using a fuzzy clustering-based EM approach,” Knowledge and Information Systems, vol. 46, no. 2, pp. 389–422, 2016.
[9] M. Amiri and R. Jensen, “Missing data imputation using fuzzy-rough methods,” Neurocomputing, vol. 205, no. 1, pp. 152–164, 2016.
[10] H. Wang and S. Wang, “Mining incomplete survey data through classification,” Knowledge and information systems, vol. 24, no. 2, pp. 221–233, 2010.
[11] C.F. Tsai and F.Y. Chang, “Combining instance selection for better missing value imputation,” Journal of Systems and Software, vol. 122, no. 1, pp. 63– 71, 2016.
[12] C. T. Tran, M. Zhang, P. Andreae, and B. Xue, “Improving performance for classification with incomplete data using wrapper-based feature selection,” Evolutionary Intelligence, vol. 9, no. 3, pp. 81–94, 2016.
[13] M. G. Rahman and M. Z. Islam, “Data quality improvement by imputation of missing values,” in 5th International Conference on Computer Science and Information Technology (CSIT-2013), pp. 82– 88, Yogyakarta, Indonesia, 2013.
[14] B. van Stein and W. Kowalczyk, “An incremental slgorithm for repairing training sets with missing values,” in International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, vol. 611, no. 1, pp. 175–186. Springer International Publishing, Eindhoven, Netherlands, 2016.
[15] G. Rahman and Z. Islam, “A decision tree-based missing value imputation technique for data pre-processing,” in Proceedings of the Ninth Australasian Data Mining Conference, vol. 121, no. 1, pp. 41-50. Australian Computer Society, Inc., 2011.
[16] L. Breiman, “Random Forests,” Machine learning, vol. 45, no. 1, pp. 5–32, 2001.
[17] C.H. Wu, C.H. Wun, and H.J. Chou, “Using association rules for completing missing data,” in 4th International Conference on Hybrid Intelligent Systems, pp. 236–241, Kitakyushu, Japan, IEEE, 2004.
[18] N. Singh, A. Javeed, S. Chhabra, and P. Kumar, “Missing value imputation with unsupervised kohonen self organizing map,” Emerging Research in Computing, Information, Communication and Applications, vol. 1, no. 1, pp. 61–76. Springer, New Delhi, India, 2015.
[19] J. Han, M. Kamber, and J. Pei, Data Mining: Concepts and Techniques. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 3rd Edition, 2011.
[20] C. Jiang and Z. Yang, “CKNNI: an improved knn-based missing value handling technique,” in 11th International Conference on Intelligent Computing, pp. 441–452, Springer, Fuzhou, China, 2015.
[21] R. Krishnamoorthy, S. S. Kumar, and B. Neelagund, “A new approach for data cleaning process,” in Recent Advances and Innovations in Engineering (ICRAIE), pp. 1-5, IEEE, Jaipur, India, 2014.
[22] C. Zhang, X. Zhu, J. Zhang, Y. Qin, and S. Zhang, “GBKII: An imputation method for missing values,” in Advances in Knowledge Discovery and Data Mining: 11th Pacific-Asia Conference, pp. 1080–1087, Springer, Nanjing, China, 2007.
[23] B. M. Patil, R. C. Joshi, and D. Toshniwal, “Missing value imputation based on k-mean clustering with weighted distance,” in 3rd International Conference on Contemporary Computing, pp. 600–609, Springer, Noida, India, 2010.
[24] V. Ayuyev, J. Jupin, P. Harris, and Z. Obradovic, “Dynamic clustering-based estimation of missing values in mixed type data,” in 11th International Conference on Data Warehousing and Knowledge Discovery, pp. 366–37, Springer, Linz, Austria, 2009.
[25] D. Li, J. Deogun, W. Spaulding, and B. Shuart, “Towards missing data imputation: A study of fuzzy k-means clustering Method,” Rough Sets and Current Trends in Computing, vol. 3066, no. 1, pp. 573–579, Springer, 2004.
[26] P. Raja and K. Thangavel, “Soft clustering based missing value imputation,” in Digital Connectivity-Social Impact: 51st Annual Convention of the Computer Society of India, pp. 119–133, Springer, Coimbatore, India, 2016.
[27] N. Ankaiah and V. Ravi, “A novel soft computing hybrid for data imputation,” in 7th International Conference on Data Mining (DMIN), Las Vegas, USA, 2011.
[28] S. Azim, S. Aggarwal, “Hybrid model for data imputation: using fuzzy c-means and multi-layer perceptron,” in Advance Computing Conference (IACC), 2014 IEEE International, pp. 1281–1285, Gurgaon, India, 2014.
[29] S. Bashir, S. Razzaq, U. Maqbool, S. Tahir, and A. R. Baig, “Using association rules for better treatment of missing values,” in 10th WSEAS International Conference on Computers, Wisconsin, USA, pp. 1133-1138, 2009.
[30] D. R. Wilson and T. R. Martinez, “Reduction techniques for instance-based learning algorithms,” Machine learning, vol. 38, no. 3, pp. 257–286, 2000.
[31] Batista, G. E., & Monard, M. C. (2002). A study of k-nearest neighbour as an imputation method. Hybrid Intell Syst (HIS), vol. 87, no. 1, pp. 251-260, 2002.
[32] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, no. 1, pp. 2825–2830, 2011.
TABRIZ JOURNAL OF ELECTRICAL ENGINEERING
Volume 48, Issue 3 - Serial Number 85
November 2018
Pages 1187-1200

Files

Share

How to cite

Statistics