لوکوویت: یک مدل کارآمد مبتنی بر ترانسفورمر بینایی برای طبقه‌بندی خودکار لکوسیت‌ها

نوع مقاله : علمی-پژوهشی

نویسندگان

گروه بیوالکتریک، دانشکده مهندسی پزشکی، دانشگاه صنعتی سهند، تبریز، ایران

چکیده

شناسایی و ارزیابی لکوسیت‌ها برای ارزیابی کیفیت سیستم ایمنی انسان مهم است. با این حال، تجزیه و تحلیل اسمیر خون به تخصص پاتولوژیست بستگی دارد. روش دستی برای تجزیه و تحلیل و طبقه بندی گلوبولهای سفید ها پرهزینه و زمان‌بر است و می تواند منجر به خطا در تشخیص شود. اکثر روش‌های یادگیری عمیق از مدل های مبتنی بر CNN برای طبقه بندی گلبول‌های سفید استفاده می‌کنند. این مقاله استفاده از یک شبکه مبتنی بر ViT را برای طبقه‌بندی لکوسیت‌ها در نمونه خون مورد بحث قرار می‌دهد. مجموعه داده مورد استفاده در این مقاله شامل 352 تصویر با اندازه 320 در 240 است که از طریق روش‌هایی برای ایجاد یک مجموعه داده متعادل از 12444 تصویر داده‌افزایی شده است. سپس داده‌های افزایش‌یافته برای آموزش معماری مبتنی بر ViT برای طبقه‌بندی انواع مختلف گلبول‌های سفید مورد استفاده قرار گرفته است. دراولین مرحله‌از روش پیشنهادی، یک توکنایزر کانولوشن برای استخراج پچ تصاویر اعمال شده است. این پچ‌ها فلت شده و به عنوان ورودی برای ساختار مبتنی بر ViT برای شناسایی زیر کلاس‌ها در مرحله دوم استفاده شده‌اند. نتایج به‌دست‌آمده با استفاده از لوکوویت نشان می‌دهد صحت شبکه پیشنهادی 99.04 درصد است که نسبت به شبکه‌های پیشرفته برتری دارد.

کلیدواژه‌ها

موضوعات

عنوان مقاله [English]

Leukovit: An efficient vision transformer-based model for automatic classification of leukocytes

نویسندگان [English]

  • Z. Asgharzadeh bonab
  • S. Shamekhi
Faculty of Biomedical Engineering, Sahand University of Technology, Tabriz, Iran.
چکیده [English]

The identification and evaluation of leukocytes are important to assess the quality of the human immune system; however, the analysis of blood smears depends on the pathologist’s expertise. The manual method for analyzing and classifying WBCs is costly and time-consuming and can result in errors in detection. Most deep learning methods use CNN-based models for white blood cell classification. This paper discusses the use of a ViT-based network, for the classification of leukocytes (WBCs) in a blood sample. The Dataset used in this paper consists of 352 images with a size of 320x240, which was augmented through techniques to create a balanced dataset of 12444 images. The augmented data was then used to train a ViT-based architecture to classify the different types of WBCs. As the first step of the proposed algorithm, a convolutional tokenizer has been applied for patch extraction of images. These patches have been flattened and have been used as input for a ViT-based structure to recognize the subclasses in the second step. The results obtained using Leukovit show that the accuracy of the proposed network is 99.04% which is outperforming the state-of-the-art networks.

کلیدواژه‌ها [English]

  • White blood cells
  • Image classification
  • Deep learning
  • Convolutional neural network
  • Vision Transformer

مراجع

[1] W. King, K. Toler, J. Woodell-May, Role of white blood cells in blood-and bone marrow-based autologous therapies, BioMed research international, 2018, 2018.
[2] K. Almezhghwi, S. Serte, Improved classification of white blood cells with the generative adversarial network and deep convolutional neural network, Computational Intelligence and Neuroscience, 2020, 2020.
[3] S. Shafique, S. Tehsin, Computer-aided diagnosis of acute lymphoblastic leukaemia, Computational and mathematical methods in medicine, 2018, 2018.
[4] V.N. Murthy, S. Maji, R. Manmatha, Automatic image annotation using deep learning representations,  Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pp. 603-606, 2015.
[5] K.A.A. Daqqa, A.Y. Maghari, W.F. Al Sarraj, Prediction and diagnosis of leukemia using classification algorithms,  2017 8th international conference on information technology (ICIT), IEEE, pp. 638-643, 2017.
[6] D.M. Ushizima, A.C. Lorena, A. De Carvalho, Support vector machines applied to white blood cell recognition,  Fifth International Conference on Hybrid Intelligent Systems (HIS'05), IEEE, pp. 6 pp., 2005.
[7] X. Zheng, Y. Wang, G. Wang, J. Liu, Fast and robust segmentation of white blood cell images by self-supervised learning, Micron, 107  55-71, 2018.
[8] سزاوار, فرسی, حسن, محمدزاده, بازیابی تصویر مبتنی بر محتوا با استفاده از شبکه‌های عصبی کانولوشن عمیق, مجله مهندسی برق دانشگاه تبریز, 48  1595-1603, 2019.
[9] P.P. Banik, R. Saha, K.-D. Kim, An automatic nucleus segmentation and CNN model based classification method of white blood cell, Expert Systems with Applications, 149  113211, 2020.
[10] A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks, Advances in neural information processing systems, 25, 2012.
[11] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556, 2014.
[12] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions,  Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-9, 2015.
[13] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition,  Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
[14] M.Z. Alom, M. Hasan, C. Yakopcic, T.M. Taha, V.K. Asari, Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation, arXiv preprint arXiv:1802.06955, 2018.
[15] G. Liang, H. Hong, W. Xie, L. Zheng, Combining convolutional neural network with recursive neural network for blood cell image classification, IEEE access, 6  36188-36197, 2018.
[16] P. Tang, H. Wang, S. Kwong, G-MS2F: GoogLeNet based multi-stage feature fusion of deep CNN for scene recognition, Neurocomputing, 225  188-197, 2017.
[17] A. Khan, A. Eker, A. Chefranov, H. Demirel, White blood cell type identification using multi-layer convolutional features with an extreme-learning machine, Biomedical Signal Processing and Control, 69  102932, 2021.
[18] A. Darvish, S. Shamekhi, A hybrid multi-scale CNN-LSTM deep learning model for the identification of protein-coding regions in DNA sequences, TABRIZ JOURNAL OF ELECTRICAL ENGINEERING, 52  137-146, 2022.
[19] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, An image is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929, 2020.
[20] Y. Dong, Z. Jiang, H. Shen, W.D. Pan, L.A. Williams, V.V. Reddy, W.H. Benjamin, A.W. Bryan, Evaluations of deep convolutional neural networks for automatic identification of malaria infected cells,  2017 IEEE EMBS international conference on biomedical & health informatics (BHI), IEEE, pp. 101-104, 2017.
[21] M. Imran Razzak, S. Naz, Microscopic blood smear segmentation and classification using deep contour aware CNN and extreme machine learning,  Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 49-55, 2017.
[22] C.L. Chen, A. Mahjoubfar, L.-C. Tai, I.K. Blaby, A. Huang, K.R. Niazi, B. Jalali, Deep learning in label-free cell classification, Scientific reports, 6  1-16, 2016.
[23] X. Wang, R. Girshick, A. Gupta, K. He, Non-local neural networks,  Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7794-7803, 2018.
[24] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, S. Zagoruyko, End-to-end object detection with transformers,  European conference on computer vision, Springer, pp. 213-229, 2020.
[25] P. Ramachandran, N. Parmar, A. Vaswani, I. Bello, A. Levskaya, J. Shlens, Stand-alone self-attention in vision models, Advances in Neural Information Processing Systems, 32, 2019.
[26] H. Wang, Y. Zhu, B. Green, H. Adam, A. Yuille, L.-C. Chen, Axial-deeplab: Stand-alone axial-attention for panoptic segmentation,  European Conference on Computer Vision, Springer, pp. 108-126, 2020.
[27] I. Bello, B. Zoph, A. Vaswani, J. Shlens, Q.V. Le, Attention augmented convolutional networks,  Proceedings of the IEEE/CVF international conference on computer vision, pp. 3286-3295, 2019.
[28] H. Hu, J. Gu, Z. Zhang, J. Dai, Y. Wei, Relation networks for object detection,  Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3588-3597, 2018.
[29] B. Wu, C. Xu, X. Dai, A. Wan, P. Zhang, Z. Yan, M. Tomizuka, J. Gonzalez, K. Keutzer, P. Vajda, Visual transformers: Token-based image representation and processing for computer vision, arXiv preprint arXiv:2006.03677, 2020.
[30] J. Chen, Y. Lu, Q. Yu, X. Luo, E. Adeli, Y. Wang, L. Lu, A.L. Yuille, Y. Zhou, Transunet: Transformers make strong encoders for medical image segmentation, arXiv preprint arXiv:2102.04306, 2021.
[31] F. Mehboob, A. Rauf, R. Jiang, A.K.J. Saudagar, K.M. Malik, M.B. Khan, M.H.A. Hasnat, A. AlTameem, M. AlKhathami, Towards robust diagnosis of COVID-19 using vision self-attention transformer, Scientific Reports, 12  1-12, 2022.
[32] X. Qu, H. Lu, W. Tang, S. Wang, D. Zheng, Y. Hou, J. Jiang, A VGG attention vision transformer network for benign and malignant classification of breast ultrasound images, Medical Physics, 49  5787-5798, 2022.
[33] Y. Dai, Y. Gao, F. Liu, Transmed: Transformers advance multi-modal medical image classification, Diagnostics, 11  1384, 2021.
[34] T. Wang, J. Lan, Z. Han, Z. Hu, Y. Huang, Y. Deng, H. Zhang, J. Wang, M. Chen, H. Jiang, O-Net: a novel framework with deep fusion of CNN and transformer for simultaneous segmentation and classification, Frontiers in Neuroscience, 16, 2022.
[35] P. Cho, S. Dash, A. Tsaris, H.-J. Yoon, Image transformers for classifying acute lymphoblastic leukemia,  Medical Imaging 2022: Computer-Aided Diagnosis, SPIE, pp. 633-639, 2022.
[36] S. Tripathi, A.I. Augustin, R. Sukumaran, S. Dheer, E. Kim, HematoNet: Expert level classification of bone marrow cytology morphology in hematological malignancy with deep learning, Artificial Intelligence in the Life Sciences, 2, 2022.
[37] Z. Dai, H. Liu, Q.V. Le, M. Tan, Coatnet: Marrying convolution and attention for all data sizes, Advances in Neural Information Processing Systems, 34  3965-3977, 2021.
[38] P. Huang, J. Wang, J. Zhang, Y. Shen, C. Liu, W. Song, S. Wu, Y. Zuo, Z. Lu, D. Li, Attention-aware residual network based manifold learning for white blood cells classification, IEEE Journal of Biomedical and Health Informatics, 25  1206-1214, 2020.
[39] Z. Wang, J. Xiao, J. Li, H. Li, L. Wang, WBC-AMNet: Automatic classification of WBC images using deep feature fusion network based on focalized attention mechanism, Plos one, 17  e0261848, 2022.
[40] O. Katar, O. Yildirim, An Explainable Vision Transformer Model Based White Blood Cells Classification and Localization, 2023.
[41] S. Chen, S. Lu, S. Wang, Y. Ni, Y. Zhang, Shifted Window Vision Transformer for Blood Cell Classification, Electronics, 12  2442, 2023.
[42] S.M. Dipto, M.T. Reza, M.N.J. Rahman, M.Z. Parvez, P.D. Barua, S. Chakraborty, An XAI Integrated Identification System of White Blood Cell Type Using Variants of Vision Transformer,  International Conference on Interactive Collaborative Robotics, Springer, pp. 303-315, 2023.
[43] D. Parthasarathy, WBC-classification, Google License, Online; accessed 2022, 2017.
[44] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, Advances in neural information processing systems, 30, 2017.
[45] S. d’Ascoli, H. Touvron, M.L. Leavitt, A.S. Morcos, G. Biroli, L. Sagun, Convit: Improving vision transformers with soft convolutional inductive biases,  International Conference on Machine Learning, PMLR, pp. 2286-2296, 2021.
[46] I.O. Tolstikhin, N. Houlsby, A. Kolesnikov, L. Beyer, X. Zhai, T. Unterthiner, J. Yung, A. Steiner, D. Keysers, J. Uszkoreit, Mlp-mixer: An all-mlp architecture for vision, Advances in Neural Information Processing Systems, 34  24261-24272, 2021.
[47] M. VASOUJOUYBARI, E. Ataie, M. Bastam, An MLP-based Deep Learning Approach for Detecting DDoS Attacks, TABRIZ JOURNAL OF ELECTRICAL ENGINEERING, 52  195-204, 2022.
[48] G. Huang, Y. Sun, Z. Liu, D. Sedra, K.Q. Weinberger, Deep networks with stochastic depth,  European conference on computer vision, Springer, pp. 646-661, 2016.
[49] A. Hassani, S. Walton, N. Shah, A. Abuduweili, J. Li, H. Shi, Escaping the big data paradigm with compact transformers, arXiv preprint arXiv:2104.05704, 2021.
[50] T. Dozat, Incorporating nesterov momentum into adam, 2016.
[51] C.-B. Zhang, P.-T. Jiang, Q. Hou, Y. Wei, Q. Han, Z. Li, M.-M. Cheng, Delving deep into label smoothing, IEEE Transactions on Image Processing, 30  5984-5996, 2021.
[52] R.R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra, Grad-cam: Visual explanations from deep networks via gradient-based localization,  Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017.
[53] F. Wu, A. Fan, A. Baevski, Y.N. Dauphin, M. Auli, Pay less attention with lightweight and dynamic convolutions, arXiv preprint arXiv:1901.10430, 2019.
[54] M. Sharma, A. Bhave, R.R. Janghel, White blood cell classification using convolutional neural network,  Soft Computing and Signal Processing, Springer, pp. 135-143, 2019.
[55] O. Dekhil, Computational techniques in medical image analysis application for white blood cells classification, 2020.
[56] V. Ranga, S. Gupta, P. Agrawal, J. Meena, Pathological analysis of blood cells using deep learning techniques, Recent Advances in Computer Science and Communications (Formerly: Recent Patents on Computer Science), 15  397-403, 2022.
[57] A. Patil, M. Patil, G. Birajdar, White blood cells image classification using deep learning with canonical correlation analysis, Irbm, 42  378-389, 2021.
[58] M.B. Khan, T. Islam, M. Ahmad, R. Shahrior, Z.N. Riya, A CNN Based Deep Learning Approach for Leukocytes Classification in Peripheral Blood from Microscopic Smear Blood Images,  Proceedings of International Joint Conference on Advances in Computational Intelligence, Springer, pp. 67-76, 2021.
[59] F. BOZKURT, Classification of Blood Cells from Blood Cell Images Using Dense Convolutional Network, Journal of Science, Technology and Engineering Research, 2  81-88, 2021.
[60] S. Parayil, J. Aravinth, Transfer Learning-based Feature Fusion of White Blood Cell Image Classification,  2022 7th International Conference on Communication and Electronics Systems (ICCES), IEEE, pp. 1468-1474, 2022.
[61] E. Başaran, Classification of white blood cells with SVM by selecting SqueezeNet and LIME properties by mRMR method, Signal, Image and Video Processing,  1-9, 2022.
[62] C. Cheuque, M. Querales, R. León, R. Salas, R. Torres, An Efficient Multi-Level Convolutional Neural Network Approach for White Blood Cells Classification, Diagnostics, 12  248, 2022.
[63] S. Standring, Gray's anatomy e-book: the anatomical basis of clinical practice, Elsevier Health Sciences2021.
 
 

فایل ها

هم رسانی

ارجاع به این مقاله

آمار