International Journal of

ADVANCED AND APPLIED SCIENCES

EISSN: 2313-3724, Print ISSN: 2313-626X

Frequency: 12
line decor
  
line decor

 Volume 11, Issue 6 (June 2024), Pages: 170-177

----------------------------------------------

 Original Research Paper

Content-based medical image retrieval method using multiple pre-trained convolutional neural networks feature extraction models

 Author(s): 

 Ahmad A. Alzahrani 1, Ali Ahmed 2, *, Alisha Raza 3

 Affiliation(s):

 1Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi Arabia
 2Faculty of Computing and Information Technology, King Abdulaziz University–Rabigh, Rabigh, Saudi Arabia
 3Department of Computer Science, Maulana Azad National Urdu University, Hyderabad, India

 Full text

  Full Text - PDF

 * Corresponding Author. 

  Corresponding author's ORCID profile: https://orcid.org/0000-0002-8944-8922

 Digital Object Identifier (DOI)

 https://doi.org/10.21833/ijaas.2024.06.019

 Abstract

Content-based medical image retrieval (CBMIR), a specialized area within content-based image retrieval (CBIR), involves two main stages: feature extraction and retrieval ranking. The feature extraction stage is particularly crucial for developing an effective retrieval system with high performance. Lately, pre-trained deep convolutional neural networks (CNNs) have become the preferred tools for feature extraction due to their excellent performance and versatility, which includes the ability to be re-trained and adapt through transfer learning. Various pre-trained deep CNN models are employed as feature extraction tools in CBMIR systems. Researchers have effectively used many such models either individually or in combined forms by merging feature vectors from several models. In this study, a method using multiple pre-trained deep CNNs for CBMIR is introduced, utilizing two popular models, ResNet-18 and GoogleNet, for extracting features. This method combines the feature vectors from both models in a way that selects the best model for each image based on the highest classification probability during training. The method's effectiveness is assessed using two well-known medical image datasets, Kvasir and PH2. The evaluation results show that the proposed method achieved average precision scores of 94.13% for Kvasir and 55.67% for PH2 at the top 10 cut-offs, surpassing some leading methods in this research area.

 © 2024 The Authors. Published by IASE.

 This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

 Keywords

 Image retrieval, Content-based medical image retrieval, Feature extraction, Pre-trained deep CNNs

 Article history

 Received 16 January 2024, Received in revised form 2 June 2024, Accepted 8 June 2024

 Acknowledgment 

This research work was funded by the Institutional Fund Project under grant no. (IFPIP:1173-611-1443). The authors gratefully acknowledge the technical and financial support provided by the Ministry of Education and King Abdulaziz University, DSR, Jeddah, Saudi Arabia.

 Compliance with ethical standards

 Conflict of interest: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

 Citation:

 Alzahrani AA, Ahmed A, and Raza A (2024). Content-based medical image retrieval method using multiple pre-trained convolutional neural networks feature extraction models. International Journal of Advanced and Applied Sciences, 11(6): 170-177

 Permanent Link to this page

 Figures

 Fig. 1 Fig. 2 Fig. 3 Fig. 4 Fig. 5 Fig. 6 Fig. 7 Fig. 8 

 Tables

 Table 1 Table 2 Table 3 Table 4 

----------------------------------------------   

 References (30)

  1. Ahmed A (2020). Implementing relevance feedback for content-based medical image retrieval. IEEE Access, 8: 79969-79976. https://doi.org/10.1109/ACCESS.2020.2990557   [Google Scholar]
  2. Ahmed A (2021). Pre-trained CNNs models for content based image retrieval. International Journal of Advanced Computer Science and Applications, 12(7): 200-206. https://doi.org/10.14569/IJACSA.2021.0120723   [Google Scholar]
  3. Ahmed A (2022). Classification of gastrointestinal images based on transfer learning and denoising convolutional neural networks. In the Proceedings of International Conference on Data Science and Applications, Springer Singapore, Kolkata, India, 1: 631-639. https://doi.org/10.1007/978-981-16-5120-5_48   [Google Scholar]
  4. Ahmed A and Malebary SJ (2020). Query expansion based on top-ranked images for content-based medical image retrieval. IEEE Access, 8: 194541-194550. https://doi.org/10.1109/ACCESS.2020.3033504   [Google Scholar]
  5. Ahmed A and Mohamed S (2021). Implementation of early and late fusion methods for content-based image retrieval. International Journal of Advanced and Applied Sciences, 8(7): 97-105. https://doi.org/10.21833/ijaas.2021.07.012   [Google Scholar]
  6. Ahmed A, Almagrabi AO, and Barukab OM (2023). A content-based medical image retrieval method using relative difference-based similarity measure. Intelligent Automation and Soft Computing, 37(2): 2355-2370. https://doi.org/10.32604/iasc.2023.039847   [Google Scholar]
  7. Ahmed A, Almagrabi AO, and Osman AH (2022). Pre-trained convolution neural networks models for content-based medical image retrieval. International Journal of Advanced and Applied Sciences, 9(12): 11-24. https://doi.org/10.21833/ijaas.2022.12.002   [Google Scholar]
  8. Ahmed A, Saeed F, Salim N, and Abdo A (2014). Condorcet and borda count fusion method for ligand-based virtual screening. Journal of Cheminformatics, 6: 19. https://doi.org/10.1186/1758-2946-6-19   [Google Scholar] PMid:24883114 PMCid:PMC4026830
  9. Alappat AL, Nakhate P, Suman S, Chandurkar A, Pimpalkhute V, and Jain T (2021). CBIR using pre-trained neural networks. Arxiv Preprint Arxiv:2110.14455. https://doi.org/10.48550/arXiv.2110.14455   [Google Scholar]
  10. Bharati S, Podder P, and Mondal MRH (2020). Hybrid deep learning for detecting lung diseases from X-ray images. Informatics in Medicine Unlocked, 20: 100391. https://doi.org/10.1016/j.imu.2020.100391   [Google Scholar] PMid:32835077 PMCid:PMC7341954
  11. Dubey SR (2021). A decade survey of content based image retrieval using deep learning. IEEE Transactions on Circuits and Systems for Video Technology, 32(5): 2687-2704. https://doi.org/10.1109/TCSVT.2021.3080920   [Google Scholar]
  12. Fu Y, Lei Y, Wang T, Curran WJ, Liu T, and Yang X (2020). Deep learning in medical image registration: A review. Physics in Medicine and Biology, 65(20): 20TR01. https://doi.org/10.1088/1361-6560/ab843e   [Google Scholar] PMid:32217829 PMCid:PMC7759388
  13. Garg M and Dhiman G (2021). A novel content-based image retrieval approach for classification using GLCM features and texture fused LBP variants. Neural Computing and Applications, 33(4): 1311-1328. https://doi.org/10.1007/s00521-020-05017-z   [Google Scholar]
  14. Hendrycks D and Gimpel K (2016). A baseline for detecting misclassified and out-of-distribution examples in neural networks. Arxiv Preprint Arxiv:1610.02136. https://doi.org/10.48550/arXiv.1610.02136   [Google Scholar]
  15. Hu H, Zheng W, Zhang X, Zhang X, Liu J, Hu W, Duan H, and Si J (2021). Content‐based gastric image retrieval using convolutional neural networks. International Journal of Imaging Systems and Technology, 31(1): 439-449. https://doi.org/10.1002/ima.22470   [Google Scholar]
  16. Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, and Fei-Fei L (2014). Large-scale video classification with convolutional neural networks. In the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Computer Vision Foundation, Columbus, USA: 1725-1732. https://doi.org/10.1109/CVPR.2014.223   [Google Scholar]
  17. Kasban H and Salama DH (2019). A robust medical image retrieval system based on wavelet optimization and adaptive block truncation coding. Multimedia Tools and Applications, 78(24): 35211-35236. https://doi.org/10.1007/s11042-019-08100-3   [Google Scholar]
  18. Ke R, Li W, Cui Z, and Wang Y (2020). Two-stream multi-channel convolutional neural network for multi-lane traffic speed prediction considering traffic volume impact. Transportation Research Record, 2674(4): 459-470. https://doi.org/10.1177/0361198120911052   [Google Scholar]
  19. Latif A, Rasheed A, Sajid U, Ahmed J, Ali N, Ratyal NI, Zafar B, Dar SH, Sajid M, and Khalil T (2019). Content-based image retrieval and feature extraction: a comprehensive review. Mathematical Problems in Engineering, 2019: 9658350. https://doi.org/10.1155/2019/9658350   [Google Scholar]
  20. Mendonça T, Ferreira PM, Marques JS, Marcal AR, and Rozeira J (2013). PH2-A dermoscopic image database for research and benchmarking. In the 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, Osaka, Japan: 5437-5440. https://doi.org/10.1109/EMBC.2013.6610779   [Google Scholar] PMid:24110966
  21. Öztürk Ş, Çelik E, and Çukur T (2023). Content-based medical image retrieval with opponent class adaptive margin loss. Information Sciences, 637: 118938. https://doi.org/10.1016/j.ins.2023.118938   [Google Scholar]
  22. Pogorelov K, Randel KR, Griwodz C, Eskeland SL, de Lange T, Johansen D, Spampinato C, Dang-Nguyen DT, Lux M, Schmidt PT, and Halvorsen P (2017). Kvasir: A multi-class image dataset for computer aided gastrointestinal disease detection. In the Proceedings of the 8th ACM on Multimedia Systems Conference, ACM, Taipei, Taiwan: 164-169. https://doi.org/10.1145/3083187.3083212   [Google Scholar]
  23. Raju GK, Padmanabham P, and Govardhan A (2022). Enhanced content-based image retrieval with trio-deep feature extractors with multi-similarity function. International Journal of Intelligent Engineering and Systems, 15(6): 511-525. https://doi.org/10.22266/ijies2022.1231.46   [Google Scholar]
  24. Satish B and Supreethi KP (2017). Content based medical image retrieval using relevance feedback Bayesian network. In the International Conference on Electrical, Electronics, Communication, Computer, and Optimization Techniques, IEEE, Mysuru, India: 424-430. https://doi.org/10.1109/ICEECCOT.2017.8284542   [Google Scholar]
  25. Sezavar A, Farsi H, and Mohamadzadeh S (2019). Content-based image retrieval by combining convolutional neural networks and sparse representation. Multimedia Tools and Applications, 78: 20895-20912. https://doi.org/10.1007/s11042-019-7321-1   [Google Scholar]
  26. Sidney S (1957). Nonparametric statistics for the behavioral sciences. The Journal of Nervous and Mental Disease, 125(3): 497. https://doi.org/10.1097/00005053-195707000-00032   [Google Scholar]
  27. Sikandar S, Mahum R, and Alsalman A (2023). A novel hybrid approach for a content-based image retrieval using feature fusion. Applied Sciences, 13(7): 4581. https://doi.org/10.3390/app13074581   [Google Scholar]
  28. Spyromitros-Xioufis E, Papadopoulos S, Kompatsiaris IY, Tsoumakas G, and Vlahavas I (2014). A comprehensive study over VLAD and product quantization in large-scale image retrieval. IEEE Transactions on Multimedia, 16(6): 1713-1728. https://doi.org/10.1109/TMM.2014.2329648   [Google Scholar]
  29. Voorhees EM, Soboroff I, and Lin J (2022). Can old TREC collections reliably evaluate modern neural retrieval models? Arxiv Preprint Arxiv:2201.11086. https://doi.org/10.48550/arXiv.2201.11086   [Google Scholar]
  30. Yang S, Zhang Y, Shen J, Dai Y, Ling Y, Lu H, Zhang R, Ding X, Qi H, Shi Y, and Zhang Z (2020). Clinical potential of UTE‐MRI for assessing COVID‐19: Patient‐and lesion‐based comparative analysis. Journal of Magnetic Resonance Imaging, 52(2): 397-406. https://doi.org/10.1002/jmri.27208   [Google Scholar] PMid:32491257 PMCid:PMC7300684