International Journal of

ADVANCED AND APPLIED SCIENCES

EISSN: 2313-3724, Print ISSN: 2313-626X

Frequency: 12
line decor
  
line decor

 Volume 11, Issue 7 (July 2024), Pages: 87-100

----------------------------------------------

 Original Research Paper

Optimizing hybrid neural networks for precise COVID-19 mRNA vaccine degradation prediction

 Author(s): 

 Hwai Ing Soon 1, 2, Azian Azamimi Abdullah 1, 3, *, Hiromitsu Nishizaki 2, Mohd Yusoff Mashor 1, Latifah Munirah Kamarudin 1, 4, Zeti-Azura Mohamed-Hussein 5, 6, Zeehaida Mohamed 7, Wei Chern Ang 8, 9

 Affiliation(s):

 1Faculty of Electronic Engineering and Technology, Universiti Malaysia Perlis (UniMAP), Arau, Perlis, Malaysia
 2Integrated Graduate School of Medicine, Engineering and Agricultural Science, University of Yamanashi, Kofu, Yamanashi, Japan
 3Medical Devices and Life Sciences Cluster, Sport Engineering Research Centre, Centre of Excellence (SERC), UniMAP, Arau, Perlis, Malaysia
 4Advanced Sensor Technology, Centre of Excellence (CEASTech), Universiti Malaysia Perlis (UniMAP), Arau, Perlis, Malaysia
 5Department of Applied Physics, Faculty of Science and Technology, Universiti Kebangsaan Malaysia (UKM), Bangi, Selangor, Malaysia
 6UKM Medical Molecular Biology Institute, Universiti Kebangsaan Malaysia (UKM), Jalan Yaacob Latif, 56000 Cheras, Kuala Lumpur, Malaysia
 7Department of Medical Microbiology Parasitology, School of Medical Sciences, Universiti Sains Malaysia (USM), 16150 Kubang Kerian Kelantan, Malaysia
 8Clinical Research Centre (CRC), Hospital Tuanku Fauziah (HTF), Ministry of Health Malaysia, Kangar, 01000, Perlis, Malaysia
 9Department of Pharmacy, Hospital Tuanku Fauziah (HTF), Ministry of Health Malaysia, Kangar, 01000, Perlis, Malaysia

 Full text

  Full Text - PDF

 * Corresponding Author. 

  Corresponding author's ORCID profile: https://orcid.org/0000-0001-5851-7705

 Digital Object Identifier (DOI)

 https://doi.org/10.21833/ijaas.2024.07.011

 Abstract

Conventional hybrid models often miss an essential factor that can lead to less effective performance: intrinsic sequence dependence when combining various neural network (NN) architectures. This study addresses this issue by highlighting the importance of sequence hybridization in NN architecture integration, aiming to improve model effectiveness. It combines NN layers—dense, long short-term memory (LSTM), and gated recurrent unit (GRU)—using the Keras Sequential API for defining the architecture. To provide better context, bidirectional LSTM (BiLSTM) and bidirectional GRU (BiGRU) replace their unidirectional counterparts, enhancing the models through bidirectional structures. Out of 25 NN models tested, 18 four-layer hybrid NN models consist of one-quarter dense layer and the rest BiLSTM and BiGRU layers. These hybrid NN models undergo supervised learning regression analysis, with mean column-wise root mean square error (MCRMSE) as the performance metric. The results show that each hybrid NN model produces unique outcomes based on its specific hybrid sequence. The Hybrid_LGSS model performs better than existing three-layer BiLSTM networks in predictive accuracy and shows lower overfitting (MCRMSEs of 0.0749 and 0.0767 for training and validation, respectively). This indicates that the optimal hybridization sequence is crucial for achieving a balance between performance and simplicity. In summary, this research could help vaccinologists develop better mRNA vaccines and provide data analysts with new insights for improvement.

 © 2024 The Authors. Published by IASE.

 This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

 Keywords

 Neural network, Hybridization, Hybridizing sequence, mRNA vaccines

 Article history

 Received 21 February 2024, Received in revised form 1 July 2024, Accepted 2 July 2024

 Acknowledgment 

The authors gratefully acknowledge funding from the Fundamental Research Grant Scheme (FRGS) funded by the Ministry of Higher Education (MOHE), Malaysia. FRGS/1/2021/TKO/UNIMAP/02/65.

 Compliance with ethical standards

 Conflict of interest: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

 Citation:

 Soon HI, Abdullah AA, Nishizaki H, Mashor MY, Kamarudin LM, Mohamed-Hussein ZA, Mohamed Z, and Ang WC (2024). Optimizing hybrid neural networks for precise COVID-19 mRNA vaccine degradation prediction. International Journal of Advanced and Applied Sciences, 11(7): 87-100

 Permanent Link to this page

 Figures

 Fig. 1 Fig. 2 Fig. 3

 Tables

 Table 1 Table 2 Table 3 Table 4 Table 5 Table 6 Table 7 Table 8 Table 9

----------------------------------------------   

 References (38)

  1. Banerjee S, Banerjee D, Singh A, Kumar S, Pooja D, Ram V, Kulhari H, and Saharan VA (2023). A clinical insight on new discovered molecules and repurposed drugs for the treatment of COVID-19. Vaccines, 11(2): 332. https://doi.org/10.3390/vaccines11020332   [Google Scholar] PMid:36851211 PMCid:PMC9967525
  2. Batur ŞC and Laith A (2021). A novel deep learning-based feature selection model for improving the static analysis of vulnerability detection. Neural Computing and Applications, 33(20): 14049-14067. https://doi.org/10.1007/s00521-021-06047-x   [Google Scholar]
  3. Buyrukoğlu S and Savaş S (2023). Stacked-based ensemble machine learning model for positioning footballer. Arabian Journal for Science and Engineering, 48(2): 1371-1383. https://doi.org/10.1007/s13369-022-06857-8   [Google Scholar]
  4. Cahuantzi R, Chen X, and Güttel S (2023). A comparison of LSTM and GRU networks for learning symbolic sequences. In the Science and Information Conference, Springer Nature, Cham, Switzerland: 771-785. https://doi.org/10.1007/978-3-031-37963-5_53   [Google Scholar]
  5. Cattaneo A, Vitali A, Mazzoleni M, and Previdi F (2022). An agent-based model to assess large-scale COVID-19 vaccination campaigns for the Italian territory: The case study of Lombardy region. Computer Methods and Programs in Biomedicine, 224: 107029. https://doi.org/10.1016/j.cmpb.2022.107029   [Google Scholar] PMid:35908330 PMCid:PMC9287580
  6. Cines DB and Greinacher A (2023). Vaccine-induced immune thrombotic thrombocytopenia. Blood, 141(14): 1659-1665. https://doi.org/10.1182/blood.2022017696   [Google Scholar] PMid:36669155 PMCid:PMC9870607
  7. Crommelin DJ, Anchordoquy TJ, Volkin DB, Jiskoot W, and Mastrobattista E (2021). Addressing the cold reality of mRNA vaccine stability. Journal of Pharmaceutical Sciences, 110(3): 997-1001. https://doi.org/10.1016/j.xphs.2020.12.006   [Google Scholar] PMid:33321139 PMCid:PMC7834447
  8. Dileep P, Das D, and Bora PK (2020). Dense layer dropout based CNN architecture for automatic modulation classification. In the National Conference on Communications, IEEE, Kharagpur, India: 1-5. https://doi.org/10.1109/NCC48643.2020.9055989   [Google Scholar]
  9. Dumpa N, Goel K, Guo Y, McFall H, Pillai AR, Shukla A, Repka MA, and Murthy SN (2019). Stability of vaccines. AAPS PharmSciTech, 20: 42. https://doi.org/10.1208/s12249-018-1254-2   [Google Scholar] PMid:30610415
  10. Fern SH, Amir A, and Azemi SN (2022). Multi-class imbalanced classification problems in network attack detections. In the Proceedings of the 6th International Conference on Electrical, Control and Computer Engineering, Springer Singapore, Kuantan, Pahang, Malaysia: 1057-1069. https://doi.org/10.1007/978-981-16-8690-0_92   [Google Scholar]
  11. Giełzak J, Dejak B, Sokołowski J, and Bociong K (2023). Changes in strength parameters of composite cements as affected by storage temperature—A review of the literature. Coatings, 13(2): 244. https://doi.org/10.3390/coatings13020244   [Google Scholar]
  12. Hasan MS, Alam R, and Adnan MA (2023). Compressed neural architecture utilizing dimensionality reduction and quantization. Applied Intelligence, 53(2): 1271-1286. https://doi.org/10.1007/s10489-022-03221-z   [Google Scholar]
  13. He K, Zhang X, Ren S, and Sun J (2016). Deep residual learning for image recognition. In the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA: 770-778. https://doi.org/10.1109/CVPR.2016.90   [Google Scholar] PMid:26180094
  14. Huang G, Liu Z, Van Der Maaten L, and Weinberger KQ (2017). Densely connected convolutional networks. In the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Honolulu, USA: 4700-4708. https://doi.org/10.1109/CVPR.2017.243   [Google Scholar] PMCid:PMC5598342
  15. Imran SA, Islam MT, Shahnaz C, Islam MT, Imam OT, and Haque M (2020). COVID-19 mRNA vaccine degradation prediction using regularized LSTM model. In the IEEE International Women in Engineering (WIE) Conference on Electrical and Computer Engineering (WIECON-ECE), IEEE, Bhubaneswar, India: 328-331. https://doi.org/10.1109/WIECON-ECE52138.2020.9398044   [Google Scholar]
  16. Ing SH, Abdullah AA, and Kanaya S (2021a). Development of COVID-19 mRNA vaccine degradation prediction system. In the International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies, IEEE, Zallaq, Bahrain: 449-454. https://doi.org/10.1109/3ICT53449.2021.9582052   [Google Scholar]
  17. Ing SH, Abdullah AA, Harun NH, and Kanaya S (2021b). COVID-19 mRNA vaccine degradation prediction using LR and LGBM algorithms. Journal of Physics: Conference Series, 1997(1): 012005. https://doi.org/10.1088/1742-6596/1997/1/012005   [Google Scholar]
  18. Ing SH, Abdullah AA, Mashor MY, Mohamed-Hussein ZA, Mohamed Z, and Ang WC (2022). Exploration of hybrid deep learning algorithms for COVID-19 mRNA vaccine degradation prediction system. International Journal of Advances in Intelligent Informatics, 8(3): 404-416. https://doi.org/10.26555/ijain.v8i3.950   [Google Scholar]
  19. Jiménez-Ruiz A, Fernández-Escribano G, Cañas-Carretón M, and Sánchez JL (2023). Using graphics processing units for the efficient dynamic simulation of wind farms. Computers and Electrical Engineering, 112: 109018. https://doi.org/10.1016/j.compeleceng.2023.109018   [Google Scholar]
  20. Kauffmann AD, Kennedy SD, Moss WN, Kierzek E, Kierzek R, and Turner DH (2022). Nuclear magnetic resonance reveals a two hairpin equilibrium near the 3′-splice site of influenza A segment 7 mRNA that can be shifted by oligonucleotides. RNA, 28(4): 508-522. https://doi.org/10.1261/rna.078951.121   [Google Scholar] PMid:34983822 PMCid:PMC8925974
  21. Kaur M and Mohta A (2019). A review of deep learning with recurrent neural network. In the International Conference on Smart Systems and Inventive Technology, IEEE, Tirunelveli, India: 460-465. https://doi.org/10.1109/ICSSIT46314.2019.8987837   [Google Scholar] PMid:30649504 PMCid:PMC6432867
  22. Kurz S, De Gersem H, Galetzka A, Klaedtke A, Liebsch M, Loukrezis D, Russenschuck S, and Schmidt M (2022). Hybrid modeling: Towards the next level of scientific computing in engineering. Journal of Mathematics in Industry, 12: 8. https://doi.org/10.1186/s13362-022-00123-0   [Google Scholar]
  23. Mittal S, Chawla T, Jajoo J, Bansal M, Parashar H, and Ruby D (2023). Performance comparison of recurrent neural networks for metamorphic malware. In the International Conference on Data Science and Network Security, IEEE, Tiptur, India: 1-6. https://doi.org/10.1109/ICDSNS58469.2023.10245978   [Google Scholar]
  24. Muneer A, Fati SM, Akbar NA, Agustriawan D, and Wahyudi ST (2022). iVaccine-Deep: Prediction of COVID-19 mRNA vaccine degradation using deep learning. Journal of King Saud University-Computer and Information Sciences, 34(9): 7419-7432. https://doi.org/10.1016/j.jksuci.2021.10.001   [Google Scholar] PMid:38620874 PMCid:PMC8513509
  25. Okamura S and Ebina H (2021). Could live attenuated vaccines better control COVID-19? Vaccine, 39(39): 5719-5726. https://doi.org/10.1016/j.vaccine.2021.08.018   [Google Scholar] PMid:34426024 PMCid:PMC8354792
  26. Olugbenga O, Oladipo EK, Kolawole OM, Oloke JK, Adelusi TI, Irewolede BA, Dairo EO, Ayeni AE, Kolapo KT, Akindiya OE, and Oluwasegun JA (2022). Bioinformatics, computational informatics, and modeling approaches to the design of mRNA COVID-19 vaccine candidates. Computation, 10(7). https://doi.org/10.3390/computation10070117   [Google Scholar]
  27. Padilla-Perez D, Medina-Sanchez I, Hernández J, and Couder-Castañeda C (2022). Accelerating electromagnetic field simulations based on memory-optimized CPML-FDTD with OpenACC. Applied Sciences, 12(22): 11430. https://doi.org/10.3390/app122211430   [Google Scholar]
  28. Qaid TS, Mazaar H, Alqahtani MS, Raweh AA, and Alakwaa W (2021). Deep sequence modelling for predicting COVID-19 mRNA vaccine degradation. PeerJ Computer Science, 7: e597. https://doi.org/10.7717/peerj-cs.597   [Google Scholar] PMid:34239977 PMCid:PMC8237341
  29. Rao AR and Reimherr M (2023). Modern non-linear function-on-function regression. Statistics and Computing, 33(6): 130. https://doi.org/10.1007/s11222-023-10299-z   [Google Scholar]
  30. Schmidt B and Hildebrandt A (2021). Deep learning in next-generation sequencing. Drug Discovery Today, 26(1): 173-180. https://doi.org/10.1016/j.drudis.2020.10.002   [Google Scholar] PMid:33059075 PMCid:PMC7550123
  31. Singhal A (2020). Predicting hydroxyl mediated nucleophilic degradation and molecular stability of RNA sequences through the application of deep learning methods. Arxiv Preprint Arxiv:2011.05136. https://doi.org/10.48550/arXiv.2011.05136   [Google Scholar]
  32. Ünal HT and Başçiftçi F (2022). Evolutionary design of neural network architectures: A review of three decades of research. Artificial Intelligence Review, 55(3): 1723-1802. https://doi.org/10.1007/s10462-021-10049-5   [Google Scholar]
  33. Vakharia V, Shah M, Nair P, Borade H, Sahlot P, and Wankhede V (2023). Estimation of lithium-ion battery discharge capacity by integrating optimized explainable-AI and stacked LSTM model. Batteries, 9(2): 125. https://doi.org/10.3390/batteries9020125   [Google Scholar]
  34. Wang Y (2021). Predicting the degradation of COVID-19 mRNA vaccine with graph convolutional networks. In the Proceedings of the 2021 6th International Conference on Machine Learning Technologies, ACM, New York, USA: 111–116. https://doi.org/10.1145/3468891.3468907   [Google Scholar]
  35. Wayment-Steele HK, Kim DS, Choe CA, Nicol JJ, Wellington-Oguri R, Watkins AM, and Das R (2021). Theoretical basis for stabilizing messenger RNA through secondary structure design. Nucleic Acids Research, 49(18): 10604-10617. https://doi.org/10.1093/nar/gkab764   [Google Scholar] PMid:34520542 PMCid:PMC8499941
  36. Xiao Y, Tang Z, Huang X, Chen W, Zhou J, Liu H, Liu C, Kong N, and Tao W (2022). Emerging mRNA technologies: Delivery strategies and biomedical applications. Chemical Society Reviews, 51(10): 3828-3845. https://doi.org/10.1039/D1CS00617G   [Google Scholar] PMid:35437544
  37. Xu S, Yang K, Li R, and Zhang L (2020). mRNA vaccine era—Mechanisms, drug platform and clinical prospection. International Journal of Molecular Sciences, 21(18): 6582. https://doi.org/10.3390/ijms21186582   [Google Scholar] PMid:32916818 PMCid:PMC7554980
  38. Zhang C, Maruggi G, Shan H, and Li J (2019). Advances in mRNA vaccines for infectious diseases. Frontiers in Immunology, 10: 429065. https://doi.org/10.3389/fimmu.2019.00594   [Google Scholar] PMid:30972078 PMCid:PMC6446947