Retinal Lesion Segmentation Using Transfer Learning with an Encoder-Decoder CNN
DOI:
https://doi.org/10.17488/RMIB.43.2.4Keywords:
Transfer learning, encoder-decoder, retinal images, lesion segmentation, deep learningAbstract
Deep learning (DL) techniques achieve high performance in the detection of illnesses in retina images, but the majority of models are trained with different databases for solving one specific task. Consequently, there are currently no solutions that can be used for the detection/segmentation of a variety of illnesses in the retina in a single model. This research uses Transfer Learning (TL) to take advantage of previous knowledge generated during model training of illness detection to segment lesions with encoder-decoder Convolutional Neural Networks (CNN), where the encoders are classical models like VGG-16 and ResNet50 or variants with attention modules. This shows that it is possible to use a general methodology using a single fundus image database for the detection/segmentation of a variety of retinal diseases achieving state-of-the-art results. This model could be in practice more valuable since it can be trained with a more realistic database containing a broad spectrum of diseases to detect/segment illnesses without sacrificing performance. TL can help achieve fast convergence if the samples in the main task (Classification) and sub-tasks (Segmentation) are similar. If this requirement is not fulfilled, the parameters start from scratch.
Downloads
References
Trucco E, MacGillivray T, Xu Y (eds). Computational Retinal Image Analysis [Internet]. Cambridge, United States: Elsevier; 2019. 481p. Available from: https://doi.org/10.1016/B978-0-08-102816-2.09994-9
Prado-Serrano A, Guido-Jiménez MA, Camas-Benítez JT. Prevalencia de retinopatía diabética en población mexicana. Rev Mex Oftalmol [Internet]. 2009; 83(5):261–266. Available: https://www.medigraphic.com/pdfs/revmexoft/rmo-2009/rmo095c.pdf
Ricci E, Perfetti R. Retinal Blood Vessel Segmentation Using Line Operators and Support Vector Classification. IEEE Trans Med Imaging [Internet]. 2007;26(10):1357-1365. Available from: https://doi.org/10.1109/TMI.2007.898551
Sarhan MH, Nasseri MA, Zapp D, Maier M, et al. Machine Learning Techniques for Ophthalmic Data Processing: A Review. IEEE J Biomed Health Inform [Internet]. 2020;24(12):3338–3350. Available from: http://dx.doi.org/10.1109/JBHI.2020.3012134
Ammu R, Sinha N. Small Segment Emphasized Performance Evaluation Metric for Medical Images. 2020 International Conference on Signal Processing and Communications (SPCOM) [Internet]. Bangalore : IEEE;2020; 1–5. Available from: http://dx.doi.org/10.1109/SPCOM50965.2020.9179617
Quellec G, Charrière K, Boudi Y, Cochener B, et al. Deep image mining for diabetic retinopathy screening. Med Image Anal [Internet]. 2017;39:178–193. Available from: https://doi.org/10.1016/j.media.2017.04.012
Zhang X, Thibault G, Decencière E, Marcotegui B, et al. Exudate detection in color retinal images for mass screening of diabetic retinopathy. Med Image Anal [Internet]. 2014;18(7):1026–1043. Available from: https://doi.org/10.1016/j.media.2014.05.004
Feng Z, Yang J, Yao L, Qiao Y, et al. Deep Retinal Image Segmentation: A FCN-Based Architecture with Short and Long Skip Connections for Retinal Image Segmentation. In: Liu D, Xie S, Li Y, Zhao D, et al. (eds). Neural Information Processing ICONIP 2017. Lecture Notes in Computer Science, vol. 10637 [Internet]. Cham: Springer; 2017; 713-722. Available from: https://doi.org/10.1007/978-3-319-70093-9_76
Ye JC, Sung WK. Understanding Geometry of Encoder-Decoder CNNs. 36th International Conference on Machine Learning, ICML 2019 [Internet]. Long Beach: Proceedings of Machine Learning Research; 2019;97:12245–12254. Available from: https://proceedings.mlr.press/v97/ye19a.html
Tan C, Sun F, Kong T, Zhang W, et al. A Survey on Deep Transfer Learning. In: Manolopoulos Y, Hammer B, Iliadis L, Maglogiannis I (eds). Artificial Neural Networks and Machine Learning – ICANN 2018. ICANN 2018. Lecture Notes in Computer Science [Internet]. Cham: Springer; 2018. 11041: 270–279. Available from: http://dx.doi.org/10.1007/978-3-030-01424-7_27
Tang S, Qi Z, Granley J, Beyeler M. U-Net with Hierarchical Bottleneck Attention for Landmark Detection in Fundus Images of the Degenerated Retina. In: Fu H, Garvin MK, MacGillivray T, Xu Y, et al (eds). Ophthalmic Medical Image Analysis. OMIA 2021. Lecture Notes in Computer Science [Internet]. Cham: Springer; 2021. 12970:62–71. Available from: https://doi.org/10.1007/978-3-030-87000-3_7
Welikala RA, Foster PJ, Whincup PH, Rudnicka AR, et al. Automated arteriole and venule classification using deep learning for retinal images from the UK Biobank cohort. Comput Biol Med [Internet]. 2017;90:23–32. Available from: https://doi.org/10.1016/j.compbiomed.2017.09.005
Fraz MM, Rudnicka AR, Owen CG, Barman SA. Delineation of blood vessels in pediatric retinal images using decision trees-based ensemble classification. Int J Comput Assist Radiol Surg [Internet]. 2014;9:795–811. Available from: https://doi.org/10.1007/s11548-013-0965-9
Adel A, Soliman MM, Khalifa NEM, Mostafa K. Automatic Classification of Retinal Eye Diseases from Optical Coherence Tomography using Transfer Learning. 2020 16th International Computer Engineering Conference (ICENCO) [Internet]. Cairo: IEEE; 2020: 37–42. Available from: https://doi.org/10.1109/ICENCO49778.2020.9357324
Xiao Z, Adel M, Bourennane S. Bayesian Method with Spatial Constraint for Retinal Vessel Segmentation. Comput Math Methods Med [Internet]. 2013:401413. Available from: https://doi.org/10.1155/2013/401413
Lam C, Yu C, Huang L, Rubin D. Retinal Lesion Detection with Deep Learning Using Image Patches. Invest Ophthalmol Vis Sci [Internet]. 2018;59(1):590–596. Available from: https://doi.org/10.1167/iovs.17-22721
Li Q, Feng B, Xie L, Liang P, et al. A Cross-Modality Learning Approach for Vessel Segmentation in Retinal Images. IEEE Trans Med Imaging [Internet]. 2016;35(1):109–118. Available from: https://doi.org/10.1109/TMI.2015.2457891
Long S, Chen J, Hu A, Liu H, et al. Microaneurysms detection in color fundus images using machine learning based on directional local contrast. Biomed Eng Online [Internet]. 2020;19(1):21. Available from: https://doi.org/10.1186/s12938-020-00766-3
Aziz T, Ilesanmi AE, Charoenlarpnopparut C. Efficient and Accurate Hemorrhages Detection in Retinal Fundus Images Using Smart Window Features. Appl Sci [Internet]. 2021;11(14):6391. Available from: https://doi.org/10.3390/app11146391
Liu Q, Liu H, Zhao Y, Liang Y. Dual-Branch Network with Dual-Sampling Modulated Dice Loss for Hard Exudate Segmentation in Colour Fundus Images. IEEE J Biomed Health Inform [Internet]. 2022;26(3):1091-1102. Available from: https://doi.org/10.1109/jbhi.2021.3108169
Zong Y, Chen J, Yang L, Tao S, et al. U-net Based Method for Automatic Hard Exudates Segmentation in Fundus Images Using Inception Module and Residual Connection. IEEE Access [Internet]. 2020;8:167225–35. Available from: http://dx.doi.org/10.1109/ACCESS.2020.3023273
Tan JH, Fujita H, Sivaprasad S, Bhandary SV, et al. Automated segmentation of exudates, haemorrhages, microaneurysms using single convolutional neural network. Inf Sci [Internet]. 2017;420:66–76. Available from: https://doi.org/10.1016/j.ins.2017.08.050
Kou C, Li W, Yu Z, Yuan L. An Enhanced Residual U-Net for Microaneurysms and Exudates Segmentation in Fundus Images. IEEE Access [Internet]. 2020;8:185514–185525. Available from: http://dx.doi.org/10.1109/ACCESS.2020.3029117
Gondal WM, Köhler JM, Grzeszick R, Fink GA, et al. Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images. 2017 IEEE International Conference on Image Processing (ICIP) [Internet]. Beijing: IEEE; 2017; 2069-2073. Available from: https://doi.org/10.1109/ICIP.2017.8296646
Joshi GD, Sivaswamy J, Krishnadas SR. Optic Disk and Cup Segmentation From Monocular Color Retinal Images for Glaucoma Assessment. IEEE Trans Med Imaging [Internet]. 2011;30(6):1192–1205. Available from: https://doi.org/10.1109/TMI.2011.2106509
Harangi B, Antal B, Hajdu A. Automatic exudate detection with improved naïve-Bayes classifier. 2012 25th IEEE International Symposium on Computer-Based Medical Systems (CBMS) [Internet]. Rome: IEEE; 2012; 1-4. Available from: https://doi.org/10.1109/CBMS.2012.6266341
Cheung CY, Xu D, Cheng CY, Sabanayagam C, et al. A deep-learning system for the assessment of cardiovascular disease risk via the measurement of retinal-vessel caliber. Nat Biomed Eng [Internet]. 2021;5(6):498–508. Available from: https://doi.org/10.1038/s41551-020-00626-4
Dai L, Wu L, Li H, Cai C, et al. A deep learning system for detecting diabetic retinopathy across the disease spectrum. Nat Commun [Internet]. 2021;12(1):3242. Available from: https://doi.org/10.1038/s41467-021-23458-5
Kou C, Li W, Liang W, Yu Z, et al. Microaneurysms segmentation with a U-Net based on recurrent residual convolutional neural network. J Med Imaging [Internet]. 2019;6(2):025008. Available from: https://dx.doi.org/10.1117%2F1.JMI.6.2.025008
Xu X, Tan T, Xu F. An Improved U-Net Architecture for Simultaneous Arteriole and Venule Segmentation in Fundus Image. In: Nixon M, Mahmoodi S, Zwiggelaar R (eds). Medical Image Understanding and Analysis. MIUA 2018. Communications in Computer and Information Science [Internet]. Cham: Springer; 2018; 333–340. Available from: https://doi.org/10.1007/978-3-319-95921-4_31
Yadav G, Maheshwari S, Agarwal A. Contrast limited adaptive histogram equalization based enhancement for real time video system. 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI) [Internet]. Delhi: IEEE; 2014; 2392-2397. Available from: https://doi.org/10.1109/ICACCI.2014.6968381
Decencière E, Zhang X, Cazuguel G, Lay B, et al. Feedback on a publicly distributed image database: The Messidor database. Image Anal Stereol [Internet]. 2014;33(3):231. Available from: https://doi.org/10.5566/ias.1155
Kaggle. Diabetic Retinopathy Detection. [Internet] Kaggle. 2015. Available from: https://www.kaggle.com/c/diabetic-retinopathy-detection
Porwal P, Pachade S, Kamble R, Kokare M, et al. Indian Diabetic Retinopathy Image Dataset (IDRiD) [Internet]. IEEE Dataport; 2018. Available from: https://dx.doi.org/10.21227/H25W98
Decencière E, Cazuguel G, Zhang X, Thibault G, et al. TeleOphta: Machine learning and image processing methods for teleophthalmology. IRBM [Internet]. 2013;34(2):196–203. Available from: https://doi.org/10.1016/j.irbm.2013.01.010
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Bengio Y, LeCun Y (eds). 3rd International Conference on Learning Representations, ICLR 2015 [Internet]. San Diego: arXiv;2015; 1–14. Available from: https://doi.org/10.48550/arXiv.1409.1556
He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) [Internet]. Las Vegas: IEEE; 2016; 770-778. Available from: https://doi.org/10.1109/CVPR.2016.90
Woo S, Park J, Lee JY, Kweon IS. CBAM: Convolutional Block Attention Module. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds). Computer Vision – ECCV 2018. ECCV 2018. Lecture Notes in Computer Science [Internet]. Cham: Springer; 2018; 3–19. Available from: https://doi.org/10.1007/978-3-030-01234-2_1
Cortes C, Research G, Mohri M, Rostamizadeh A. L2 Regularization for Learning Kernels. 25th Conference on Uncertainty in Artificial Intelligence (UAI 2009) [Internet]. Montreal: Association for Uncertainty in Artificial Intelligence (AUAI); 2009; 109-116. Available from: https://dl.acm.org/doi/pdf/10.5555/1795114.1795128
Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37 (ICML'15) [Internet]. Lille: JMLR; 2015; 448-456. Available from: http://proceedings.mlr.press/v37/ioffe15.pdf
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, et al. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. J Mach Learn Res [Internet]. 2014; 15(56):1929–1958. Available from: http://jmlr.org/papers/v15/srivastava14a.html
Wisaeng K, Sa-Ngiamvibool W. Exudates Detection Using Morphology Mean Shift Algorithm in Retinal Images. IEEE Access [Internet]. 2019;7:11946–11958. Available from: http://dx.doi.org/10.1109/ACCESS.2018.2890426
van Grinsven MJJP, van Ginneken B, Hoyng CB, Theelen T, et al. Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images. IEEE Trans Med Imaging [Internet]. 2016;35(5):1273–1284Available from: https://doi.org/10.1109/TMI.2016.2526689
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Revista Mexicana de Ingeniería Biomédica
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Upon acceptance of an article in the RMIB, corresponding authors will be asked to fulfill and sign the copyright and the journal publishing agreement, which will allow the RMIB authorization to publish this document in any media without limitations and without any cost. Authors may reuse parts of the paper in other documents and reproduce part or all of it for their personal use as long as a bibliographic reference is made to the RMIB. However written permission of the Publisher is required for resale or distribution outside the corresponding author institution and for all other derivative works, including compilations and translations.
