Deep Learning-based Automated Detection of Facial Surgeries Using HDA Plastic Surgery Database

Abstract

Plastic surgery has gained significant traction in modern medicine, with procedures like facial bone correction and nose correction becoming increasingly popular. These surgeries often result in substantial changes to facial features, challenging traditional methods of image analysis and recognition. This study leverages the HDA Plastic Surgery Face Database and state-of-the-art deep learning models—Xception, Vision Transformer (ViT), and Swin Transformer—to classify facial images into five distinct categories: eyebrow correction, eyelid correction, facelifts, facial bones correction, and nose correction. The dataset was preprocessed with image augmentation, normalization, and resizing to enhance model performance. Each model was fine-tuned to capture the subtle variations introduced by different surgeries. Results demonstrate the effectiveness of deep learning in this domain, with Swin Transformer achieving the highest accuracy of 95.5%, precision of 96.9%, and sensitivity of 95.1%.

References

[1] Branford OA, Kamali P, Rohrich RJ, Song DH, Mallucci P, Liu DZ, Lang D, Sun K, Stubican M, Lin SJ. # PlasticSurgery. Plastic and reconstructive surgery. 2016 Dec 1;138(6):1354-65. 

[2] King TW. Plastic surgery. Building a Clinical Practice. 2020:203-14.

[3] Rahmani A, Norouzi F, Machado BL, Ghasemi F. Psychiatric Neurosurgery with Advanced Imaging and Deep Brain Stimulation Techniques. International Research in Medical and Health Sciences. 2024 Nov 1;7(5):63-74.

[4] Chuang J, Barnes C, Wong BJ. Overview of facial plastic surgery and current developments. The Surgery Journal. 2016 Mar;2(01):e17-28.

[5] Abbasi H, Afrazeh F, Ghasemi Y, Ghasemi F. A Shallow Review of Artificial Intelligence Applications in Brain Disease: Stroke, Alzheimer's, and Aneurysm. International Journal of Applied Data Science in Engineering and Health. 2024 Oct 5;1(2):32-43.

[6] Kumar Y, Koul A, Singla R, Ijaz MF. Artificial intelligence in disease diagnosis: a systematic literature review, synthesizing framework and future research agenda. Journal of ambient intelligence and humanized computing. 2023 Jul;14(7):8459-86.

[7] Ghaffar Nia N, Kaplanoglu E, Nasab A. Evaluation of artificial intelligence techniques in disease diagnosis and prediction. Discover Artificial Intelligence. 2023 Jan 30;3(1):5. 

[8] Zhang B, Shi H, Wang H. Machine learning and AI in cancer prognosis, prediction, and treatment selection: a critical approach. Journal of multidisciplinary healthcare. 2023 Dec 31:1779-91.

[9] Liao J, Li X, Gan Y, Han S, Rong P, Wang W, Li W, Zhou L. Artificial intelligence assists precision medicine in cancer treatment. Frontiers in oncology. 2023 Jan 4;12:998222.

[10] Ghasemi F, Minoo S. Dental Imaging Analysis with Artificial Intelligence. Available at SSRN 5008235. 2024 Oct 21.

[11] Kebaili A, Lapuyade-Lahorgue J, Ruan S. Deep learning approaches for data augmentation in medical imaging: a review. Journal of Imaging. 2023 Apr 13;9(4):81.

[12] Rezaei M, Molani S, Firoozeh N, Abbasi H, Vahedifard F, Orouskhani M. Evolving Tsukamoto Neuro Fuzzy Model for Multiclass Covid 19 Classification with Chest X Ray Images. arXiv preprint arXiv:2305.10421. 2023 May 17. 

[13] Radak M, Lafta HY, Fallahi H. Machine learning and deep learning techniques for breast cancer diagnosis and classification: a comprehensive review of medical imaging studies. Journal of Cancer Research and Clinical Oncology. 2023 Sep;149(12):10473-91.

[14] Orouskhani M, Zhu C, Rostamian S, Zadeh FS, Shafiei M, Orouskhani Y. Alzheimer's disease detection from structural MRI using conditional deep triplet network. Neuroscience Informatics. 2022 Dec 1;2(4):100066.

[15] Adam R, Dell’Aquila K, Hodges L, Maldjian T, Duong TQ. Deep learning applications to breast cancer detection by magnetic resonance imaging: a literature review. Breast Cancer Research. 2023 Jul 24;25(1):87.

[16] Bosma JS, Saha A, Hosseinzadeh M, Slootweg I, de Rooij M, Huisman H. Semisupervised learning with report-guided pseudo labels for deep learning–based prostate cancer detection using biparametric MRI. Radiology: Artificial Intelligence. 2023 Jul 26;5(5):e230031.

[17] Conze PH, Andrade-Miranda G, Singh VK, Jaouen V, Visvikis D. Current and emerging trends in medical image segmentation with deep learning. IEEE Transactions on Radiation and Plasma Medical Sciences. 2023 Apr 10;7(6):545-69. 

[18] Jiang H, Diao Z, Shi T, Zhou Y, Wang F, Hu W, Zhu X, Luo S, Tong G, Yao YD. A review of deep learning-based multiple-lesion recognition from medical images: classification, detection and segmentation. Computers in Biology and Medicine. 2023 May 1;157:106726.

[19] Ahmad A, Tariq A, Hussain HK, Gill AY. Revolutionizing Healthcare: How Deep Learning is poised to Change the Landscape of Medical Diagnosis and Treatment. Journal of Computer Networks, Architecture and High Performance Computing. 2023 Jul 11;5(2):458-71. 

[20] Rana MS, Shuford J. AI in Healthcare: Transforming Patient Care through Predictive Analytics and Decision Support Systems. Journal of Artificial Intelligence General Science (JAIGS) ISSN: 3006-4023. 2024 Jan 22;1(1). 

[21] Choi E, Leonard KW, Jassal JS, Levin AM, Ramachandra V, Jones LR. Artificial intelligence in facial plastic surgery: a review of current applications, future applications, and ethical considerations. Facial Plastic Surgery. 2023 Jun 23.

[22] Morris MX, Rajesh A, Asaad M, Hassan A, Saadoun R, Butler CE. Deep learning applications in surgery: Current uses and future directions. The American Surgeon. 2023 Jan;89(1):36-42.

[23] Kato RM, Parizotto JD, Oliveira PH, Gonçalves JR. Artificial intelligence in orthognathic surgery–a narrative review of surgical digital tools and 3D orthognathic surgical planning. Journal of the California Dental Association. 2023 Dec 31;51(1):2202444. 

[24] Atiyeh B, Emsieh S, Hakim C, Chalhoub R. A narrative review of artificial intelligence (AI) for objective assessment of aesthetic endpoints in plastic surgery. Aesthetic Plastic Surgery. 2023 Dec;47(6):2862-73.

[25] Zhang H, Zhao J, Farzan R, Alizadeh Otaghvar H. Risk predictions of surgical wound complications based on a machine learning algorithm: A systematic review. International Wound Journal. 2024 Jan;21(1):e14665.

[26] Elfanagely O, Toyoda Y, Othman S, Mellia JA, Basta M, Liu T, Kording K, Ungar L, Fischer JP. Machine learning and surgical outcomes prediction: a systematic review. Journal of Surgical Research. 2021 Aug 1;264:346-61. 

[27] Boonipat T, Asaad M, Lin J, Glass GE, Mardini S, Stotland M. Using artificial intelligence to measure facial expression following facial reanimation surgery. Plastic and reconstructive surgery. 2020 Nov 1;146(5):1147-50.

[28] Murphy DC, Saleh DB. Artificial intelligence in plastic surgery: what is it? Where are we now? What is on the horizon?. The Annals of The Royal College of Surgeons of England. 2020 Oct;102(8):577-80.

[29] Mirchi N, Bissonnette V, Yilmaz R, Ledwos N, Winkler-Schwartz A, Del Maestro RF. The Virtual Operative Assistant: An explainable artificial intelligence tool for simulation-based training in surgery and medicine. PloS one. 2020 Feb 27;15(2):e0229596.

[30] Tseng PY, Chen YT, Wang CH, Chiu KM, Peng YS, Hsu SP, Chen KL, Yang CY, Lee OK. Prediction of the development of acute kidney injury following cardiac surgery by machine learning. Critical care. 2020 Dec;24:1-3.

[31] Khalid S, Goldenberg M, Grantcharov T, Taati B, Rudzicz F. Evaluation of deep learning models for identifying surgical actions and measuring performance. JAMA network open. 2020 Mar 2;3(3):e201664-.

[32] Rathgeb C, Dogan D, Stockhardt F, De Marsico M, Busch C. Plastic surgery: An obstacle for deep face recognition?. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops 2020 (pp. 806-807).

[33] Chollet F. Xception: Deep learning with depthwise separable convolutions. InProceedings of the IEEE conference on computer vision and pattern recognition 2017 (pp. 1251-1258).

[34] Dosovitskiy A. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. 2020.

[35] Khan M, Hanan A, Kenzhebay M, Gazzea M, Arghandeh R. Transformer-based land use and land cover classification with explainability using satellite imagery. Scientific Reports. 2024 Jul 20;14(1):16744.

[36] Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B. Swin transformer: Hierarchical vision transformer using shifted windows. InProceedings of the IEEE/CVF international conference on computer vision 2021 (pp. 10012-10022).