COMPARATIVE EVALUATION OF ARTIFICIAL INTELLIGENCE AND ENDOSCOPISTS' ACCURACY IN ENDOSCOPIC ULTRASOUND FOR IDENTIFYING NORMAL ANATOMICAL STRUCTURES: A MULTI-INSTITUTIONAL, CROSS-SECTIONAL STUDY

Background/aim: Endoscopic ultrasound (EUS) has emerged as a powerful diagnostic and therapeutic tool in gastroenterology, providing detailed images of the structure’s layers, as well as nearby structures. However, EUS is operator-dependent and variability in operator proficiency results in discrepancies in diagnostic accuracy. Artificial intelligence (AI) can be trained to process and interpret EUS images in real-time, potentially mitigating the operator-dependent variability and improving diagnostic accuracy. Moreover, AI can analyze vast datasets offering a level of consistency and efficiency that is challenging for human operators to match. The aim of this study is to compare the diagnostic accuracy of an AI-based model (AIWorks-EUS, mdconsgroup, Guayaquil, Ecuador) against that of expert and nonexpert gastrointestinal endoscopists visual impression in the identification of normal anatomical structures during EUS procedures.
Methods: A multi-institutional, cross-sectional, comparative study designed to assess the diagnostic accuracy of the AIWorks-EUS (mdconsgroup, Guayaquil, Ecuador), in identifying normal anatomical structures during EUS procedures compared to that of endoscopists of different expertise. Consecutive patient videos of linear-array EUS were included for analysis. Then, the video dataset was reviewed by 9 endoscopists (5 expert endoscopists and 4 nonexpert endoscopists (<250 EUS procedures), who were blinded to the AI model interpretations. Finally, their visual impression was compared to the observation outputs provided by the AI models.
Results: A total of 29 normal anatomic structures were evaluated by the endoscopists and the AIWorks-EUS software in 39 videos of linear-array EUS. The software reported an observed agreement (OA) of 100% in 15/29 structures (4/9 mediastinal window, 6/12 in gastric window, 5/8 in duodenal window). Subcarinal space and portal vein (gastric window) obtained the lowest OA (92.35%) (Table 1). The AIWorks-EUS software obtained a pooled sensitivity, specificity, positive (PPV) and negative predictive value (NPV), and OA for anatomical structure recognition of 95.2%, 99.3%, 97.9%, 98.4%, and 98.2%, respectively. Additionally, the software obtained higher diagnostic accuracy than expert and nonexpert endoscopist (P <0.05) (Table 2). Agreement between endoscopists and the AI per organ evaluated was obtained, with a difference of 18% between the organ with higher agreement (Aorta) and the lower agreement (Ampulla) (Figure 1).
Conclusion: The EUS-based AI model achieved higher diagnostic accuracy than both expert and nonexpert endoscopists in the identification of normal anatomical structures. This model can provide assistance to both groups during live procedures, and studies evaluating its use for training should be conducted to evaluate its application in EUS training programs.