673
EFFECT OF AN ARTIFICIAL INTELLIGENCE-ASSISTED ENDOSCOPY IN THE DIAGNOSIS OF SUPERFICIAL GASTRIC NEOPLASMS: A MULTICENTER PROSPECTIVE RANDOMIZED CONTROL TRIAL(GASTRO-AI-STUDY)
Date
May 20, 2024
Aim: The significant benefit of an artificial intelligence (AI) diagnostic support system for early gastric cancer (gastroAI™-modelG - SG: AI Medical Service, Tokyo, Japan) in preclinical studies have reported. This RCT was designed to evaluate the effect of the AI assistance in a clinical setting.
Methods: We used the AI system, which displays the diagnostic confidence for the gastric neoplasia in a still white light image of a target lesion. The cut-off of the confidence level for the diagnosis of neoplasia is set at 60%; a lesion with a confidence level < 60% is indicated as low confidence, and a lesion = or >60% is indicated as neoplasia with a percentage of level. Patients undergoing endoscopy for preoperative evaluation prior to endoscopic resection (ER) of gastric neoplasia or for surveillance after gastric ER were enrolled. After patients were randomized (1:1) to AI-assisted and non-AI-assisted groups, a non-expert endoscopist blinded to patient clinical information performed white light EGD diagnosis with or without the AI assistance. Next, an expert endoscopist with prior patient clinical information performed white light EGD diagnosis, followed by NBI magnification. The primary endpoint was non-expert diagnostic accuracy.
Results: Of the 312 patients enrolled, 155 patients (143 neoplasms, 96 non-neoplasms) in the AI-assisted group and 142 patients.122 neoplasms, 68 non-neoplasms) in the non-AI-assisted group were analyzed. The patient background (age, gender, neoplasm/non-neoplasm ratio, target lesion size) of the two groups was identical. Non-expert diagnostic accuracy was 65.3% and 59.9% in the AI-assisted and non-AI-assisted groups, respectively (p=0.24). Sensitivity was 68.6% vs 63.9% (p=0.42); specificity 60.8% vs 53.3% (p=0.34); PPV 72.1% vs 70.9% (p=0.83); NPV 56.6% vs 45.5% (p=0.13), in the AI-assisted vs non-AI-assisted group of the non-experts. The AUC of ROC analysis for the confidence level of the AI was 0.691, and a cut-off value for the confidence level at the highest Youden’s index was 78.5%. In the AI-assisted group, the endoscopists changed the final diagnosis from neoplasia to non-neoplasia or from non-neoplasia to neoplasia in 5% of lesions after the AI assistance; the changed diagnoses were pathologically correct in 91% of cases.
Conclusions: This RCT showed that 1) non-expert diagnostic accuracy, sensitivity and specificity increased by 5 to 7% with the AI-assistance, but the increases were not significant, 2) the positive interaction between the AI and endoscopist occurred in 5% of target lesions. Further improvement of the AI performance and the confidence level setting could lead to more positive interaction between the AI and endoscopist.
Methods: We used the AI system, which displays the diagnostic confidence for the gastric neoplasia in a still white light image of a target lesion. The cut-off of the confidence level for the diagnosis of neoplasia is set at 60%; a lesion with a confidence level < 60% is indicated as low confidence, and a lesion = or >60% is indicated as neoplasia with a percentage of level. Patients undergoing endoscopy for preoperative evaluation prior to endoscopic resection (ER) of gastric neoplasia or for surveillance after gastric ER were enrolled. After patients were randomized (1:1) to AI-assisted and non-AI-assisted groups, a non-expert endoscopist blinded to patient clinical information performed white light EGD diagnosis with or without the AI assistance. Next, an expert endoscopist with prior patient clinical information performed white light EGD diagnosis, followed by NBI magnification. The primary endpoint was non-expert diagnostic accuracy.
Results: Of the 312 patients enrolled, 155 patients (143 neoplasms, 96 non-neoplasms) in the AI-assisted group and 142 patients.122 neoplasms, 68 non-neoplasms) in the non-AI-assisted group were analyzed. The patient background (age, gender, neoplasm/non-neoplasm ratio, target lesion size) of the two groups was identical. Non-expert diagnostic accuracy was 65.3% and 59.9% in the AI-assisted and non-AI-assisted groups, respectively (p=0.24). Sensitivity was 68.6% vs 63.9% (p=0.42); specificity 60.8% vs 53.3% (p=0.34); PPV 72.1% vs 70.9% (p=0.83); NPV 56.6% vs 45.5% (p=0.13), in the AI-assisted vs non-AI-assisted group of the non-experts. The AUC of ROC analysis for the confidence level of the AI was 0.691, and a cut-off value for the confidence level at the highest Youden’s index was 78.5%. In the AI-assisted group, the endoscopists changed the final diagnosis from neoplasia to non-neoplasia or from non-neoplasia to neoplasia in 5% of lesions after the AI assistance; the changed diagnoses were pathologically correct in 91% of cases.
Conclusions: This RCT showed that 1) non-expert diagnostic accuracy, sensitivity and specificity increased by 5 to 7% with the AI-assistance, but the increases were not significant, 2) the positive interaction between the AI and endoscopist occurred in 5% of target lesions. Further improvement of the AI performance and the confidence level setting could lead to more positive interaction between the AI and endoscopist.
