Browsing by Author "Han, Seung Seog"

Now showing 1 - 3 of 3
  • No Thumbnail Available
    Item
    Evaluation of Artificial Intelligence-Assisted Diagnosis of Skin Neoplasms: A Single-Center, Paralleled, Unmasked, Randomized Controlled Trial
    (2022) Han, Seung Seog; Kim, Young Jae; Moon, Ik Jun; Jung, Joon Min; Lee, Mi Young; Lee, Woo Jin; Won, Chong Hyun; Lee, Mi Woo; Kim, Seong Hwan; Navarrete-Dechent, Cristian; Chang, Sung Eun
    Trial design: This was a single-center, unmasked, paralleled, randomized controlled trial. Methods: A randomized trial was conducted in a tertiary care institute in South Korea to validate whether artificial intelligence (AI) could augment the accuracy of nonexpert physicians in the real-world settings, which included diverse outof-distribution conditions. Consecutive patients aged >19 years, having one or more skin lesions suspicious for skin cancer detected by either the patient or physician, were randomly allocated to four nondermatology trainees and four dermatology residents. The attending dermatologists examined the randomly allocated patients with (AI-assisted group) or without (unaided group) the real-time assistance of AI algorithm (https:// b2020.modelderm.com#world; convolutional neural networks; unmasked design) after simple randomization of the patients. Results: Using 576 consecutive cases (Fitzpatrick skin phototypes III or IV) with suspicious lesions out of the initial 603 recruitments, the accuracy of the AI-assisted group (n = 295, 53.9%) was found to be significantly higher than those of the unaided group (n = 281, 43.8%; P = 0.019). Whereas the augmentation was more significant from 54.7% (n = 150) to 30.7% (n = 138; P < 0.0001) in the nondermatology trainees who had the least experience in dermatology, it was not significant in the dermatology residents. The algorithm could help trainees in the AI-assisted group include more differential diagnoses than the unaided group (2.09 vs. 1.95 diagnoses; P = 0.0005). However, a 12.2% drop in Top-1 accuracy of the trainees was observed in cases in which all Top-3 predictions given by the algorithm were incorrect. Conclusions: The multiclass AI algorithm augmented the diagnostic accuracy of nonexpert physicians in dermatology.
  • No Thumbnail Available
    Item
    Generation of a Melanoma and Nevus Data Set From Unstandardized Clinical Photographs on the Internet
    (2023) Cho, Soo Ick; Navarrete-Dechent, Cristian; Daneshjou, Roxana; Cho, Hye Soo; Chang, Sung Eun; Kim, Seong Hwan; Na, Jung-Im; Han, Seung Seog
    ImportanceArtificial intelligence (AI) training for diagnosing dermatologic images requires large amounts of clean data. Dermatologic images have different compositions, and many are inaccessible due to privacy concerns, which hinder the development of AI.ObjectiveTo build a training data set for discriminative and generative AI from unstandardized internet images of melanoma and nevus.Design, Setting, and ParticipantsIn this diagnostic study, a total of 5619 (CAN5600 data set) and 2006 (CAN2000 data set; a manually revised subset of CAN5600) cropped lesion images of either melanoma or nevus were semiautomatically annotated from approximately 500 000 photographs on the internet using convolutional neural networks (CNNs), region-based CNNs, and large mask inpainting. For unsupervised pretraining, 132 673 possible lesions (LESION130k data set) were also created with diversity by collecting images from 18 482 websites in approximately 80 countries. A total of 5000 synthetic images (GAN5000 data set) were generated using the generative adversarial network (StyleGAN2-ADA; training, CAN2000 data set; pretraining, LESION130k data set).Main Outcomes and MeasuresThe area under the receiver operating characteristic curve (AUROC) for determining malignant neoplasms was analyzed. In each test, 1 of the 7 preexisting public data sets (total of 2312 images; including Edinburgh, an SNU subset, Asan test, Waterloo, 7-point criteria evaluation, PAD-UFES-20, and MED-NODE) was used as the test data set. Subsequently, a comparative study was conducted between the performance of the EfficientNet Lite0 CNN on the proposed data set and that trained on the remaining 6 preexisting data sets.ResultsThe EfficientNet Lite0 CNN trained on the annotated or synthetic images achieved higher or equivalent mean (SD) AUROCs to the EfficientNet Lite0 trained using the pathologically confirmed public data sets, including CAN5600 (0.874 [0.042]; P = .02), CAN2000 (0.848 [0.027]; P = .08), and GAN5000 (0.838 [0.040]; P = .31 [Wilcoxon signed rank test]) and the preexisting data sets combined (0.809 [0.063]) by the benefits of increased size of the training data set.Conclusions and RelevanceThe synthetic data set in this diagnostic study was created using various AI technologies from internet images. A neural network trained on the created data set (CAN5600) performed better than the same network trained on preexisting data sets combined. Both the annotated (CAN5600 and LESION130k) and synthetic (GAN5000) data sets could be shared for AI training and consensus between physicians.
  • No Thumbnail Available
    Item
    The degradation of performance of a state-of-the-art skin image classifier when applied to patient-driven internet search
    (2022) Han, Seung Seog; Navarrete-Dechent, Cristian; Liopyris, Konstantinos; Kim, Myoung Shin; Park, Gyeong Hun; Woo, Sang Seok; Park, Juhyun; Shin, Jung Won; Kim, Bo Ri; Kim, Min Jae; Donoso, Francisca; Villanueva, Francisco; Ramirez, Cristian; Chang, Sung Eun; Halpern, Allan; Kim, Seong Hwan; Na, Jung-Im
    Model Dermatology (; Build2021) is a publicly testable neural network that can classify 184 skin disorders. We aimed to investigate whether our algorithm can classify clinical images of an Internet community along with tertiary care center datasets. Consecutive images from an Internet skin cancer community ('RD' dataset, 1,282 images posted between 25 January 2020 to 30 July 2021; ) were analyzed retrospectively, along with hospital datasets (Edinburgh dataset, 1,300 images; SNU dataset, 2,101 images; TeleDerm dataset, 340 consecutive images). The algorithm's performance was equivalent to that of dermatologists in the curated clinical datasets (Edinburgh and SNU datasets). However, its performance deteriorated in the RD and TeleDerm datasets because of insufficient image quality and the presence of out-of-distribution disorders, respectively. For the RD dataset, the algorithm's Top-1/3 accuracy (39.2%/67.2%) and AUC (0.800) were equivalent to that of general physicians (36.8%/52.9%). It was more accurate than that of the laypersons using random Internet searches (19.2%/24.4%). The Top-1/3 accuracy was affected by inadequate image quality (adequate = 43.2%/71.3% versus inadequate = 32.9%/60.8%), whereas participant performance did not deteriorate (adequate = 35.8%/52.7% vs. inadequate = 38.4%/53.3%). In this report, the algorithm performance was significantly affected by the change of the intended settings, which implies that AI algorithms at dermatologist-level, in-distribution setting, may not be able to show the same level of performance in with out-of-distribution settings.