ESTRO 2020 Abstract book

S374 ESTRO 2020

reliable approach to automated organ-at-risk (OAR) contour QA. Material and Methods DL models were trained to generate contours for the parotid (PG) and submandibular glands (SMG), using 1418/1152 non curated clinical training cases. Two metrics, Sørensen–Dice coefficient (SDC) and Hausdorff distance (HD), were used to assess the agreement between the DL and clinical contours. The approach was tested on 62 patients from the EORTC-1219-DAHANCA-29 clinical trial. To test the DL models’ ability to detect sub-optimal contours, 3 types of systematic errors (expansion, contraction and displacement) were gradually applied to all clinical contours (which for this purpose was considered the ground-truth) and the effect on SDC and HD was evaluated. The agreement between the DL and clinical OAR contours was then evaluated using a threshold for SDC of the average for all clinical cases-1 standard deviation (SD), and for HD of the average+1SD. All contours in the original data highlighted as potentially sub-optimal were visually inspected by a Radiation Oncologist and Medical Physicist. A sample of the non-highlighted contours (same size as number of highlighted contours) was inspected for false-negatives. Results The DL model performed similarly on our in-house training cases (using cross validation) and trial data: SDC for PG/SMG were 0.84/0.85 for both data sets. Increasing the magnitude of all 3 types of deliberate error resulted in progressively severe deterioration/increase in the test- set’s average SDC/HD. Out of 124 clinical PG contours, 19 were highlighted as potentially sub-optimal contours based on either SDC, HD or both. After visual inspection, 5 of these 19 (26%) clinical contours were deemed sub-optimal versus 2 out of 19 non-highlighted contours (11%). Out of 69 clinical SMG contours, 15 were highlighted based either SDC, HD or both. After visual inspection, 7 of these 15 (47%) were deemed sub-optimal versus 2 out of 15 non- highlighted contours (13%). Figure 1 shows a scatter plot with the thresholds for both metrics (sub-optimal contour=red cross). For 9/14 PG and 6/8 SMG contours where quality was deemed clinically acceptable, clear causes for low agreement were found: OAR deformation (e.g. secondary to displacement by tumor), deviating CT slice thickness, missing clinical contour or missing PG anterior lobe in the DL contour.

Conclusion Automated DL-based contour QA is feasible but visual inspection remains essential. We had a substantial number of false positive flags, due to sub-optimal performance of the DL model, especially for unusual anatomical deviations and deviating CT slice thickness. DL model improvement to better handle “outlier” cases will facilitate the adoption of DL-based contour QA into clinical trial QA and routine clinical practice. PH-0608 Identifying systematic edits in the clinical use of Deep Learning Contours J. Mateo 1 , P. Aljabar 1 , C. Brouwer 2 , S. Both 2 , M. Gooding 1 , H. Langendijk 2 1 Mirada Medical, Science, Oxford, United Kingdom ; 2 University Medical Centre Groningen, Radiotherapy, Groningen, The Netherlands Purpose or Objective To identify systematic edits of Deep-Learning Contouring (DLC) auto-contours of organs at risk (OAR) structures in a CT images were acquired for 77 head and neck cancer patients undergoing radiation therapy (RT) at the University Medical Centre Groningen. Contours for 21 OARs were obtained using an automated deep learning model (DLC Expert TM , Mirada Medical Ltd). Auto-contours were edited by clinical experts to ensure clinical validity for treatment planning, edited and raw (unedited) structures were then analysed. Not all OAR were present in all cases, resulting in variable counts for each structure. Edited structures were converted into meshes with scalar data representing the surface displacement to the corresponding raw contours. After rigid registration, and scaling to reference structures using an Iterative Closest Point algorithm, structures were re-meshed to a common topology with point correspondence. head and neck model. Material and Methods