MVision Contour+ improves consistency and saves time, according to HARMONY, a multi-institutional international study. The research was conducted in seven clinics from four countries and evaluated 18 regions of interest on 35 CT scans. The evaluation had two phases and included both organs at risk and lymph node levels.
In May 2025, the results were published in npj Digital Medicine, which is an international, peer-reviewed journal dedicated to publishing the highest quality research relevant to all aspects of digital medicine and health. The publication is part of the Nature portfolio (1).
Previous studies and a new perspective
Contouring the organs at risk (OARs) and target volumes in radiotherapy has changed due to the emergence of deep learning (DL)-based auto-segmentation models. In head-and-neck cancer, manual delineation is particularly challenging due to the anatomical complexity and the large number of OARs and lymph node levels. This often leads to significant variations in delineation between professionals. AI-based auto-segmentation addresses these challenges by significantly reducing time and minimizing inter-observer variability.
Many of the previously published studies have focused on algorithm development and single-institution evaluations. Previous research on the Contour+ Head and Neck model has shown significant time saving. A study by Strolin et. al, for example, revealed up to 73% time saving. In the same study, after analyzing a total of 2100 structures on scans from the head and neck region, the satisfaction grade was 4.08 from a maximum of five (2). In another study, published by Doolan P et al, when testing the benefit of Contour+ for head and neck cancer patients, and including 27 structures, a time saving of 89.9% was reported (3).
Additionally, MVision previously contributed to creating unique maxillary and mandibular substructures that can be used to refine the treatment planning for the head and neck cancers radiotherapy, through a collaboration project with Charité University Clinic, Berlin (4). Another important contribution in this field supported by MVision is the ELAISA study, published by Rasmussen ME et al, which involved institutions from 23 countries. The results showed the potential of Artificial Intelligence in education and improving contouring skills for head and neck cancer radiotherapy (5).
The HARMONY project (HeAd neck Rapid deep-learning auto-segmentation tool – a Multi-clinic evaluatiON studY), presented below, aimed to provide new data, including participants and scans from different geographic regions and to identify the hypothetical heterogeneity of the auto-segmenting software benefits across institutions. It is worth mentioning that the evaluation included distinct assessment of lymph node levels, an aspect not previously covered in a multi-center setting.
A brief description of the study design
This research involved seven clinics from four countries (Singapore, Finland, the Netherlands and Estonia) and had two phases:
Phase 1 evaluated the time savings and accuracy of the auto-segmentation algorithm within each institution using their own CT scans.
A radiation oncologist from each clinic manually contoured the 18 structures on five anonymized CT scans (the bilateral parotid glands, submandibular glands, brachial plexus and lymph node levels IA, IB, II, III, IVA and IVB and the constrictor muscle). MVision AI generated the auto-segmentations by DL and after a minimum two weeks, the same clinician reviewed and edited the contours, when needed.
Phase 2 investigated time savings and reduction in inter-observer variability by pooling scans from different institutions.
Two scans were randomly selected from each clinic (one for manual contouring, and the other one for editing of auto-segmented contours). Each clinic received scans from other clinics to evaluate if the benefit is similar as in phase one. Additionally, the inter-observer variation was compared using similarity metrics (Dice score and Hausdorf Distance). The first approach was to take as reference a consensus contour made from all seven manual contours. The second approach was to calculate pairwise contour similarity metrics among the seven manual contours.
Using Contour+ saves time
The analysis revealed consistent time savings with Contour+, with key findings summarized below:
- Statistically significant time reduction was observed for the majority of the ROIs in both phases.
- Lymph node levels had the highest variability in editing time, reflecting different contouring styles among clinics. Lymph node level IVB went through the most extensive editing by clinicians.
- Parotid glands and brachial plexus benefitted the most with more than 50% reduction in contouring time.
- The time saving was non-uniform across the clinics.
- The total contouring time in phase one decreased from an average of 90.7 min to 52.2 min, and in phase two decreased from 74.4 min to 37.8 min.
Inter-observer variation improves when using Contour+
In addition, the use of Contour+ had a positive impact on consistency, as highlighted by the following observations:
- The inter-observer variation in contouring decreased when clinicians edited the auto-segmentation results, compared to manually contouring.
- The brachial plexus showed the most significant reduction in inter-observer variability.
- All the LN contours showed an enhanced consistency, with LN IVB showing the highest average improvement.
What is the significance of the HARMONY study findings?
This multi-institutional and inter-continental study is the first to demonstrate the heterogeneity of clinical benefits of employing head-and-neck auto-segmentation software.
The edited contours were closer to the auto-segmentation compared to the original manual contours, highlighting the value of auto-segmentation in harmonizing the contouring practice, globally. The consistency plays an important role when performing data pooling analysis in multi-institutional clinical trials.
The differential time saving benefits across the ROIs and clinics can be attributed to multiple factors such as the structure complexity, experience of the clinician and the contouring software in the institution.
Clinics can optimize auto-segmentation software by identifying ROIs that require more editing time than manual contouring, and then omitting auto-segmentation for these specific ROIs. This optimization should occur during software commissioning, following a thorough assessment.
In summary
The HARMONY study results address professionals treating head and neck cancer patients using radiation therapy. The main findings are that
- Overall time savings were found to be 42% in phase one and 49% in phase two.
- Lymph node levels IA, IB, III, IVA, and IVB showed the greatest variability in contouring styles across clinics.
- All the edited ROIs showed reduced inter-observer variability compared to manual segmentation, showing the role of quality auto-segmentation software in harmonizing contouring practices globally.
- The clinical benefits of auto-segmentation can vary significantly across ROIs and between clinics, so Institution-specific commissioning is recommended.
MVision AI acknowledges the effort of investigators from the seven clinics which collaborated for this study: National Cancer Centre Singapore; Docrates Cancer Center, Kuopio University Hospital, Oulu University Hospital, and Turku University Hospital, Finland; Erasmus MC Cancer Institute, Rotterdam, Netherlands; North Estonia Medical Centre, Estonia.
References
- Pang EPP, Tan HQ, Wang F, et al. Multicentre evaluation of deep learning CT autosegmentation of the head and neck region for radiotherapy. NPJ Digit Med. 2025;8(1):312. Published 2025 May 27. doi:10.1038/s41746-025-01624-z
- Strolin S, Santoro M, Paolani G, et al. How smart is artificial intelligence in organs delineation? Testing a CE and FDA-approved Deep-Learning tool using multiple expert contours delineated on planning CT images. Front Oncol. 2023;13:1089807. Published 2023 Mar 2. doi:10.3389/fonc.2023.1089807
- Doolan PJ, Charalambous S, Roussakis Y, et al. A clinical evaluation of the performance of five commercial artificial intelligence contouring systems for radiotherapy. Front Oncol. 2023;13:1213068. Published 2023 Aug 4. doi:10.3389/fonc.2023.1213068
- Melerowitz L, Sreenivasa S, Nachbar M, et al. Design and evaluation of a deep learning-based automatic segmentation of maxillary and mandibular substructures using a 3D U-Net. Clin Transl Radiat Oncol. 2024;47:100780. Published 2024 Apr 18. doi:10.1016/j.ctro.2024.100780
- Rasmussen ME, Akbarov K, Titovich E, et al. Potential of E-Learning Interventions and Artificial Intelligence-Assisted Contouring Skills in Radiotherapy: The ELAISA Study. JCO Glob Oncol. 2024;10:e2400173. doi:10.1200/GO.24.00173
