S Warren, N Richmond
MVision AI software has been under evaluation at NCCC (Newcastle upon Tyne Hospitals NHS Foundation Trust) since 28 February 2022* in order to:
1. compare AI contour quality and consistency with manual contouring;
2. estimate potential improvements in efficiency;
3. identify opportunities to improve clinical workflow.
In the preliminary phase of the project, the use of MVision for the following sites has tested:
1. Breast with lymph node volumes;
2. Head and Neck including lymph node volumes;
3. Lung SABR volumes;
4. Prostate and Nodes volumes.
1. Breast with lymph node volumes
An initial study of 15 retrospective patients at NCCC found median time required for manual lymph node volume delineation by consultant oncologist(s) was 35 mins (range 30-45mins) , and up to 60 minutes in the case of bilateral breast patients.
Previous clinical practice at NCCC therefore did not routinely delineate these volumes for all patients (due to the clinician time required) and used bony anatomy to determine treatment field boundaries, in a semi-manual forward-planned approach. Whilst this is a well-established process, for many patients this resulted in larger volumes being irradiated; reduced possibilities for optimisation of dose to targets and organs-at-risk; and limited options for heart sparing breath-hold treatment (DIBH) when supra-clavicular nodal fields were required.
An assessment of the quality of the MVision contours was made by the clinical oncologist(s) for these patients, using the categorisation scale in Table 1 below (Greenham et al, J Med Rad Sci 61 (2014) 151-158.
76% of lymph node contours were deemed acceptable ‘as is’ or with very minor differences (category 1 and 2). The worst assessment was category 5 (moderate edits) observed for only 5% of contours.
Given the positive feedback from the pilot study in terms of contour quality and potential time- saving, the department is now trialling a new AI improved workflow (see Figure 2, central column), with several potential benefits:
- more organ-at-risk contours are created (oesophagus, contra-lateral breast);
- true breast and nodal target volumes (PTV) can be created;
- time-saving in manual delineation (AI contours are reviewed by dosimetrists and clinicians);
- dose metrics available for all OAR and PTV (full compliance with national breast scorecards);
- does not require a separate field placement evaluation step by the clinician.
This AI improved workflow has been implemented after only a few months of introducing MVision software into the department. A detailed analysis will be carried out in several months’ time, in particular to monitor any modification requests by the clinician, when plans are optimised using the AI contours for field placement.
A fully AI optimised workflow is also proposed in the future (Figure 2, right column). It is anticipated that significant time-saving and reducing the patient pathway from CT scan to treatment could be possible, but the most important benefits to patient care will be improved heart and lung sparing, due to full inverse plan optimisation and/or the possibility of breath-hold treatments for all left- sided breast patients with nodes.
Breast and lymph node volume preliminary outcomes:
Generation of required lymph node and all OAR contours (contra-lateral breast and oesophagus) for all patients to ensure compliance with NHS Quality Improvement National Scorecard Breast Metrics.
Pilot study: 76% of lymph node contours were deemed acceptable ‘as is’ or very minor differences.
Time-saving for both clinicians and dosimetrists in manual delineation, and potentially in more efficient pathway in planning.
MVision offers excellent support for training and maintaining competencies for both senior and less- experienced dosimetry staff.
2. Head and Neck including lymph node volumes
An initial appraisal of lymph node contour quality and accuracy by clinical oncologists was very favourable, and MVision contours have now been fully incorporated into the clinical pathway, which includes target volume peer review prior to treatment planning. Our department peer review questionnaire (within the oncology information system) has been modified to include MVision evaluation. Prospective data collection is ongoing, and will allow analysis of oncologist peer reviewed lymph node quality and editing time in the future.
Initial evaluation of MVision for OAR volumes demonstrates some significant differences in manual and AI contours can occur, with 7% of OAR contours classed as ‘Gross error’ by the dosimetrists (using Table 1 to categorise volumes).
NCCC clinical practice currently uses structures labelled as ‘Larynx’ and ‘Lips’ which are used to drive the dose optimisation process. They are not defined as a strict anatomical structures and are not used for dose metrics. MVision models follow peer-reviewed consensus guidelines (e.g Brouwer 2015) for the anatomical definition of these structures. Discussion is ongoing at Newcastle for improvements to some of our local OAR delineation protocols in the future, which is hoped to resolve some of these issues.
Structures where there is a difference in the delineation protocol being used at NCCC compared to MVision (Larynx, Lips) were excluded from the analysis in Figure 3, which shows 54% of OAR contours were scored acceptable ‘as is’ or with very minor differences (category 1 and 2).
Head and Neck volume preliminary outcomes:
MVision lymph node contours now included in routine clinical pathway for peer review by clinicians.
Extensive streamlining of MVision models to reduce the number of contours is underway to avoid a large number of unwanted volumes.
Evaluation of current NCCC protocols underway for some OAR definitions to improve consistency and compliance with national and international guidelines, prompted by use of MVision.
3. Lung SABR volumes
MVision generated organ-at-risk contours for Lung SABR patients have been reviewed favourably by the dosimetrists and oncologists, with comparison to manual contours as per the UK SABR consortium guidelines (2019).
Several structures (Chestwall, Lungs, SpinalCord) generated by MVision have been assessed as improved compared to automated contours produced by the treatment planning system using non- AI atlas-based methods. The MVision contours for these structures are now used in our clinical practice.
Introduction of MVision now means that all Lung SABR patients have all OAR delineated as best practice. With manual contouring, this was deemed too labour intensive: for example, Liver and Brachial Plexus (both left and right) were delineated only when required due to close proximity to the target. Data collection and analysis will now be possible for all patients for a full range of OAR, as defined in the NHS Quality Improvement National Lung SABR Scorecards.
Some minor differences in volume definition for Bronchus and Trachea have been observed between MVision and current NCCC protocols. This may be related to updates to the volume definitions and nomenclature as per the Global Harmonisation Group (2020) paper, and discussion is ongoing with MVision to clarify.
4. Prostate and Nodes volumes
A review of MVision contours compared to manual contours for a series of retrospective prostate and node patients has been very positive, and the MVision male pelvis and lymph node model is shortly to be introduced as standard into the clinical pathway.
A significant challenge for these patients is the manual contouring of iliac vessels and bowel loops, which are labour intensive to delineate, but also critical to plan quality, as they abut or overlap with the pelvic lymph node target volume. Data collection is ongoing to benchmark manual contouring delineation time for dosimetry staff, which can be prone to large variation, due to staff experience and training.
It is anticipated that the delineation time between different staff members when using MVision contours (with minor edits as necessary) will be more consistent, with expected significant time saving and increased confidence for many staff members.
* Update 05 August 2022.
References
Brouwer et al CT-based delineation of organs at risk in the head and neck region: DAHANCA, EORTC, GORTEC, HKNPCSG, NCIC CTG, NCRI, NRG Oncology and TROG consensus guidelines. Radiother Oncol. 117, 1, 83-90 (2015) https://dx.doi.org/10.1016/j.radonc.2015.07.041
Greenham et al, Evaluation of atlas-based auto-segmentaion software in prostate cancer patients J Med Rad Sci 61 (3) 151-158 (2014) https://doi.org/10.1002/jmrs.64
Mir et al Organ at risk delineation for radiation therapy clinical trials: Global Harmonisation Group consensus guidelines Radiother Oncol. 150, 30-39 (2020) https://doi.org/10.1016/j.radonc.2020.05.038
The UK SABR Consortium. Stereotactic Ablative Body Radiation Therapy (SABR): A Resource. https://www.sabr.org.uk/wpcontent/uploads/2019/04/SABRconsortium-guidelines-2019- v6.1.0.pdf
