Observer agreement for small bowel ultrasound in Crohn’s disease: results from the METRIC trial1
Authors: Gauraang Bhatnagar, Laura Quinn, Antony Higginson, Andrew Plumb, Steve Halligan, Damian Tolan, Roger Lapham, Susan Mallett, Stuart A. Taylor & METRIC study investigators
Journal Abdominal Radiology 45:3036-3045. 2020 Feb 10. DOI: https://doi.org/10.1007/s00261-020-02405-w
The METRIC trial2 was a prospective multicentre trial which recently demonstrated the high sensitivity of small bowel ultrasound (SBUS) for both the presence and extent of small bowel Crohn’s disease. Small bowel ultrasound has advantages over magnetic resonance enterography (MRE); it is widely available, cheaper, does not require IV contrast and is generally preferred by patients3. The perceived drawback of SBUS is the high operator dependence to both identify subtle disease but also to localise disease. Whilst Parente et al.4 found excellent inter-observer agreement when localising affected segments of bowel in Crohn’s disease, there is a paucity of research assessing inter-observer variability of active disease on SBUS which is particularly important to guide management.
This study by Taylor et al. set out to assess inter-observer variability for the presence, extent and descriptive features of small bowel and colonic Crohn’s disease.
A proportion of patients previously recruited in the METRIC trial underwent a repeat ultrasound by a different practitioner who was blinded to the findings from the previous SBUS as well as other imaging, and all endoscopic and clinical data other than under which cohort the patient was recruited. There were two cohorts, patients with a new diagnosis of Crohn’s disease and patients with established disease and clinically suspected of luminal relapse. The second SBUS was performed on the same day as the first SBUS by one of 6 practitioners (5 radiologists and 1 sonographer trained in SBUS) and assessed disease presence, activity and location with more detailed mural and extra-mural observations for each segment based on a case report form which included the practitioner’s confidence of these findings. Findings in the main METRIC trial protocol such as wall thickening, focal hyperechoic mesentery (with or without fat wrap), isolated thickened submucosa, increased doppler vascular pattern, ulceration or abscess were assessed. Statistical analysis identified the percentage agreement between practitioners as well as prevalence adjusted bias adjusted kappa (PABAK) to assess inter-observer variability.
There was excellent sonographic agreement for small bowel disease presence in both cohorts with an 82% agreement in patients within the new diagnosis cohort and 81% agreement in the relapsed disease cohort, both with substantial agreement upon assessment of kappa coefficient. There was a lower percentage agreement for colonic disease presence in new diagnosis (64%) with fair agreement (κ = 0.27) although there was 78% agreement in the relapsing cohort with moderate agreement (κ = 0.56). The study also replicated the high levels of sensitivity of SBUS for small bowel disease presence as in the METRIC trial2.
There were cases where practitioners agreed with one another, but they were both incorrect when compared to MRE which was likely due to the limitations of SBUS. These include difficulty identifying disease due to patient factors, subtle findings or abnormal disease site. SBUS had limitations when assessing small bowel disease extent in both cohorts (64% agreement in new diagnosed patients and 56% in suspected relapse patients). There were also much lower kappa values suggestive of slight (relapse cohort) and fair (new diagnosis cohort) agreement in segmental localisation which differs to the aforementioned findings by Parente et al.4
The radiologists participating in this study all had a declared subspecialty interest in gastrointestinal radiology, and the sonographer had undergone formal training and performed SBUS routinely, so these results may not be applicable to general radiologists in a non-specialist setting.
The study demonstrates that there is substantial agreement between practitioners using SBUS when assessing for the presence of small bowel Crohn’s disease, although assessing the extent of disease had lower inter-observer agreement in keeping with the known difficulties of assessing small bowel with ultrasound. In clinical practice this study supports the use of using a combination of MRE for disease identification and SBUS to assess for ongoing disease presence.
References
- Bhatnagar G, Quinn L, Higginson A, Plumb A, Halligan S, Tolan D, et al. Observer agreement for small bowel ultrasound in Crohn’s disease: results from the METRIC trial. Abdom Radiol. 2020
- Taylor SA, Mallett S, Bhatnagar G, Baldwin-Cleland R, Bloom S, Gupta A, et al. Diagnostic accuracy of magnetic resonance enterography and small bowel ultrasound for the extent and activity of newly diagnosed and relapsed Crohn’s disease (METRIC): a multicentre trial. Lancet Gastroenterol Hepatol. 2018
- Miles A, Bhatnagar G, Halligan S, et al. Magnetic resonance enterography, small bowel ultrasound and colonoscopy to diagnose and stage Crohn’s disease: patient acceptability and perceived burden. Eur Radiol 2019;29:1083-1093
- Parente F, Greco S, Molteni M, Anderloni A, Sampietro GM, Danelli PG, et al. Oral contrast enhanced bowel ultrasonography in the assessment of small intestine Crohn’s disease. A prospective comparison with conventional ultrasound, x ray studies, and ileocolonoscopy. Gut. 2004
Rahul Munyal (Radiology ST5 Registrar, Queen’s Medical Centre, Nottingham University Hospitals NHS Trust, Nottingham, England)
Dr. Rahul Munyal is a fifth and final year radiology resident at Nottingham University Hospitals NHS Trust (UK) and an ESGAR member. He is a fellow of the RCR (Royal College of Radiologists) and is undertaking subspecialty training in Gastrointestinal and Hepatobiliary imaging with an interest in non-vascular intervention.
Comments may be sent to Rahul.munyal@nhs.net