University Hospitals Plays Key Role in Study of AI Technology to Reduce Missed Fractures on X-rays

Study done here shows that AI tool improves diagnostic accuracy and reduces X-ray reading time, suggesting path for patients to receive quicker, more secure results

UH Clinical Update | June 2022

A study from University Hospitals played a pivotal role in demonstrating efficacy, which was reviewed as part of the recent FDA approval of artificial intelligence (AI) technology aimed at reducing the problem of missed fractures on X-rays. The “deep learning” tool, developed by the French company AZMed, uses artificial intelligence algorithms to identify and outline fractures in extremity radiographs. In addition, the technology helps with prioritizing patients for further care, given that using artificial intelligence as a preliminary identification tool can help flag issues much more quickly. UH is the only healthcare system in the U.S. to collaborate with AZMed in testing its new technology.

“The deep learning tool demonstrated high stand-alone accuracy, aided diagnostic accuracy, and decreased interpretation time,” said Navid Faraji, MD, musculoskeletal radiologist at UH and clinical lead of the study and one of 27 UH radiologists and emergency medicine physicians involved with the study of the AI technology. “This tool will further optimize the time to read and the time to diagnose patients who have fractures. It can also help bring the x-ray to the attention of the reading radiologist faster.”

To help validate the AZMed software, three UH board-certified musculoskeletal radiologists, including Dr. Faraji, read and annotated fractures on 2,626 normal and abnormal x-rays of UH patients’ shoulders, arms and legs taken at four UH locations, including at least 140 exams per body region. These evaluations served as the “ground truth” against which the AZMed tool would be judged, says abdominal radiologist Leonardo Kayat Bittencourt, MD, PhD, Vice-Chair of Innovation of the Department of Radiology, and Director of the UH Radiology AI Collaborative. Then, three different groups of UH physicians were asked to identify fractures in 186 randomly selected cases, with and without the aid of the AI tool. The readers included eight ED physicians, eight non-musculoskeletal radiologists and eight musculoskeletal radiologists. The X-ray reading sessions were separated by a washout period of at least one month, so that readers would have “fresh eyes” and not rely on remembering previously viewed cases. The research team then compared differences in fracture detection in features such as accuracy, sensitivity, specificity and interpretation time with and without the AI tool.

Results showed that the across all physicians in the study, review using the AI tool increased accuracy by 5.6%. In addition, review and interpretation overall time decreased 27%. Not surprisingly, an even higher benefit of the technology was noted in non-radiologist emergency physicians and non-musculoskeletal radiologists.

Sensitivity – the ability of the AI tool to assist in the identification of true fractures and avoid false negatives -- was improved from 0.865 to 0.955 with the aid of the AI tool. And specificity – the ability of the AI tool to assist in the identification of healthy bone and avoid false positives – also improved, from 0.826 without the AI tool to 0.831 with it.

“The deep learning tool demonstrated high accuracy, and decreased interpretation time, with a higher benefit noted in ED physicians and non-musculoskeletal radiologists,” Dr. Kayat Bittencourt says. “In addition, there was no loss in specificity.”

Dr. Faraji further adds that one especially helpful feature of this AI software going forward will be its ability to flag x-rays that indicate a fracture and send them to the “top of the list” for viewing by reading radiologists using UH systems.

“That will further optimize the time to read and the time to diagnose those patients who have fractures and create better communication with the ED team,” he says. “It’s not just the time to read the x-ray that’s important. It’s that it can help bring the x-ray to the attention of the reading radiologist faster. The whole window of time from the acquisition of the exam to the delivery of the report is what’s key. What takes more time is the point from when the x-ray is acquired to when it is finally read. If we can expedite that process where the radiologist can start looking at the scan, the amount of time you gain could be even bigger.”

Beyond improving the timeliness of x-ray results for patients, this AI tool also adds an additional layer of quality of care, Dr. Kayat Bittencourt says, enhancing but not replacing the role of the clinician.

“UH is committed to bringing top-notch technology to clinical practice,” he says. “We don’t replace the role and centrality of the radiologist in healthcare delivery, but rather enhance the capabilities of the provider.”