The Lean and Six Sigma quality improvement (QI) methodologies are useful, but according to the Journal of the American College of Radiology, more high-quality studies are needed for radiology departments to get the most benefit out of using them.

Thelina Amaratunga, MD, MSc, and Julian Dobranowski, MD, department of radiology at the University of Manitoba in Winnipeg, Canada, wanted to review studies assessing these methodologies to see if they were being used to their full potential. The authors found more than 250 studies from peer-reviewed journals that assessed the use of Lean, Six Sigma, or Lean Six Sigma, and chose 23 to study closely.

Amaratunga and Dobranowski noted that the 23 studies had “a diverse array of primary aims (with many having multiple primary aims),” but the most common themes were decreasing defects (8 studies), reducing appointment wait times (7 studies), and reducing patient and staff satisfaction or safety (5 studies).

The authors found that the studies did all demonstrate improvements, but there were still “high rates of systematic bias and imprecision.”

“Imprecision, otherwise known as random error, is based on confidence intervals (CIs),” the authors wrote. “Only one study provided a CI, for a single outcome, which proved to have a low degree of imprecision. The CIs for the remaining 96 percent of included studies were not provided or could not be determined, and thus imprecision was considered high.”

Overall, Amaratunga and Dobranowski concluded that the quality of evidence in these 23 studies was “low” and the completeness of reporting was “poor.”

But how can these studies be improved going forward? The authors shared several suggestions:

1. Train staff members in data collection and statistical methods

If providers are going to allow a study to take place, Amaratunga and Dobranowski explained, the staff should be properly trained or it is a bit of a wasted opportunity.

“Lean and Six Sigma methodologies require substantial staff training and data collection not only in the application of the methodology but also in statistical analysis,” the authors wrote. “If potential for large cost savings exists, then naturally there is an argument for investment in technology, training staff members, and using proper methodology to aid this process. However, without initial investment in these projects, sufficient high-quality evidence is not available to ascertain their true level of effectiveness, consequently preventing one from recommending their widespread implementation. Therefore, investing in high-quality studies is critical.”

2. Determine, and properly document, the study protocol

Amaratunga and Dobranowski found several ways these studies could have decreased their risk of showing bias, and this was one of the most prominent examples.

3. Measure numerous outcomes, not just the study's intended benefit

What if the study improved one metric, but hurt the radiology department in three other ways?

A diverse set of outcomes should be tracked,  and even negative outcomes should be reported.

“Three studies did not report outcomes that were listed as part of the primary aims of the studies, hinting at selective outcome reporting within these studies,” the authors wrote.

4. Collect data before, during, and after QI implementation

Amaratunga and Dobranowski said outcomes should be measured for at least one year before and after a study is complete “to reduce the risk for observation bias and ensure sustainability.”