Elusive Relevant Papers

Machine learning (ML) techniques are increasingly being used to assist with study selection in systematic reviews, addressing the substantial time and resource demands of screening large volumes of literature. Despite growing interest, an up-to-date overview of the current landscape of ML methods for study selection was lacking.

Master鈥檚 student Fionn Byrne, supervised by the DISC-AI Lab, conducted a scoping review to systematically map the literature on ML-based study selection. The review aimed to provide an overview of available methods, how they are applied, and how their performance is reported.

Progress

The scoping review followed the PRISMA-ScR guidelines. A comprehensive search was conducted in Scopus and Web of Science, resulting in 194 included publications after screening. The included studies were analyzed based on publication characteristics, machine learning methods used, application settings, and performance evaluation practices.

The review found a steady increase in publications since 2016, with active learning being the most commonly applied approach. Most studies focused on binary classification tasks, assisting reviewers by prioritizing or filtering studies during screening. However, the review also noted a lack of standardization in how ML methods are described and evaluated, with considerable variation in reporting practices.

A key recommendation from the review is the development of more transparent and standardized reporting guidelines to improve reproducibility and comparability across studies. The results provide a valuable foundation for future research and practical implementation of machine learning in systematic review workflows.

The project was published as a peer-reviewed article in Systematic Reviews.

Funding

This project and its resulting output are being funded by the Dutch Research Council (NWO), project number 406.22.GO.048.

People involved

More info