We were beginning to notice something. Study authors would occasionally refer to their report of a randomised trial not by study design (e.g. ‘this is an RCT’) but by using the language of a study phase (e.g. ‘this is a phase 3 trial’). We wondered if established search filters, which focus on words for randomisation and study design language, were sufficiently sensitive to identify studies which only report trials using the language of trial phase.

Methods

In 2015, we developed a pragmatic search filter called the P3 filter to identify studies reporting by trial phase and compared this to five established search filters in 2018, namely:

1. Cochrane’s Highly Sensitive Search Strategy (HSSS)(sensitivity- and precision-maximizing version);

2. the Royle and Waugh Brief RCT Search Strategy (BRSS);

3. the SIGN RCT filter;

4. the CADTH RCT filter (pre April 2018 version); and

5. the CADTH Controlled Clinical Trial (CCT/RCT) filter April 2018.

Findings

The P3 filter identified seven marker studies which reported eligible trials or reports of trials using language for trial phase but not the process or randomisation or study design. Filters 1-4 missed eligible studies: in particular a primary report of a randomised trial.

This suggests that the P3 filter does improve the sensitivity of searches when combined with established search filters. Our finding indicates that researchers should incorporate the P3 filter when searching for studies reporting trials. We found that incorporating the P3 filter to established search filters did not greatly increase the number of studies identified overall.

Only filter 5 identified all the articles missed by filters 1-4. The CADTH CCT/RCT filter was actually based on the P3 filter which explains the finding.

Study: Established search filters may miss studies when identifying randomized controlled trials (https://doi.org/10.1016/j.jclinepi.2019.04.002)

See also: further analysis to test the P3 filter