Systematic literature reviews: Expert searchers can help avoid errors
Systematic reviews are considered one of the highest levels of evidence in clinical practice, so how can it be that research teams are not making their methods reproducible and following the most stringent of standards when it comes to their search methods?
Errors in search strategies negatively affect the quality and validity of systematic reviews. For investigators, whether they are considering publishing a systematic review or consulting them for practice, it’s important to know how to evaluate the search methods. A recently published study, Errors in search strategies used in systematic reviews and their effects on information retrieval, explains the effects a search strategy has in producing a high quality review. The article, by Salvador-Oliván, J. A., Marco-Cuenca, G., and Arquero-Avilés, R., was published in the Journal of the Medical Library Association.
The study included 137 systematic reviews. 92.7 percent of the included systematic reviews contained at least one error in the search strategy. 78.1 percent of the included systematic reviews contained search statement errors that affected the number of results found in the database search step. These errors often involved important terms that were missing from the search and a failure to use Medical Subject Headings (MeSH), which also affected recall.
A table from the study describing common errors and showing the differences in search results with an erroneous search compared to an effective search is shown above. It shows that the smallest of errors can change search results by the thousands.
It cannot be stressed enough the importance of having an expert searcher perform the searches for systematic reviews so that the search strategy is:
- sensitive, so that information is not missed;
- specific, so that the investigator does not have to work through hundreds or even thousands of articles to find the ones that meet criteria; and
- systematic, meaning using a carefully defined strategy that can be repeated by others if necessary.
Too many published reviews state that they followed the published standard, “Preferred Reporting Items for Systemic Reviews and Meta-Analysis” (the PRISMA statement) or the guidelines set forth by the National Academy of Medicine, but yet they do not follow the standards.
The PRISMA Checklist offers 27 points to verify in a systematic literature review. In particular, the eighth item states “Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.”
To consult with a Harrell Health Sciences Library librarian on a systemic review, email Amy Knehans, MLIS, AHIP, at aknehans@pennstatehealth.psu.edu.
Read more
If you're having trouble accessing this content, or would like it in another format, please email the Penn State College of Medicine web department.