Impact of Librarians on Reporting of the Literature Searching Component of Pediatric Systematic Reviews
Document Type
Article
Publication Date
10-2016
Volume
104
Issue
4
Journal
Journal of the Medical Library Association
First Page
Systematic reviews generally form the highest level of evidence for most clinical question types, it is vital to critically appraise their methods and findings. Since Oxman et al. (1) originally presented their framework for critical appraisal of systematic reviews. Tools to help with the evaluation of systematic reviews have evolved over time such as Quality of Reporting of Meta-analyses (QUOROM), its successor Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Meta-analysis Of Observational Studies in Epidemiology (MOOSE). A recent study has made note of some of the key tools in this area (2). The goals of these tools were not just to facilitate the critical appraisal of existing reviews, but also designed to assist with the reporting of systematic reviews. These tools could also provide useful guidelines for the design of new reviews. A more contemporary examination of critical appraisal speaks to the need for judgement in two critical areas: a) credibility - which addresses the rigour of the review methods and b) confidence in estimates - which directly relates to the quality of evidence used in the review (3). By meeting both of these judgement areas, systematic reviews can then be viewed as sources of high quality authoritative information based on the integration of the results of high quality studies. While recognizing the importance of these critical areas, we chose to focus on the credibility component. In order for a systematic review to fulfill the credibility component it should look to define an answerable clinical question; include a comprehensive and reproducible literature search process; provide clear study selection and assessment criteria and present the results in meaningful manner (2). Moher et al. demonstrated great variability in the transparency of reporting which would ultimately impact the overall quality of reviews (4). There is ample literature assessing the quality of systematic reviews across many disciplines (5-20) and a common theme that has emerged from a number of these studies has been the need for improving the conduct and reporting of systematic reviews. Despite the acknowledgement that the problem exists and the availability of standards and guidelines, the situation seems to be ongoing. McDonald et al. (21) studied the methodological quality of systematic reviews in the urology literature over a 10 year span and found no evidence that the quality had improved over time. On the other hand Dijkman et al. (22) studied meta-analyses in orthopaedic surgery over a 20 year time span, and did report a significant improvement
Last Page
45. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007; 7:[10 p.]. 46. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264-9. 47. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet. 1999;354(9193):1896-900. 48. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000;283(15):2008-12. 49. Abdi H. Bonferroni Test. In: Salkind NJ, editor. Encyclopedia of Measurement and Statistics. 1. Thousand Oaks, CA.: SAGE Publications; 2007. p. 103-7. 50. Wang JL, Sun TT, Lin YW, Lu R, Fang JY. Methodological reporting of randomized controlled trials in major hepato-gastroenterology journals in 2008 and 1998: A comparative study. BMC Med Res Methodol. 2011;11. 51. The Equator Network. The EQUATOR Network | Enhancing the QUAlity and Transparency Of Health Research [cited 2015 Web Page]. Available from: http://www.equator-network.org/. 52. Green S, Higgins JPT, Alderson P, Clarke M, Mulrow CD, AD O. Introduction Section 1.2.2 What is a systematic review? . In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 510 [updated March 2011] edition The Cochrane Collaboration; 2011. 53. Faggion CM, Jr., Atieh MA, Park S. Search strategies in systematic reviews in periodontology and implant dentistry. J Clin Periodontol. 2013;40(9):883-8. 54. Gerber S, Tallon D, Trelle S, Schneider M, Juni P, Egger M. Bibliographic study showed improving methodology of meta-analyses published in leading journals 1993-2002. J Clin Epidemiol. 2007;60(8):773-80. 55. McGowan J, Sampson M. Systematic reviews need systematic searchers. JMLA. 2005;93(1):74-80. 56. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. Ann Intern Med. 2009; 339:[b2700-b pp.].
Abstract
Abstract
Objective: The goal of this study was to compare the reporting rigour of the literature searching component of systematic reviews with, and without, the help of a librarian.
Methods: Systematic reviews published from 2002 to 2011 in the 20 highest Impact Factor paediatrics journals were collected from MEDLINE. Corresponding authors were contacted via an email survey to determine if a librarian was involved, the role played, and functions performed. The reviews were scored using a 15 item checklist by two reviewers independently. The overall reporting for each role category was expressed as the mean value of the total score. Analysis of variance (ANOVA) and Pairwise comparisons were used for multiple comparisons. The comparison of frequency of reported methodological elements between different role categories was accomplished using Pearson Chi-square analysis, and expressed as the risk ratio (RR) with 95% confidential interval (CI).
Results: 186 reviews met the inclusion criteria. 44% of the authors indicated the involvement of a librarian in conducting the systematic review. In the presence of a librarian as the co-author/team member, the mean score was 8.40, (95% CI 7.69-9.11) compared to 6.61, (95% CI 6.26-6.95, p <0.001) for reviews without a librarian. The presence of the librarian as a co-author/team member was correlated with higher scores for reporting in various components of the methodology, in comparison to studies without a librarian, particularly the flow diagram of study inclusion process (RR=1.76, 95% CI 1.36-2.30), the date search was updated (RR= 2.29, 95% CI 1.32-3.98), the full search strategy (RR=3.55, 95% CI 1.81-6.97), the use of subject-specific and regional bibliographic databases (RR=1.94, 95% CI 1.57-2.40), and searching grey literature (RR=1.72, 95% CI 1.17-2.53).
Conclusions: Our results show that having a librarian or information professional as a team member or co-author contributes to more rigorous reporting of the literature searching component of systematic reviews.
Notes