Electronic Thesis and Dissertation Repository

Thesis Format

Integrated Article

Degree

Master of Science

Program

Epidemiology and Biostatistics

Supervisor

Martin, Janet

Abstract

Research problem: Randomized controlled trials (RCTs) are essential; however, their validity can be threatened through distortion or spin. This study quantifies publication bias and distorted outcome reporting.

Methodology: All surgical RCTs registered on ClinicalTrials.gov from 1997-2017 were identified and a sample was obtained through random and intentional selection. Failure to publish (proportion of studies that remain unpublished), outcome distortion (changing intended outcomes), and spin (distorted presentation) were explored. Comparisons were made for positive versus negative studies and for high-income (HICs) versus low-middle income countries (LMICs).

Results: In total, 13,761 RCTs were registered (median enrollment size = 96, 94% from HICS). From a sample of 5,094 studies, 1,718 of them were published (34%). In total, 62% of published conclusions declared a significant difference (1,058/1,718), of which 41% (436/1,058) had “turned” positive due to spin or distortion.

Conclusion: While a large volume of RCTs have been registered, many remain unpublished. High proportions of spin and distortion raises concerns for validity of the evidence base.

Summary for Lay Audience

Randomized controlled trials (RCTs) are a necessary tool to support evidence-based decision making in clinical practice. To provide valid information, it is essential that RCTs publish all their results and report the outcomes without bias. In this study, we examine characteristics and trends of surgical RCTs including evidence of publication rates and outcome distortion. To do this, we retrieved all surgical RCTs registered on ClinicalTrials.gov between 1997 and 2017. We examined the research base describing the global spread of studies and volume of study registration across the years. After this, we took a sample of the studies to search for publication and compared the outcomes that were intended to be reported with the outcomes the publication reported. Differences between intended outcomes and reported outcomes are known as outcome distortion, and misrepresentation of results in the presentation of the outcomes is known as spin. These characteristics were compared between positive and negative studies, as well as for high-income countries (HIC) and low-middle income countries (LMIC). In total, we retrieved 13,761 RCTs that met the inclusion criteria. These studies had a median enrollment size of 96 that decreased across the years, and mainly pertained to HICs (94%). The number of studies registered each year was increasing; however, the proportion labelled “Completed” was decreasing. The sample of studies consisted of 5,094, where 1,718 were published (34%). Of published studies, conclusions with significant differences were declared in 62% of the studies (1,058/1,718), of which 41% (436/1,058) turned positive due to spin or distortions of outcomes. HIC had a higher failure to publish rate than LMIC (63% vs. 49% unpublished), but overall had more studies published, and increasing proportion of positive studies published. The high proportion of studies in HIC is a concern for global generalization and should be more reflective of where majority of the world’s population lies. Additional concerns for research integrity and validity of the evidence base lie within the large proportion of unpublished studies (66%) despite the large volume of registered studies. In order to ensure adequate decision making, further efforts to ensure publications are conducted without biases are necessary.

Creative Commons License

Creative Commons Attribution-Noncommercial 4.0 License
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 License.

Download

Share

COinS