Electronic Thesis and Dissertation Repository

Thesis Format

Integrated Article

Degree

Master of Science

Program

Health and Rehabilitation Sciences

Collaborative Specialization

Musculoskeletal Health Research

Supervisor

Dianne Bryant

Abstract

In research, appropriate statistical interpretation and methodology are essential to conduct quality work. To interpret results, p-values are frequently used in isolation, but this is insufficient as treatment effects, confidence intervals (CIs), and clinically important thresholds should also be reported. Further, the equality, superiority, non-inferiority, and equivalence frameworks have critical differences not well delineated in current literature. We conducted a systematic review of studies published in high-impact orthopaedic journals and examined a) how well studies interpreted the results of patient-reported outcome measures, and b) whether a consistent framework was used throughout studies. We found that the majority of studies do not report CIs around between-group differences and do not define a clinically meaningful difference. Half of studies reporting sample size calculations had inconsistency between framing of their research question, sample size calculation, and conclusion. Authors should report results with clinical context and maintain framework consistency to prevent misleading treatment recommendations.

Summary for Lay Audience

A general understanding of important concepts such as basic statistics and methods are needed to conduct research, however, published research may still contain misinterpreted results. For example, authors rely on the widely used p-value statistic to measure the difference between groups. However, p-values only tell us that two treatment differ but not how large that difference is. The size of the difference is best communicated through providing the treatment effect, confidence intervals (CIs), and a threshold of clinical importance. Clinical importance indicates whether the effect of a treatment is meaningful from a clinician’s perspective. Researchers are also interested in knowing whether their findings are applicable to patients, which requires the use of correct study methods, particularly the right framework (i.e. equality, superiority, non-inferiority, and equivalence).

The purpose of our study was to review studies published in top journals in the field of orthopaedic surgery and evaluate whether studies correctly reported their results and whether authors followed a consistent framework throughout their study. We looked at studies published in 2017 and 2019 in five journals that compared two different treatments and assessed patient-reported outcome measures, which are tools used to gain the patient’s perspective. We found that the majority of studies relied on a p-value statistic, and only approximately one in five studies reported treatment effect with CIs. We also found that 52.2% of studies switched the framework throughout their study, which meant that many relied on a p-value statistic to make treatment recommendations and sample size calculations were not appropriately applied. Overall, when statistics are misinterpreted and the inappropriate methodology is applied, the study findings can lead clinicians into making misleading treatment recommendations to patients. We encourage journal editors and authors to work on ensuring that the results of their research are interpreted with clinical relevance and the correct framework is used. We believe that this will improve the quality of orthopaedic literature moving forward.

Download

Share

COinS