This month’s May 2015 issue has two RCTs published on common clinical problems in orthopaedic trauma. I have offered my comments on the methodology and reporting of the Laflamme et al article (RCT of syndesmosis repair with cortical screw vs tightrope) using the GRADE approach to assess risk of bias.
I hope you find these comments useful and consider using some of these discussion points during your own local journal club review of this article.
Cochrane Risk of Bias
1. Sequence Generation: Computer generated randomization (low risk of bias)
2. Allocation concealment: Central allocation via online randomization tool (low risk of bias)
3. Blinding of participants, personnel, and outcome assessors: Blinding of participants and outcome assessors. It is possible that some patients may have discovered their treatment allocation during the 1-year follow-up period; however there is no reason to suspect differential rates of discovery between the groups. Similarly, the outcome assessors may also have learned the treatment allocation of patients during the follow-up period; however, the primary outcome (OM score) is patient-reported and should not be influenced by the outcome assessors (low risk of bias).
If the outcome assessors did discover the treatment allocation, it is possible this may introduce some bias for many of the secondary outcomes such as the objective component of the AOFAS scores and the range of motion measurements (uncertain risk of bias). Similarly, the radiographic evaluator was not blinded and an increased risk of bias is possible (high risk of bias); this comment is made recognizing it is very difficult to blind these types of outcomes.
Therefore, there is low risk of bias for the primary outcome; the secondary outcomes have uncertain risk of bias and high risk of bias depending on the specific outcome being evaluated.
4. Incomplete outcome data: Overall, there is excellent follow-up of the primary outcome at 1 year (>90%). There does not appear to be differential follow-up between the treatment groups (low risk of bias).
5. Selective outcome reporting: The full study protocol does not appear to be available as supplementary digital content; this is pretty standard for most orthopaedic trials. The trial registration was not reported in the manuscript, but can be found at clinicaltrials.gov (NCT01109303). The manuscript does not explicitly state at what follow-up visit the primary outcome will be analyzed for the primary analysis. I initially assumed 12 months after reading the paper; however, the trial registration states 3 months.
The results in the abstract highlight the outcomes that were statistically better or trended to significance with the tightrope; whereas, the manuscript discussion recognizes that the a priori 15-point clinical significant difference they sought to identify was not realized. Using the 3 month primary end-point, an 8 point difference was observed and the sample size was underpowered to reach statistical significance (but was close, p=0.067).
Other secondary analyses such as the proportion of patients reaching an OM score of >90 points were also performed in the results section but did not appear to be part of the original analysis plan—this also trended towards significance. The multiple testing of numerous secondary outcome measures without adjustment of the significance level also can be concerning. The remainder of the secondary outcomes were well-reported.
Overall, there were small differences favouring the tight rope across most outcomes, in which the clinical significance is uncertain. Most of these measured differences approached statistical significance; however, the authors conclude the tightrope leads to improved outcomes. I believe there is uncertain risk of bias in this section.
6. Other potential threats to validity: The last potential threat to validity is the funding disclosures. Arthrex has provided unrestricted funding to one author and institutional support to another author (uncertain risk of bias).
When performing an overall assessment of the risk of bias for this study, I would assess this study as “High” quality evidence for the primary outcome and “Moderate” quality evidence for the secondary outcomes based on the comments above. Although the methodology and reporting for the primary outcome are high quality, I believe the conclusions and interpretation of the data remain somewhat controversial. Further topics for discussion include the Minimum Clinically Important Difference (MCID), the Minimum Detectable Change (MDC), and multiple testing.
Congratulations to Dr. Leflamme and colleagues for a well-conducted clinical trial and furthering our understanding of dynamic versus static fixation of the syndesmosis! I hope this article is discussed among our readers for its clinical and methodology merits.
Levels of Evidence Section Editor