Methodological and Logistic Limitations of the Systematic Review

This review had methodological and logistic limitations. Some of the important limitations are listed here.

In many cases, the methodological and/or reporting quality of the primary studies was poor, as judged by the Jadad scale and the Schulz allocation concealment component. For example, the adequacy of methods used for randomization, treatment allocation concealment, or blinding could not be ascertained for majority of the reviewed studies. In turn, the absence of this information compromised the valid interpretation of the study results.

There was substantial heterogeneity with respect to efficacy/harms outcomes, types of interventions, diverse concurrent clinical conditions, and reporting quality across the reviewed studies. Clinical and/or methodological heterogeneity limited the extent of statistical pooling of the efficacy- and harms-related data.

In crossover trials, pre-crossover quantitative data was usually not reported making it difficult to incorporate the results into the meta-analyses.

Primary studies did not always provide sufficient quantitative information (e.g. standard deviations, mean endpoint estimates of treatment efficacy, statistical test results) needed to pool the individual trial results and to judge the treatment-related between-group differences in the outcomes. Due to limited resources and the timelines of this review, the authors of individual studies could not be contacted for additional information that was not provided in the reports.

Empirical evidence has shown that harms occurring during a trial are generally underreported. Overall, the occurrence and details of adverse events was poorly reported in the primary studies. Many trial reports did not provide the data on the incidence of any all-cause adverse events and serious adverse events. Moreover, the types of adverse events across the trials, as well as the definition of adverse events and in particular serious adverse events were not reported consistently from study to study. The authors often did not provide statistical test results for the between-group differences in adverse events. Thus, the reviewers resorted solely to qualitative judgment.

This review did not investigate the effectiveness and harms of interventions such as natural health products, vacuum constriction devices, penile prosthesis implants, surgery, or lifestyle modifications. This review also did not include trials in ED patients with spinal cord injury.

The interpretation of the study results was complicated by the lack of well accepted guideline(s) regarding the magnitude of clinically important (or meaningful) difference for a given validated outcome. It is well recognized that the interpretation based solely on the statistical test results may be misleading. The clinically important difference for a valid and relevant outcome may or may not be statistically significant and the opposite also holds true. In many cases, study authors did not report whether the study power to detect a pre-specified minimally relevant clinical difference was estimated.

The evidence needed to evaluate the utility of routine endocrinological testing administered to the ED population is sparse and inconclusive.

Go to: