European Heart Journal Advance Access originally published online on November 15, 2006
European Heart Journal 2006 27(23):2744-2745; doi:10.1093/eurheartj/ehl372
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
The meta-analysis: supportive or illuminating?
1 Main Line Health Heart Center, Wynnewood, PA 19096, USA
2 Jefferson Medical College, Philadelphia, PA 19107, USA
* Corresponding author: Suite 558, Medical Office Building East, 100 Lancaster Avenue, Wynnewood, PA 19096, USA. Tel: +1 610 645 2682; fax: +1 610 896 0643. E-mail address: koweypr{at}mlhheart.org
This editorial refers to ‘Interventions for prevention of post-operative atrial fibrillation and its complications after cardiac surgery: a meta-analysis’
by D.C. Burgess et al., on page 2846
Many of us use statistical methods like an unsteady person might use a lamppost—more for support (of our pre-conceived notions) than for illumination. After all, Yogi Bera once said, ‘If I didn't believe it, I wouldn't have seen it.’ So it is easy to scoff at some of the flimsy constructs our colleagues have brought forward to ‘illuminate’ important issues that have not been answered by definitive trial data.
In no case has the derision reached as high a level as with the meta-analysis, an attempt to compile data from small trials to answer important clinical questions. Though there may be good scientific rationale for such an idea all of us enjoy pointing to numerous examples in which poorly executed meta-analyses led to conclusions that were swept away later by rigorous clinical trial results.
The truth is that the meta-analysis, like any statistical method, is susceptible to error when performed incorrectly.1,2 And one of the most common mistakes is study selection. Inclusion of poor studies and exclusion of valuable ones can dramatically affect the quality of the analysis to the point of reaching conclusions that are exactly contrary to the truth. A second potential weakness of this methodology is that, even with the compilation of well-executed trials, numbers may still be insufficient to reach satisfactory overall conclusions. The reliability of an observation is improved with a large number of events. If events occur only at a low rate, or if the difference between groups is small, the outcome described may be attributable to the play of chance. Small meta-analyses are susceptible (perhaps even more susceptible) to this kind of statistical error.
Enter the meta-analysis of Burgess et al.3 in which they seek to describe the relative value of interventions for the prevention of atrial fibrillation (AF) that regularly occurs after cardiac surgery. The authors complied articles on the basis of relatively strict criteria and applied standard methodology to extract their conclusions. Were their findings unexpected or surprising? We think not. The most commonly tested interventions, beta-blockers, sotalol, amiodarone, and pacing were effective in preventing AF. Magnesium also had an effect but was confounded by concomitant beta-blocker use. Amiodarone and pacing decreased the length of stay, and only alone reduced strokes.
So why is this paper valuable? The answer is simple. Because of the size of the study and the scrupulous culling of the literature to exclude study selection bias, as well as the excellence of the analyses, the results can be considered reliable. The fact that the results of the study are consistent directionally with the best studies in this area provides added assurance.
However, lest the reader believe we have reached the end of the rainbow on this subject, these editorialists would like to provide a few important caveats about meta-analyses in general and the present study in particular. First, meta-analyses should never be viewed as anything but hypothesis generating. They are susceptible to many confounders including publication bias that cannot be completely accounted for with any statistical methodology. For these reasons, the thoughtful reader might consider taking away a zero from the P-value generated in a meta-analysis and/or to use a 99% confidence interval for ratios of relative risk. As an example, in the present study, the finding of stroke reduction with amiodarone would be lost, an appropriate adjustment we think because this is the weakest part of the study results.
Secondly, statistical significance can be a far cry from what a clinician might find compelling. Most of our colleagues embrace the concept of beta-blocker prophylaxis but few use sotalol. Amiodarone's uptake has been spotty, and virtually no one uses prophylactic pacing. Why? The clear perception of most clinicians is that post-operative AF in most patients is simply not worth the risk associated with these interventions. Even if one were to decide to treat prophylactically, this meta-analysis illustrates how much heterogeneity exists for important issues such as dose, duration of therapy, and definition of success.
Thirdly, the length of stay issue is complex. We have learnt that there are many reasons why patients remain in the hospital after surgery, and that trying to impact this parameter simply by preventing AF may be a false hope. We suspect that re-educating surgeons and cardiologists to the concept that AF can be managed conservatively would be helpful. To convince all constituencies of this might require an AFFIRM-like study in post-operative patients, an idea that needs to be developed and pursued.
Fourthly, stroke is the overwhelming morbidity associated with AF. The amortized risk in post-operative patients is substantial and cannot be ignored.4 Clinicians should not rely on measures that prophylax against AF to protect patients from this devastating complication. Despite the challenges of the post-operative state, patients need to receive anticoagulants until their period of risk of atrial arrhythmia has passed.5
Finally, the efficacy of all of the proposed interventions needs to be examined in the context of risk and cost, neither of which are adequately addressed in this paper. Without knowing that a treatment reduces morbid or mortal events, it is impossible to justify whatever risk that intervention brings. Thus, a profligate policy of amiodarone prophylaxis seems unreasonable and unsupportable to us, despite what this meta-analysis tells us. The study cannot tell us if there is justification for using more aggressive measures in patients at highest risk for AF post-surgery, such as the elderly with valvular disease.
We consulted Webster's Ninth Collegiate Dictionary and discovered that the third definition of the prefix meta- was ‘more comprehensive: transcending.’ In this sense, Burgess et al. have helped us to ‘transcend’ what we know by applying sound statistical principles to an otherwise bewildering literature to provide insight—and for that they are to be congratulated. But we should not be prepared to make major changes in our therapeutic approach to post-operative patients until we can be convinced that any method of preserving sinus rhythm on a wholesale basis, or even in a significant subset, is worthwhile in terms of the kinds of outcomes we really care about like heart failure, stroke, and death. Burgess et al. have helped us frame the question and now it is our job to think about ways to answer it.
Acknowledgement
Supported in part by the Rose and Adolph Levis Foundation, Haverford, PA, USA.
Conflict of interest: none declared.
Footnotes
The opinions expressed in this article are not necessarily those of the Editors of the European Heart Journal or of the European Society of Cardiology.
doi:10.1093/eurheartj/ehl272 
References
- Higgins MS and Stiff JL. (1993) Pitfalls in performing meta-analysis: I. Anesthesiology 79:405.[CrossRef][ISI][Medline]
- Jacobson RM. (1999) Promises and pitfalls of meta-analysis in vaccine research. Vaccine 17:1628–1634.[CrossRef][ISI][Medline]
- Burgess DC, Kilborn MJ, Keech AC. (2006) Interventions for prevention of post-operative atrial fibrillation and its complications after cardiac surgery: a meta-analysis. Eur Heart J 27:2846–2857 First published on October 2, 2006, doi:10.1093/eurheartj/ehl272.
[Abstract/Free Full Text] - Kangasniemi OP, Luukkonen J, Biancari F, Leo E, Vuorisalo S, Pokela R, Juvonen T. (2006) Risk-scoring methods for prediction of postoperative stroke after coronary artery bypass surgery. J Thorac Cardiovasc Surg 131:734–735.
[Free Full Text] - Epstein AE, Alexander JC, Gutterman DD, Maisel W, Wharton JM. (2005) Anticoagulation: American College of Chest Physicians guidelines for the prevention and management of postoperative atrial fibrillation after cardiac surgery. Chest 128:24S–27S.
CiteULike 
Connotea 
Del.icio.us What's this?
Related articles in EHJ:
- Interventions for prevention of post-operative atrial fibrillation and its complications after cardiac surgery: a meta-analysis
- David C. Burgess, Michael J. Kilborn, and Anthony C. Keech
EHJ 2006 27: 2846-2857.[Abstract] [Full Text]
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||