European Heart Journal Advance Access originally published online on May 23, 2006
European Heart Journal 2006 27(12):1394-1395; doi:10.1093/eurheartj/ehl047
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
The demystification of magnetic resonance imaging?
Medical Clinic I, University Aachen, Pauwelsstrasse, 52074 Aachen, Germany
* Corresponding author. Tel: +49 241 8088468; fax: +49 241 8082303. E-mail address: rhoffmann{at}ukaachen.de
This editorial refers to ‘Determination of interobserver variability for identifying inducible left ventricular wall motion abnormalities during dobutamine stress magnetic resonance imaging’
by I. Paetsch, on page 1459
The interobserver variability in test interpretation is a key parameter to evaluate the diagnostic reliability of a diagnostic examination. It has been evaluated for most clinically relevant cardiac imaging modalities. In many instances the results have been rather sobering. A considerable variability in the visual assessment of coronary artery stenosis severity has been shown between different readers of coronary angiograms.1 A high reader disagreement has been shown for interpretation of dobutamine stress echocardiograms.2 The high variability in the interpretation of dobutamine stress echocardiograms has become a matter of debate and has been used as an argument to doubt the diagnostic competence of the method. Low image quality, a large variability in test performance, and insufficient standardization in image interpretation without any objective parameter to define the test result has been referred to as the major limitations of stress echocardiography. Lower observer variability due to more standardized image acquisition, better image quality, and objective parameters of test interpretation has been shown for perfusion scintigraphy and has been used as an argument for its favourable diagnostic performance.3 Regarding stress echocardiography, multiple shortcomings could be reduced over time. Image quality could be improved. Along with digital image acquisition and more standardized interpretation criteria refinements in the stress echocardiographic techniques have resulted in higher interobserver agreement.4 Still, interobserver agreement remained far from optimal.
Dobutamine stress cardiovascular magnetic resonance imaging (DSMR) incorporates multiple advantages when compared with stress echocardiography. It is performed using standardized acquisition protocols. The acquired imaging views as well as quality of images is less dependent on the individual sonographers performance and the image quality is excellent in most cases. Thus, interobserver agreement is expected to be high considering the most obvious limitations of dobutamine stress echocardiography. However, data on interobserver agreement in the interpretation of DSMR were sparse until recently. In this issue of the Journal, Paesch et al. report on the interobserver variability in the interpretation of DSMR evaluated in a multicenter setting.5 The study confirms an excellent image quality in the majority of imaging views with the use of DSMR as reported earlier by the investigators as well as other authors.6 The study confirms also a high diagnostic accuracy for detection of coronary artery disease for all four readers using this technique. However, while the accuracy was almost similar among the four readers, sensitivity for detection of CAD ranged from as low as 71% to as high as 86%, and the specificity had a discordant trend indicating already that readers vary in their interpretation of image data. The surprise of this study is the relatively low interobserver agreement between the four expert observers from different institutions. The disagreement between the four readers was equally distributed and not the effect of one reader with significant deviations from the other readers.
How do we explain this result? Excellent images were thought to ensure high diagnostic accuracy but also to facilitate consistent interpretation because of a high confidence in the perception and interpretation of images. Impaired echocardiographic image quality has been a major cause of lower diagnostic accuracy when comparing dobutamine stress echocardiography with dobutamine stress MRI.6 Poor imaging conditions with resulting insecurity on the interpretation of images have also been shown to be a major cause of disagreement when analysing stress echocardiograms.2 The rather disappointing level of interobserver agreement confirms other recently published studies which have also indicated that use of MRI is no warranty to reach nearly perfect agreement between different readers on analysis of regional left ventricular function.7,8 The interobserver agreement was on a level previously reported for dobutamine echocardiography applied with most advanced echocardiographic techniques.4 Given these results it appears that reader agreement may be method and image-quality dependent on a low level of agreement. However, once a relatively high image quality is reached this aspect looses importance and other aspects come into play. Thus, we do not reach almost perfect reader agreement based on excellent image quality alone.
The difficulty of divergent test results relates to a different issue, involving in particular, the final pathway of image interpretation, which remains a subjective process based on the individual perception. Analysis of regional systolic function is a highly complex process. It relates to the analysis of dynamic images considering changes in wall thickness as well as endocardial inward motion and synchronicity of analysed left ventricular segments. There are obviously different thresholds in the subjective evaluation of regional wall motion abnormalities. Although there are doubtful, conservative readers who have hard times in deciding about an abnormality, other more liberal readers are quicker in their decision, which is rather driven by an intention to detect abnormalities. What are the reasons for a different perception of visual data? Current research in the artificial intelligence and computer science area attempts to understand the complex human perception and response to visual data.9 Beyond the initial perception of images, the cognitive analysis of specific patterns of image configurations comes into play. Furthermore, a definite level of expectation towards the test result has an impact on test interpretation. Finally, the level of confidence in the findings can vary. A profound knowledge of these processes is the key to algorithms, which might eventually support or even reliably replace the human subjective visual interpretation.
DSMR might, in particular, encounter a difficulty related to induction of new wall motion abnormalities as a sign of test abnormality. Reader disagreement occurs particularly in cases with rather subtle new wall motion abnormalities, whereas cases with pronounced wall motion abnormalities tend to be read more homogenously between different readers. DSMR tests are normally stopped after induction of even minor new wall motion abnormalities to prevent difficulty in handling emergency situations with the patient being in the MR scanner. Thus, DSMR cases might be stopped in many instances at an earlier stage of ischaemia than for instance dobutamine stress echocardiograms given the stress situation.
Considering the complex analysis process related to the interpretation of moving images, an imaging technique based on dynamic images may be at a disadvantage when compared with a technique which requires analysis of still frame images only, such as nuclear perfusion imaging techniques. It appears more vulnerable by the individual perception of the reader.
High interobserver agreement on test interpretation is required to ensure similar treatment decisions and avoid clinical misdirections. As long as we are dependent on subjective visual interpretation reading circumstances should be optimized. This relates to factors such as monitor brightness, contrast, playback rates, and side-by-side display of rest and stress images, which might have an impact on the individual perception of imaging data and confidence in reported test results. We might have to consider reporting the confidence level in our findings as an additional information to prevent inadequate rating of test results with subsequent inappropriate clinical decisions or treatments.
Objective quantitative measures of regional left ventricular function are desirable and may assist the evaluation of regional wall motion in the future. These techniques are becoming available for different imaging techniques and have reached a high level for MR imaging.10 However, they may function only as a supportive parameter to bolster our own subjective analysis of image data.
Conflict of interest: none declared.
Footnotes
The opinions expressed in this article are not necessarily those of the Editors of the European Heart Journal or of the European Society of Cardiology.
doi:10.1093/eurheartj/ehi883 
References
- De Rouen TA, Murray JA, Owen W. (1977) Variability in the analysis of coronary angiograms. Circulation 55:324–328.
[Abstract/Free Full Text] - Hoffmann R, Lethen H, Marwick T, Arnese M, Fioretti P, Pingitore A, Picano E, Buck T, Erbel R, Flachskampf FA, Hanrath P. (1996) Analysis of interinstitutional observer agreement in interpretation of dobutamine stress echocardiograms. J Am Coll Cardiol 27:330–336.[Abstract]
- Wacker FJT, Bodenheimer M, Fleiss JL, Brown M. (1993) Factors affecting uniformity in interpretation of planar thallium-201 imaging in a multicenter trial. J Am Coll Cardiol 21:1064–1074.[Abstract]
- Hoffmann R, Marwick TH, Poldermans D, Lethen H, Ciani R, van der Meer P, Tries HP, Gianfagna P, Fioretti P, Bax JJ, Katz MA, Erbel R, Hanrath P. (2002) Refinements in stress echocardiographic techniques improve inter-institutional agreement in interpretation of dobutamine stress echocardiograms. Eur Heart J 23:821–829.
[Abstract/Free Full Text] - Paetsch I, Jahnke C, Ferrari VA, Rademakers FE, Pellikka PA, Hundley WG, Poldermans D, Bax JJ, Wegscheider K, Fleck E, Nagel E. (2006) Determination of interobserver variability for identifying inducible left ventricular wall motion abnormalities during dobutamine stress magnetic resonance imaging. Eur Heart J 27:1459–1465 First published on April 13, 2006, doi:10.1093/eurheartj/ehi883.
[Abstract/Free Full Text] - Nagel E, Lehmkuhl HB, Bocksch W, Klein C, Vogel U, Frantz E, Ellmer A, Dreysse S, Fleck E. (1999) Non-invasive diagnosis of ischemia-induced wall motion abnormalities with the use of high-dose dobutamine stress MRI. Comparison with dobutamine stress echocardiography. Circulation 99:763–770.
[Abstract/Free Full Text] - Janssen C, Kuijpers D, van Dijkman P, Tineke W, Oudkerk M. (2004) Interobserver agreement in dobutamine cardiovascular magnetic resonance imaging (Abstract). Radiology 233:326.
- Hoffmann R, von Bardeleben S, Kasprzak JD, Borges AC, ten Cate F, Firschke C, Lafitte S, Al-Saadi N, Kuntz-Hehner S, Horstick G, Greis C, Engelhardt M, Vanoverschelde JL, Becher H. (2006) Analysis of regional left ventricular function by cineventriculography, cardiac magnetic resonance imaging, and unenhanced and contrast-enhanced echocardiography. J Am Coll Cardiol 47:121–128.
[Abstract/Free Full Text] - Tory M and Möller T. (2004) Human factors in visualization research. IEEE Trans Vis Comput Graph 10:72–84.[Medline]
- Moore CC, McVeigh ER, Zerhouna EA. (2000) Quantitative tagged magnetic resonance imaging of the normal human left ventricle. Top Magn Reson Imag 11:359–371.[CrossRef][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
Related articles in EHJ:
- Determination of interobserver variability for identifying inducible left ventricular wall motion abnormalities during dobutamine stress magnetic resonance imaging
- Ingo Paetsch, Cosima Jahnke, Victor A. Ferrari, Frank E. Rademakers, Patricia A. Pellikka, W. Gregory Hundley, Don Poldermans, Jeroen J. Bax, Karl Wegscheider, Eckart Fleck, and Eike Nagel
EHJ 2006 27: 1459-1464.[Abstract] [FREE Full Text]
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||