OUP user menu

Development and validation of a prognostic risk score for major bleeding in patients undergoing percutaneous coronary intervention via the femoral approach

Eugenia Nikolsky , Roxana Mehran , George Dangas , Martin Fahy , Yingbo Na , Stuart J. Pocock , A. Michael Lincoff , Gregg W. Stone
DOI: http://dx.doi.org/10.1093/eurheartj/ehm194 1936-1945 First published online: 16 June 2007


Aims Major bleeding after percutaneous coronary intervention (PCI) is an independent risk factor for early and late mortality. We developed and validated a risk score predictive of major bleeding after PCI using the femoral approach.

Methods and results Baseline clinical and procedural variables from two contemporary, multicentre, randomized PCI trials were used for risk score development (the REPLACE-2 trial, n = 6002) and validation (the REPLACE-1 trial, n = 1056). On the basis of the odds ratio, independent risk factors were assigned a weighted integer, the sum of which comprised a total risk score. Seven variables were identified as independent correlates of major bleeding (age >55 years, female gender, estimated glomerular filtration rate <60 mL/min/1.73 m2, pre-existing anaemia, administration of low-molecular-weight heparin within 48 h pre-PCI, use of glycoprotein IIb/IIIa inhibitors, and intraaortic balloon pump use). In the development set, the risk of major bleeding varied from 1.0% in patients without risk factors to 5.4% in high-risk patients. The discriminatory power of this risk model was confirmed in the validation data set (area under the receiver operating curve = 0.62).

Conclusion A simple risk score of baseline clinical and procedural variables is useful to predict the incidence of major peri-procedural bleeding after contemporary PCI using the femoral approach.

  • Major bleeding
  • Angioplasty
  • Risk score
See page 1917 for the editorial comment on this article (doi:10.1093/eurheartj/ehm265)


Potent adjunctive pharmacologic therapies, including aspirin, clopidogrel, heparin, and glycoprotein (GP) IIb/IIIa inhibitors, are utilized to reduce ischaemic events following percutaneous coronary interventions (PCIs), the combination of which not infrequently results in peri-procedural bleeding complications. Haemorrhagic events have a major deleterious impact on the clinical outcomes and costs after PCI.16 Major bleeding has been strongly associated with increased rates of in-hospital and late mortality, myocardial infarction (MI), and repeat revascularization procedures after PCI.13 Blood product transfusions have also been significantly associated with reduced survival.4,5 The additional resources required for the diagnosis and management of haemorrhagic complications and resultant prolonged duration of in-hospital stay markedly increase the cost of hospitalization.6

Previous studies have associated certain clinical and procedural characteristics with bleeding complications after PCI, including advanced age,7 female gender,8 lower weight,9 chronic renal insufficiency,10,11 type of antithrombotic regimen,1214 increased activated clotting time (ACT),1517 and larger sheath diameter.18 Given the frequent co-existence of these and other co-morbid conditions, a clinical tool to aid in accurate risk stratification would be useful to identify high-risk cohorts at excessive risk for major bleeding and to subsequently guide clinical decision-making in determining the risk–benefit ratio of PCI. We, therefore, sought to develop and validate a risk score of PCI-related major bleeding in patients undergoing PCI via the femoral approach utilizing data from the contemporary large-scale, multicentre randomized REPLACE-1 and REPLACE-2 trials.


Protocols, inclusion and exclusion criteria, and principal results of the prospective, randomized pilot REPLACE-1 and pivotal REPLACE-2 trials have been reported in detail elsewhere.19,20 In brief, in REPLACE-1, 1056 patients were randomized to either bivalirudin (0.75 mg/kg bolus and 1.75 mg/kg/h infusion for the duration of the procedure) or unfractionated heparin (initial bolus 60–70 IU/kg, adjusted to achieve an ACT of 200–300 s), each with or without GP IIb/IIIa inhibitors (abciximab, eptifibatide, or tirofiban) at the operator's discretion, which was pre-specified prior to randomization. In REPLACE-2, 6002 patients were assigned to undergo PCI via the femoral approach either with unfractionated heparin (65 IU/kg bolus, maximum 7000 U) and planned GP IIb/IIIa inhibition (abciximab or eptifibatide) or bivalirudin (0.75 mg/kg bolus and 1.75 mg/kg/h infusion for the duration of the procedure) with provisional GP IIb/IIIa inhibition.

In both trials, in addition to aspirin, pretreatment with a 300 mg clopidogrel loading dose was encouraged followed by 75 mg/day for at least 30 days. The choice of GP IIb/IIIa inhibitor was at the discretion of the investigators in both trials. Major exclusion criteria included acute MI, intervention on left main coronary artery, serum creatinine >4.0 mg/dl or dependence on dialysis, significant haemorrhagic risk (active internal bleeding or bleeding diathesis, surgery, trauma, or gastrointestinal or genitourinary tract bleeding within 6 weeks), intracranial neoplasm or vascular malformation, prior intracranial bleeding, and thrombocytopenia <100 × 109 L−1. Patients were also excluded if they required therapy with warfarin or had been treated with unfractionated heparin within 6 h before randomization, low-molecular-weight heparin (LMWH) within 12 h (REPLACE-1) or within 8 h (REPLACE-2), abciximab within 7 days, or eptifibatide or tirofiban within 12 h of randomization. In both trials, unless a closure device was used to achieve haemostasis, vascular access sheaths were to be removed when the ACT or activated partial thromboplastin time fell below 175 or 50 s, respectively. Femoral access was exclusively used in the REPLACE-2 trial. In the REPLACE-1 trial, radial artery access was utilized in 2.5% of patients; these patients were excluded from the present analysis.

In the REPLACE-1 trial, major bleeding was defined as intracranial, intraocular, or retroperitoneal haemorrhage, or clinically overt blood loss resulting in a decrease in haemoglobin by >3 g/dl, or red cell transfusion of ≥2 units. In the REPLACE-2 trial, the definition of major bleeding included all of the above plus any decrease in haemoglobin >4 g/dl without an overt source of bleeding. For the purpose of this analysis, the REPLACE-2 definition of major bleeding was reconstructed from the detailed clinical and laboratory data from the REPLACE-1 data set. Anaemia was defined using World Health Organization criteria: baseline haematocrit value of <39% for men and <36% for women.21 Chronic renal insufficiency was a baseline estimated glomerular filtration rate (eGFR) of <60 mL/min/1.73 m2 (Levey modified MDRD formula).22 All bleeding events were adjudicated by an independent clinical event committee.

Statistical analysis and risk score development and validation

Risk score development and validation data sets were created from the REPLACE-2 and REPLACE-1 trials, respectively. The development risk score model was formed by identifying univariate predictors of major bleeding, based on the available clinical and procedural variables. Subsequently, multivariable logistic regression was used to identify independent predictors of major bleeding with significance level of entry/stay 0.05. Two sets of models were created: one using clinical plus procedural variables and one using just clinical variables. The following clinical variables were considered for entry into the model: age, gender, smoking status, presence/absence of diabetes mellitus, congestive heart failure and/or anaemia, weight, acute coronary syndrome at presentation, systolic and diastolic blood pressure pre-procedure, eGFR, administration of LMWH within 48 h before PCI, and the use of thienopyridines loading dose. The following procedural variables were considered: maximal ACT during PCI, target lesion type by American College of Cardiology/American Heart Association (ACC/AHA) class, baseline Thrombolysis in Myocardial Infarction (TIMI) flow grade, use of stent and/or atherectomy, procedure duration, use of an intraaortic balloon pump (IABP), and administration of GP IIb/IIIa inhibitors. The internal validity of the final predictive models was assessed by the bootstrap re-sampling method.23 For this purpose, 100 bootstrap samples of size 3000 were obtained from the REPLACE-2 data set, and the model was refitted within each sample. Covariates that were selected in at least 90% of the bootstrap samples were included in the final multivariable models. We used this technique to select the most robust set of covariates for the risk score model. Based on the z-score (the model coefficient divided by its standard error), chosen variables were assigned a weighted integer, the sum of the integers representing the total risk score for each patient.

Continuous variables (in this case, age) were investigated for non-linearity, and cut-off values were determined to create a binary variable when appropriate. For this purpose, we examined plots of age vs. rates of major bleeding. The age value corresponding to an increased risk of major bleeding was considered as the cut-off value. The risk score was then tested in the validation data set. Each model's goodness-of-fit was defined using the Hosmer–Lemeshow method.24 Model discrimination was assessed using the area under the receiver operating curve (ROC) or the c-statistic. Interactions were evaluated between the identified significant predictors and previously reported predictors of major bleeding.718

Continuous variables are expressed as mean ± 1 SD and were compared using Student's t-test. Categorical data are presented as frequencies and were compared using χ2-statistics.


Clinical and procedural characteristics of patients in the development and validation sets are shown in Table 1. Patients in the development set (REPLACE-2 trial) compared with the patients in the validation set (REPLACE-1 trial) were younger, more frequently male, were less likely to have diabetes mellitus, hypertension, congestive heart failure, prior MI, prior coronary artery bypass grafting, and baseline anaemia, had less frequent left circumflex artery and saphenous vein graft intervention, a higher frequency of atherectomy device usage, a lower frequency of GP IIb/IIIa inhibitor administration, and had shorter procedure duration and greater maximal intra-procedure ACT.

View this table:
Table 1

Baseline clinical and procedural data examined in the development and validation data sets

Predictors of major bleeding and risk score development

Of the 6002 patients in the REPLACE-2 development data set, in-hospital major bleeding developed in 195 patients (3.2%). A total of 24 bleeding events were related to CABG and were excluded from the analysis. In-hospital major bleeding not related to CABG thus occurred in 171 patients (2.9%). Univariate associations between baseline demographic and procedural characteristics and major bleeding appear in Table 2. Patients who developed major bleeding compared with those who did not were older (67.4 ± 10.9 vs. 62.5 ± 10.9 years, P < 0.0001), more frequently female, had a higher incidence of congestive heart failure, anaemia, and chronic renal insufficiency at baseline, more frequently had lesion type B2/C by ACC/AHA classification and baseline TIMI flow grade 0/1, had lower weight (84.6 ± 18.0 vs. 87.5 ± 18.2 kg, P = 0.042) and higher pre-procedure systolic blood pressure (140.7 ± 23.2 vs. 136.9 ± 21.1 mm Hg, P = 0.032), were more likely to be treated with LMWH within 48 h pre-PCI as well as with atherectomy and IABP during PCI, and were more likely to receive GP IIb/IIIa inhibitors. Major bleeding was also associated with longer PCI duration (36.4 ± 41.4 vs. 23.5 ± 22.2 min; P = 0.0002), and higher rates of procedural complications including coronary artery dissection with reduced intraprocedural TIMI flow (6.4% vs. 2.5%, P = 0.006) and thrombus formation (4.7% vs. 1.1%, P = 0.0009). The maximal value of intraprocedural ACT was not related to the development of major bleeding (353 ± 81 s vs. 356 ± 83 s, P = 0.68).

View this table:
Table 2

Univariate association of baseline clinical and procedural characteristics to major bleeding in the development data set

The multivariable model of predictors of major bleeding was obtained using data for 5395 patients with no missing covariate values and included 151 of the 171 patients that developed major bleeding (88.3%). Multivariable predictors of major bleeding included (in order of their statistical strength of contribution to the model): use of IABP, administration of GP IIb/IIIa inhibitors, older age, female gender, chronic renal insufficiency, baseline anaemia, and administration of LMWH within 48 h pre-procedure. There were no statistically significant interactions between these independent predictors and previously reported predictors (weight and maximal ACT) of major bleeding, when specifically tested in multivariable models. On the basis of the z-scores, a weighted integer score was assigned to each of the multivariate predictors (Table 3). For age, 55 years was identified as the optimal cut-off value for increased risk of major bleeding.

View this table:
Table 3

Multivariable predictors of major bleeding in the development data set in order

As shown in Figure 1, the incidence of major bleeding increased from 0.9 to 22.2% depending on the risk score group (Cochran Armitage test for trend P < 0.0001). Patients were further categorized into four groups: very low risk [risk score 0; 421 (7.8%) patients, with a major bleeding rate of 1.0%], low risk [risk score 2–6; 2280 (42.3%) patients, with a major bleeding rate of 1.5%]; moderate risk [risk score 7–9; 1131 (21.0%) patients, with a major bleeding rate of 2.6%]; and high risk [risk score ≥10; 1563 (28.9%) patients, with a major bleeding rate of 5.4%] (Figures 2 and 3). Increased rates of both access-site-related and non-access-site major bleeding were observed with increasing risk score (Figure 4). The incidence of any transfusion increased from 0.7% in very-low-risk patients (3 of 421 patients), to 0.8% in low-risk patients (18 of 2280 patients), to 1.9% in moderate-risk patients (21 of 1131 patients) and to 4.7% in high-risk patients (73 of 1563 patients) (P for trend <0.0001).

Figure 1

Risk of major bleeding with increasing numbers of clinical and procedural risk variables present in the development data set.

Figure 2

Algorithm used to determine the risk score for major bleeding using clinical and procedural variables. eGFR, estimated glomerular filtration rate; SCr, serum creatinine; PCI, percutaneous coronary intervention.

Figure 3

Rates of major bleeding in patients with very low, low, moderate, and high-risk scores, and discriminatory performance of the risk score model using both clinical and procedural variables in the development data set (REPLACE-2 trial) and the validation data set (REPLACE-1 trial).

Figure 4

Rates of access-site-related and non-access site major bleeding in patients with very low, low, moderate, and high-risk score, and discriminatory performance of the risk score model using both clinical and procedural variables in the development data set (REPLACE-2 trial) and the validation data set (REPLACE-1 trial).

Validation of the risk score

In-hospital non-CABG-related major bleeding occurred in 26 of 944 patients (2.8%) in the REPLACE-1 validation cohort. The rates of major bleeding in the validation set increased across the four risk groups (P < 0.0001) (Figure 3). The data did not deviate significantly from the logistic model as indicated by the non-significant Hosmer–Lemeshow goodness-of-fit test (P = 0.95). The area under the ROC was 0.62, indicating good ability to discriminate between patients with various degree of risk for major bleeding.

Clinical risk score model of major bleeding

When only clinical variables present at the time of admission were included (i.e. age, gender, baseline anaemia, chronic renal insufficiency, and the use of LMWH within 48 h pre-PCI, with the intraprocedural use of IABP and administration of GP IIb/IIIa inhibitors no longer considered), the ability of the development data set risk score model to reliably predict the likelihood of major bleeding was preserved, with the bleeding risk rising from 1.3–12.1% with increasing risk score (Cochran Armitage test for trend P < 0.0001). The rates of major bleeding in patients with no risk factors (risk score 0), and those at low (risk score 2–6), moderate (7–9), and high risk (≥10) in the development set were 1.3, 1.8, 2.7, and 5.0%, respectively (Cochran Armitage trend test P < 0.0001) (Figure 5), similar to the incidence of major bleeding across risk groups in the validation set (1.7, 1.9, 1.7, and 5.0%, respectively; Cochran Armitage test for trend, P = 0.029). The developed model demonstrated satisfactory discriminative power in the validation cohort (c-statistic, 0.65).

Figure 5

Algorithm used to determine the risk score for major bleeding using clinical variables only. eGFR, estimated glomerular filtration rate; SCr, serum creatinine; PCI, percutaneous coronary intervention.

Risk score and different anticoagulation regimens

Applying the clinical model to the groups stratified by different anticoagulation regimens in the REPLACE-2 trial showed that the increasing risk score was strongly associated with major bleeding both in patients treated with bivalirudin alone (1.0, 1.0, 1.9, and 4.3% for the very low, low, moderate, and high-risk patients, respectively; P < 0.0001) and in patients treated with heparin plus GP IIb/IIIa inhibitors (1.7, 2.6, 3.4, and 5.8%; P < 0.0001). Treatment with bivalirudin alone rather than heparin plus GP IIb/IIIa inhibitors was associated with a significantly reduced risk of major bleeding in low-risk patients, with similar numerical trends toward reduced bleeding in the other risk groups (Figure 6).

Figure 6

Rates of major bleeding in patients treated with bivalirudin vs. heparin plus GP IIb/IIIa receptor inhibitors in very low, low, moderate, and high-risk patients defined using the modified (clinical variables only) risk score model in the development data set (the REPLACE-2 trial). GP, glycoprotein.


In the present post hoc study, a risk score for major bleeding after PCI utilizing readily available clinical and procedural information was developed and validated with two independent data sets and showed good discrimination in predicting adverse haemorrhagic events in patients treated with several different anticoagulation regimens. Although many of the variables predictive of major bleeding have been individually identified in the past, the reported risk score is the first to be developed from two large, systematically collected data sets allowing detailed assessment of the independent correlates of major bleeding and their relative contribution to a clinically useful risk model. For the online assessment of major bleeding risk score in patients undergoing PCI the reader is referred to www.bleedingriskscore.org.

The risk score model utilizing both clinical and procedural characteristics demonstrated good prognostic accuracy for major bleeding in both the derivation and validation sets, being able to distinguish between cohorts at very low, low, moderate, and high risk for major bleeding, whether treated with a heparin+GP IIb/IIIa inhibitors or bivalirudin-based regimens. This combined clinical and procedural model may be useful to determine which patients are at greatest risk after angioplasty, potentially warranting closer observation. The modified model using only five pre-procedural clinical variables was also able to discriminate between patients at low, moderate, and high risk for major haemorrhage. This model may be used prior to intervention to assess haemorrhagic risk and guide treatment decisions.

Comment is warranted regarding several variables which were strongly related to major bleeding in the multivariate model.

Older age and bleeding risk

The elderly represent a rapidly growing segment of the population.25 Despite a greater absolute reduction of ischaemic complications after revascularization, the risk of procedural haemorrhagic events is significantly increased in elderly compared with young patients.7,2629 In this study, older age was the most powerful clinical predictor of major bleeding after PCI; the unadjusted risk of major bleeding was doubled in patients older than 55 years of age. The reasons for the higher haemorrhagic risk in the elderly have not been specifically studied but are likely multifactorial, including reduced renal function, concomitant peripheral vascular disease with more frequent access site bleeding, as well as greater sensitivity to anticoagulant agents and drug interactions.29,30

Gender and bleeding risk

In agreement with previous studies, the present analysis provides evidence that women are at increased risk of major bleeding after PCI.8,31,32 Although the exact mechanisms are unknown, this finding may be related to smaller vessel size and a higher incidence of vascular access-site-related complications, as well as a tendency to over-anticoagulation because of smaller body mass.

Renal insufficiency and bleeding risk

Consistent with prior studies,10,11 chronic kidney disease was associated with 2.3-fold higher unadjusted odds of major bleeding, which persisted after multivariate adjustment. The excess risk of bleeding in the setting of renal insufficiency has been attributed to disturbances in the coagulation system coupled with altered responses to medications.10,11,33,34

Of note, in the CRUSADE (Can Rapid Risk Stratification of Unstable Angina Patients Suppress Adverse Outcomes With Early Implementation of the American College of Cardiology/American Heart Association Guidelines) National Quality Improvement Initiative prospective Registry, elderly patients, women, and those with lower body weight and chronic renal insufficiency were more likely to receive excessive doses of antithrombotic agents, which correlated with increased haemorrhagic complications.35 This observation may provide an additional explanation for the excess bleeding rates observed in these patient cohorts.

Baseline anaemia and risk of bleeding

In this study, baseline anaemia and the likelihood of PCI-related major bleeding were strongly correlated. One-third of patients in both REPLACE trials had anaemia at baseline, and although a recent major haemorrhagic episode was an exclusion criterion for both trials, the risk of major bleeding was almost doubled in patients presenting with baseline anaemia. Gastrointestinal bleeding was four-fold increased in patients with vs. without anaemia at baseline (0.8% vs. 0.2%; P < 0.0001), emphasizing the importance of a thorough search for predisposing bleeding sites and haemorrhagic diatheses in patients with baseline anaemia.

Lower weight and risk of bleeding

In a prior pooled analysis from four large, contemporary PCI trials, lower weight was inversely correlated with bleeding, and greater weight-adjusted doses of unfractionated heparin were independently associated with higher bleeding rates.15 Excess vascular complications may explain the association between thin body habitus and haemorrhagic risk.8 In this report, lower weight was associated with an increased incidence of major bleeding by univariate but not multivariable analysis.

Activated clotting time and risk of bleeding

In our study, a longer peak procedural ACT was not a predictor of major bleeding. Prior reports examining the association between the ACT and bleeding risk have been mixed. In one study of 429 patients undergoing elective or urgent PCI, the maximum ACT was a multivariable predictor of bleeding, although most events were minor in severity.15 In the Enhanced Suppression of the Platelet IIb/IIIa Receptor with Integrilin Therapy (ESPRIT) trial, major bleeding rates increased as the ACT increased in the heparin plus eptifibatide-treated patients, but not in the heparin plus placebo group.17

Anticoagulation regimen and risk of bleeding

GP IIb/IIIa inhibitors have been shown to increase the rates of bleeding complications in the setting of PCI, despite the use of lower doses of weight-adjusted heparin and early removal of vascular sheaths.1214,29 In this study, administration of GP IIb/IIIa inhibitors was also one of the most powerful independent predictors of major bleeding. Although administration of LMWH within 8 h (REPLACE-2) or 12 h (REPLACE-1) pre-procedure was an exclusion criterion in the trials, receiving LMWH within 48 h pre-PCI also predicted major bleeding in this study. Although speculative, residual anti-Xa activity from LMWH prior to PCI may have contributed to excess of major bleeding in this setting.36 A higher risk of bleeding may also result if patients cross-over between different anticoagulant therapies.37

Other factors and risk of bleeding

Procedural IABP use was the strongest determinant of major bleeding in this study. Minimizing sheath diameter, as well as early IABP removal when possible, may decrease the risk of access-site-related complications.38


Several study limitations warrant discussion. The baseline features of the patients enrolled in the REPLACE-1 and REPLACE-2 trials differed, perhaps reflecting the fact that the former was USA-based investigation, whereas the latter international study recruited patients from nine countries. Nonetheless, the fact that the risk score model performed equally well in the validation set as in the development set despite these differences suggests that it is robust and generalizable. The GP IIb/IIIa inhibitors used in the REPLACE-2 trial included abciximab and eptifibatide, whereas tirofiban was also used in the REPLACE-1 trial. Larger comparative studies are required to determine whether there are differences in bleeding risk between these agents. GP IIb/IIIa inhibitors were used in only 7.2% of bivalirudin-treated patients in REPLACE-2 (solely for bail-out purposes after procedural complications)20; thus, the independent influences of these two agents on bleeding cannot be determined from the development data set. Information on the presence of peripheral arterial disease was lacking in both trial databases. Sheath size was also not available as a variable to consider, and the degree to which IABP use was a risk factor because the sheath size cannot be determined. Multivariable models cannot account for unmeasured baseline differences, and adjusted effects may still be influenced by residual confounding. Furthermore, given the selection bias inherent in the recruitment of randomized trial populations, validation of the risk score developed in this report is warranted in other registry databases including unselected ‘real-world’ patients. Moreover, as the definition of major bleeding differs between contemporary PCI trials, the applicability of our risk score model should be further assessed utilizing different scales of major bleeding. Finally, the results of the current analysis apply only to patients undergoing PCI using femoral access, and cannot be extrapolated to either the radial or brachial approaches, although the risk model was able to predict non-access-site-related as well as femoral access-site-related bleeding.

Conclusions and clinical implications

With the routine use of more potent antithrombotic and antiplatelet agents, the occurrence of major bleeding is increasing in frequency after PCI, significantly raises hospitalization costs, and confers an unfavourable early and late prognosis.1,6 This report demonstrates that risk stratification for the development major bleeding is possible using readily available clinical and procedural variables. Accurate prognostication of haemorrhagic risk should provide clinical utility helping to select between different revascularization options or medical therapy and to guide treatment decisions to minimize bleeding (including choice of anticoagulation regimen).


The authors wish to thank Masha Nikolski (Department of Computer Science, Technion-Israel Institute of Technology, Haifa, Israel) for designing and implementation of the website for quick calculation of major bleeding risk score.

Conflict of interest: A.M.L is a principal investigator of the REPLACE-2 study and assumes responsibility for the integrity of the data from that trial. G.W.S has received research support from, and is a consultant for, The Medicines Company.


View Abstract