Table 3.

Details of performance of models

Performance in development populationPerformance in bootstrap or cross-validationPerformance in subset of populationPerformance in external population
Author, yearOutcomeDiscrimination AUROC (95% CI)CalibrationAccuracyDiscrimination AUROC (95% CI)CalibrationAccuracyDiscrimination AUROC (95% CI)CalibrationAccuracyDiscrimination AUROC (95% CI)CalibrationAccuracyExternal reference
Betes 2003a (21)ACN+0.65PPV 12.0-50.0
Betes 2003b (21)ACN0.67PPV 7.3-33.30.65 (0.61–0.69)Cai 2012 (22)
0.71 (0.64–0.78)Chen 2014 (24)
Cai 2012 (22)ACN0.74 (0.72–0.77)Sens 82.8; Spec 50.80.74 (0.72–0.77)0.74 (0.70–0.78)H-L P = 0.77Sens 80.3; Spec 51.20.65 (0.58–0.72)Chen 2014 (24)
Chen 2013 (23)ACN0.65 (0.61–0.69)H-L P = 0.093Sens 65.1; Spec 57.2; PPV 44.4; NPV 75.70.66 (0.62–0.68)
Chen 2014 (24)ACN0.75 (0.69–0.82)H-L P = 0.205Sens 93.8; Spec 47.6; PPV 9.1; NPV 99.30.75 (0.70–0.82)
Hassan 2013 (25)ACNH-L P = 0.30
Kaminski 2014 (26)ACN0.64aH-L P = 0.74a0.62 (0.60–0.64)E/O ratio 1 (0.95–1.06). H-L P = 0.16aSens 92.4, Spec 13.9, PPV 7.55, NPV 96.0;
Lin 2006 (27)ACN0.65 (0.61–0.70)Cai 2012 (22)
0.71 (0.64–0.77)Chen 2014 (24)
Lin 2013 (28)ACNMen 0.61 (0.58–0.65), Women 0.62 (0.58–0.66)
Stegeman 2014 (29)ACN0.76H-L P = 0.94Sens 40; Spec 93
Tao 2014a (31)ACN0.67 (0.65–0.69)H-L P = 0.210.66 (0.63–0.69)H-L P = 0.65Tao 2014 (31)
Yeoh 2011 (32)ACN0.66 (0.62–0.70)H-L P = 0.290.64 (0.60–0.68)H-L P = 0.49
Colditz 2000 (34)CCWomen 0.67 (0.64–0.70); Men 0.71 (0.68–0.74)bKim 2004 (17)
0.6Schroy 2012 (19)
Driver 2007a (36)CC0.72H-L P = 0.43
Ma 2010a (38)CC0.71 (0.68–0.74)0.66 (0.62–0.70)χ2 P = 0.20; E/O 1.19 (1.03–1.37)Ma 2010 (38)
Wei E 2009 (40)CC0.61 (0.59–0.63)
Shin 2014a (39)CC (male)0.77 (0.76–0.78)χ2 P = 0.220.77 (0.75–0.79)χ2 P = 0.029
Shin 2014d (39)CC (female)0.71 (0.69–0.73)χ2 P = 0.730.72 (0.70–0.74)χ2 P = 0.49
Driver 2007b (36)CRC0.70H-L P = 0.910.69
Dunlop 2013 (43)CRC0.59PPV 0.71; NPV 0.510.57Dunlop 2013 (43)
Han 2008 (45)CRC0.88 (0.81–0.94)Sens 94; Spec 77 PPV 82, NPV 9279% (71.5–86.5)Sens 88; Spec 64. PPV 67; NPV 87
Ma 2010c (38)CRC0.70 (0.68–0.72)0.64 (0.61–0.67)χ2 P = 0.08; E/O 1.09 (0.98–1.23)Ma 2010 (38)
Marshall 2010 (49)CRC0.80 (0.74–0.85)Sens 82; Spec 64; PPV 68, NPV 790.80 (0.76–0.84)Sens 72; Spec 70; PPV 70, NPV 720.76 (0.70–0.82)Sens 71.7; Spec 71.2Yip 2010 (20)
Tao 2014b (31)CRC0.71 (0.67–0.75)0.68 (0.57–0.79)Tao 2014 (31)
Taylor 2011 (50)CRC0.67
Wang 2013 (51)CRC0.770.72
Yarnall 2013 (54)CRC0.63
Freedman 2009a,b,c (37)CRC (male)0.61 (0.60–0.62)E/O ratio 0.99 (0.96–1.04)Park 2008 (18)
Jo 2012b (46)CRC (male)0.73 (0.68–0.77)0.70 (0.65–0.74)
Shin 2014c (39)CRC (male)0.76 (0.76–0.77)χ2 P = 0.10350.78 (0.77–0.79)χ2 P = 0.0003
Wells 2014b (53)CRC (male)0.690.68 (0.67–0.69)
Freedman 2009d,e,f (37)CRC (female)0.61 (0.59–0.62)E/O ratio 1.05 (0.98–1.11)Park 2008 (18)
Jo 2012a (46)CRC (female)0.65 (0.62–0.68)0.60 (0.56–0.64)
Shin 2014f (39)CRC (female)0.71 (0.70–0.72)χ2 P = 0.61230.73 (0.71–0.74)χ2 P = 0.1569
Wells 2014a (53)CRC (female)0.690.68 (0.67–0.69)
Ma 2010b (38)Rectal cancer0.68 (0.64–0.71)0.62 (0.57–0.66)χ2 P = 0.19; E/O 0.94 (0.78–1.12)Ma 2010 (38)
Shin 2014b (39)Rectal cancer (male)0.75 (0.74–0.76)χ2 P = 0.290.78 (0.77–0.79)χ2 P = 0.0003
Shin 2014e (39)Rectal cancer (female)0.70 (0.68–0.71)χ2 P = 0.0840.72 (0.70–0.74)χ2 P = 0.198

Abbreviations: ACN+, advanced colorectal neoplasia including moderate dysplasia; ACN, advanced colorectal neoplasia defined as invasive cancer, an adenoma 10 mm or more, a villous adenoma (at least 25% villous) or an adenoma with high-grade dysplasia; CC, colon cancer; CRC, colorectal cancer; AUROC, area under the receiver operating characteristic curve. Values given as mean and 95% confidence intervals; Sens, sensitivity; Spec, specificity; PPV, positive predictive value; NPV, negative predictive value; H-L, Hosmer–Lemeshow test; E/O, expected over observed ratio.

  • aThese values are from the model prior to conversion of the coefficients to scores (Kaminski).

  • bRemoved aspirin use from men and history of chronic IBD from both genders as not available so actually not validating original score.