
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Research Articles |
Authors' Affiliations: 1 The Pulmonary Center, Boston University Medical Center; 2 Bioinformatics Program and 3 School of Public Health, Boston University; 4 Biostatistics Solutions Consulting; and 5 Department of Genetics and Genomics, Boston University School of Medicine, Boston, Massachusetts
Requests for reprints: Avrum Spira, The Pulmonary Center, Boston University Medical Center, 715 Albany Street, R304, Boston, MA 02118. Phone: 617-414-6980; Fax: 617-536-8093; E-mail: aspira{at}bu.edu.
| Abstract |
|---|
|
|
|---|
Training (n = 76) and test (n = 62) sets consisted of smokers undergoing bronchoscopy for suspicion of lung cancer at five medical centers. Logistic regression models describing the likelihood of having lung cancer using the biomarker, clinical factors, and these data combined were tested using the independent set of patients with nondiagnostic bronchoscopies. The model predictions were also compared with physicians' clinical assessment.
The gene expression biomarker is associated with cancer status in the combined clinicogenomic model (P < 0.005). There is a significant difference in performance of the clinicogenomic relative to the clinical model (P < 0.05). In the test set, the clinicogenomic model increases sensitivity and negative predictive value to 100% and results in higher specificity (91%) and positive predictive value (81%) compared with other models. The clinicogenomic model has high accuracy where physician assessment is most uncertain.
The airway gene expression biomarker provides information about the likelihood of lung cancer not captured by clinical factors, and the clinicogenomic model has the highest prediction accuracy. These findings suggest that use of the clinicogenomic model may expedite more invasive testing and definitive therapy for smokers with lung cancer and reduce invasive diagnostic procedures for individuals without lung cancer.
|
|
|---|
Smokers are often suspected of having lung cancer based on abnormal radiographic findings and/or symptoms that are not specific for lung cancer. Fiberoptic bronchoscopy represents a relatively noninvasive initial diagnostic test used in this setting, with cytologic examination of materials obtained via endobronchial brushings, bronchoalveolar lavage, and endobronchial and transbronchial biopsies of the suspect area (4, 5). Whereas cytopathology is 100% specific for lung cancer, the sensitivity of cytologic examination of materials obtained at bronchoscopy ranges from 30% for small peripheral lesions to 80% for centrally located endobronchial tumors (6). Given the relatively low sensitivity of bronchoscopy, additional and more invasive diagnostic tests are routinely needed, which are costly, incur risk, and prolong the diagnostic evaluation of patients with suspect lung cancer. Determining which suspect lung cancer patients with lung cancer-negative bronchoscopies should undergo these additional diagnostic tests is currently a matter of clinical judgment. We have recently reported a gene expression profile in cytologically normal large airway epithelial cells obtained via brushing at the time of bronchoscopy, which serves as a diagnostic biomarker for lung cancer (7). This biomarker is an accurate predictor of lung cancer at an early and potentially curable stage, and the sensitivity of the biomarker could substantially reduce the number of individuals requiring additional invasive diagnostic testing following a lung cancer–negative bronchoscopy.
Many groups have developed gene expression profiles that can be used to distinguish between different diagnostic and prognostic subgroups in a variety of cancers. An unexplored issue for many of these biomarkers is whether the gene expression patterns are independent of other clinical risk factors. If so, it presents an opportunity to create clinicogenomic models that incorporate both clinical and gene expression predictors of disease likelihood. There are several examples of such clinicogenomic approaches. Pittman et al. (8) have shown improved prediction accuracy for breast cancer recurrence through an integrative clinicogenomic model. Similarly, Li (9) combined genomic and clinical data in a survival model to predict the outcome of patients with diffuse large B-cell lymphoma after chemotherapy. Stephenson et al. (10) integrated gene expression and clinical data using logistic regression modeling to predict prostate carcinoma reoccurrence after radial prostatectomy, and showed that a combined model had the highest predictive accuracy. In the near future, diverse sources of data such as gene expression, genetic, proteomic, and clinical data will likely be integrated to make accurate diagnoses or prognostic predictions for complex diseases such as cancer (11).
With
90 million former and current smokers in the United States (12) and the emergence of sensitive but nonspecific chest imaging technologies, patients increasingly present to clinicians with abnormal radiographic findings that are suspicious for lung cancer. Whereas no definitive predictive model for lung cancer exists for use in this setting, numerous clinical and radiographic variables have been associated with the likelihood of lung malignancy: age (13), smoking history (ref. 14; including number of pack-years, age started, intensity of smoking and years since quitting), history of asbestos exposure, clinical symptoms including hemoptysis and weight loss (15), size of the nodule or mass and radiographic appearance on chest imaging (15, 16), presence of lymphadenopathy, clinical or radiographic evidence for metastatic disease, evidence of airflow obstruction on spirometry (16), and uptake of fluorodeoxyglucose on positron emission tomography scan (17, 18). Several groups have developed predictive models using combinations of the above variables in the setting of solitary pulmonary nodules (15, 19, 20). Swensen et al. (21) compared such a model for the presence of solitary pulmonary nodules with predictions made by physicians and found that there was no significant difference, although they suggested that the model had potential in the management of patients with benign nodules. In addition, risk prediction models for lung cancer, including a recent large case-control study of never, former, and current smokers, have been reported (22).
In this study, we sought to evaluate whether the lung cancer predictions made by our large airway gene expression biomarker are independent of other clinical risk factors and, if so, to determine the relative performance of a clinicogenomic model that combines clinical risk factors with the biomarker. We show that the biomarker provides information about the likelihood of a patient having lung cancer beyond that which is contained in the available clinical data, despite the clinical model predictions being highly associated with the subjective clinical assessment of patient risk made by pulmonary physicians. Furthermore, we find that the clinicogenomic model has better diagnostic accuracy than either the clinical model or the gene expression biomarker alone. Our data suggest that the clinicogenomic model could be efficacious in predicting the likelihood of lung cancer in those patients where physicians are most uncertain about the likelihood of disease.
| Materials and Methods |
|---|
|
|
|---|
|
|
|
Construction of logistic regression models
Logistic regression models to quantify the probability of a patient having lung cancer were generated using the training set samples (n = 76). This training set included patients who had cytopathology findings that confirmed a diagnosis of either lung cancer or alternate noncancer pathology. Patients with diagnostic bronchoscopies were included in the training set to maximize the number of samples and because exclusion of these samples was unnecessary to develop models capable of accurately predicting the lung cancer status of patients with nondiagnostic bronchoscopies (data not shown). For the clinical and clinicogenomic models, the available clinical variables (Table 1) included age, pack-years of smoking, and the following dichotomous variables: gender (male, 1; female, 0), race (1, Caucasian; 0, otherwise), smoking status (1, former smokers that quit
10 y ago; 0, otherwise), hemoptysis (1, presence; 0, otherwise), lymphadenopathy (1, mediastinal or hilar lymph nodes >1 cm on computed tomography chest scan; 0, otherwise), and mass size (1, having a mass size >3 cm; 0, otherwise). Positron emission tomography scan information was only available for 15 patients and was not included in the model. Backward stepwise model selection using Akaike's information criterion (24) was used to select the optimal clinical model for the probability of a patient having lung cancer.
To create an integrated clinicogenomic model and determine the independence and magnitude of the contribution of the gene expression biomarker after adjusting for the effects of the clinical variables, we first added the biomarker to the optimal clinical model. The biomarker scores and all of the available clinical variables were then used with backward stepwise model selection by Akaike's information criterion to select the optimal model. Both approaches yielded the same combined model. To verify that the biomarker score performs similarly in logistic regression as in the weighted-voting prediction algorithm used in our previous work (7), the accuracy, sensitivity, specificity, positive predictive value, and negative predictive value were compared for the weighted-voting predictions and the predictions made by a logistic regression model that included only the biomarker score across the independent test samples.
Comparison of model performance on independent patients
The performance of the logistic regression models (clinical, biomarker, and clinicogenomic) was initially evaluated on the subset of patients in the training set (n = 76) in which the cytopathology of materials obtained at bronchoscopy was nondiagnostic (n = 56; Fig. 1). We chose to focus on nondiagnostic bronchoscopies to specifically assess the utility of the gene expression biomarker and clinical parameters in the setting of patients that require further diagnostic evaluation for lung cancer. More importantly, we also tested the models in the nondiagnostic bronchoscopy test set (n = 62; Fig. 1). For each of the models, patients that had a probability of lung cancer
0.5 were classified as having lung cancer, and patients with a probability <0.5 were classified as not having lung cancer. Receiver operating characteristic (ROC) curves were also used to compare the clinical model with the clinicogenomic model in the training set patients with nondiagnostic bronchoscopies, the independent test set, and combined set of all patients with nondiagnostic bronchoscopies (n = 118). To assess whether or not two ROC curves based on the same set of samples were significantly different, methods developed for comparing ROC curves derived from the same cases were used (25, 26). To compare ROC curves based on different sample sets, we used a two-sample z test. The ROC curves serve as a common scale for evaluating the additional merit of variables added to the model because odds ratios for two different variables may not be comparable (27). The accuracy, sensitivity, specificity, positive predictive value, and negative predicate value were also calculated across the independent test set for the clinical model, the biomarker model, and the clinicogenomic model.
Subjective clinical assessment
Three independent pulmonary clinicians practicing at a tertiary medical center, blinded to the final diagnoses, evaluated each patient's clinical history at the time of bronchoscopy. The history included, but was not limited to, age, smoking status, cumulative tobacco exposure, comorbidities, symptoms/signs, radiographic findings, and positron emission tomography scan results if available. Based on this information, the clinicians classified each patient into one of the three risk groups: low (<10% assessed probability of lung cancer), medium (10-50% assessed probability of lung cancer), and high (>50% assessed probability of lung cancer). The final subjective assignment for each subject was decided by choosing the median opinion. The interrater reliability for the clinical classification of patients' nondiagnostic bronchoscopies was significant, indicating that the level of agreement between the clinicians was greater then would be expected by chance as measured by the
statistic (
= 0.57; P < 0.001; ref. 28).
Comparison of subjective clinical assessment with the clinicogenomic model
The sample size for building a comprehensive clinical model to predict the risk of having lung cancer was limited as was the scope of variables that were available for inclusion in the clinical and clinicogenomic models. We therefore sought to determine if the clinical model performs similarly to the subjective clinical assessment made by pulmonary specialists because this assessment is (a) "trained" on the large number of patients seen over each clinician's career and (b) considers all of the information contained within a patient's medical records. A Wilcoxon test was used to assess whether or not the clinical model–derived probability of having lung cancer varied between samples classified as low, medium, or high cancer risk by the clinicians.
Statistical analysis
All statistical analyses were conducted using R statistical software version 2.2.1.
| Results |
|---|
|
|
|---|
|
Evaluating the performance of the clinicogenomic model
The three models were used to predict the cancer status of a subset of the training samples with nondiagnostic bronchoscopies (n = 56), the independent test samples (n = 62), and these two sets combined (n = 118). ROC curves were used to compare the performance of the clinical model with that of the clinicogenomic model (Fig. 2). The clinicogenomic model had better performance than the clinical model in all three sample sets. Whereas this difference in performance does not reach statistical significance in the test set, when the training and test sets were combined, there was a significant difference in the area under the curve between the clinicogenomic and clinical models (P < 0.05). The performance of the models in the training set samples does not seem to be any better than in the test set samples (P = 0.25, for the difference in the area under the ROC curves; the area under the curve difference is 0.065; 95% confidence interval, –0.046 to 0.174). This suggests that the models do not overfit the training data and that it is therefore reasonable to combine the training and test sets to assess the significance of the difference in the performance of the clinical and clinicogenomic models.
|
|
|
|
|
| Discussion |
|---|
|
|
|---|
Our results suggest that the pattern of gene expression in large airway epithelial cells reflects information about the presence of lung cancer that is independent of other clinical risk factors. This interpretation results from a comparison of models that contain either clinical variables or the biomarker with a combined clinicogenomic model. The comparison shows that the biomarker is significantly associated with the probability of having lung cancer in both the biomarker and clinicogenomic models and that the importance of each of the variables in the combined clinicogenomic model is similar to their importance in the initial uncombined models.
Given the independence of the biomarker and clinical models, it is not surprising that the clinicogenomic model is a better predictor of lung cancer than either of the initial models in an independent test set. ROC curve analysis shows that the clinicogenomic model performs significantly better than the clinical model. Furthermore, the clinicogenomic model increases the sensitivity, specificity, positive predictive value, and negative predictive value of the clinical model, and its accuracy does not seem to be influenced by the size or location of the lesion. However, these findings need to be validated in larger patient cohorts. One way to accomplish such validation would be to incorporate gene expression measurements into large epidemiologic studies investigating lung cancer risk or lung cancer screening trials involving high-risk smokers.
Despite the limitations of a small sample size and limited clinical parameters, we are encouraged that subjective clinical assessment based on a patient's complete medical record is associated with the clinical model probabilities. This is particularly important given that certain variables, such as positron emission tomography scan findings, were not included in the clinical model because these studies were done on only a small number of the subjects in our cohort. All available data, such as positron emission tomography scan findings, were, however, considered by the pulmonary physicians as part of their subjective assessment of lung cancer likelihood. Further, the clinicogenomic model seems to correctly classify patients assigned to the medium risk subgroup by the clinical subjective assessment. This subgroup of patients is one that is likely to be especially challenging to manage clinically as almost a third of these patients went on to have a final diagnosis of lung cancer.
Our data suggest that a clinicogenomic model that combines gene expression with clinical risk factors for lung cancer has high diagnostic specificity and positive predictive value among patients with nondiagnostic bronchoscopies, including those with small and/or peripheral lesions on chest imaging. This model might therefore serve to identify those patients who would benefit from further invasive testing (e.g., lung biopsy) to confirm the presumptive lung cancer diagnosis and thereby expedite the diagnosis and treatment for their underlying malignancy. In addition, the clinicogenomic model also results in modest increases in diagnostic sensitivity and negative predictive value. Utilization of this clinicogenomic diagnostic might therefore also result in a reduction in the number of individuals without lung cancer who are subjected to additional and more invasive procedures to rule out a lung cancer diagnosis following a nondiagnostic bronchoscopy. If the ultimate sensitivity and negative predictive value of the clinicogenomic model remains close to 100%, this would allow clinicians to confidently use less invasive and less costly approaches (e.g., repeat computed tomography scan in 3-6 months) to follow-up patients with a low clinicogenomic lung cancer risk score.
The ability of gene expression profiles within cytologically normal airway epithelium to serve as a biomarker for lung cancer raises questions about the underlying biology of the cancer-specific molecular changes observed in these cells. The high diagnostic accuracy for the biomarker in the setting of small peripheral lung lesions suggests that changes in airway gene expression between smokers with and without lung cancer are unlikely to be a direct effect of the tumor. The presence of antioxidant and inflammation-related genes in the gene expression biomarker (7) raises the possibility that the biomarker detects an airway-wide cancer-specific difference in response to tobacco smoke exposure. Given the hypothesis that this field of injury may provide information about the host-carcinogen interaction, alterations in gene expression could precede the development of lung cancer and explain the somewhat lower specificity of the biomarker relative to its sensitivity. If this is true, the biomarker might potentially be a useful tool to identify smokers at highest risk for disease who may benefit from chemopreventive strategies.
| Conclusion |
|---|
|
|
|---|
| Disclosure of Potential Conflicts of Interest |
|---|
|
|
|---|
| Acknowledgments |
|---|
|
|
|---|
| Footnotes |
|---|
Received for publication January 16, 2008.
Revision received February 20, 2008.
Accepted February 20, 2008
| References |
|---|
|
|
|---|
2. Shields PG. Molecular epidemiology of lung cancer. Ann Oncol 1999;10 Suppl 5:S7–11.
3. Hoffman PC, Mauer AM, Vokes EE. Lung cancer. Lancet 2000;355:479–85.[Medline]
4. Postmus PE. Bronchoscopy for lung cancer. Chest 2005;128:16–8.[CrossRef][Medline]
5. Mazzone P, Jain P, Arroliga AC, Matthay RA. Bronchoscopy and needle biopsy techniques for diagnosis and staging of lung cancer. Clin Chest Med 2002;23:137–58, ix.[CrossRef][Medline]
6. Schreiber G McCrory DC. Performance characteristics of different modalities for diagnosis of suspected lung cancer: summary of published evidence. Chest 2003;123:115–28S.[CrossRef]
7. Spira A, Beane JE, Shah V et al . Airway epithelial gene expression in the diagnostic evaluation of smokers with suspect lung cancer. Nat Med 2007;13:361–6.[CrossRef][Medline]
8. Pittman J, Huang E, Dressman H et al . Integrated modeling of clinical and gene expression information for personalized prediction of disease outcomes. Proc Natl Acad Sci U S A 2004;101:8431–6.
9. Li L. Survival prediction of diffuse large-B-cell lymphoma based on both clinical and gene expression information. Bioinformatics 2006;22:466–71.
10. Stephenson AJ, Smith A, Kattan MW et al . Integration of gene expression profiling and clinical variables to predict prostate carcinoma recurrence after radical prostatectomy. Cancer 2005;104:290–8.[CrossRef][Medline]
11. West M, Ginsburg GS, Huang AT, Nevins JR. Embracing the complexity of genomic data for personalized medicine. Genome Res 2006;16:559–66.
12. McWilliams A Lam S. New approaches to lung cancer prevention. Curr Oncol Rep 2002;4:487–94.[Medline]
13. Trunk G, Gracey DR, Byrd RB. The management and evaluation of the solitary pulmonary nodule. Chest 1974;66:236–9.[CrossRef][Medline]
14. Thurston SW, Liu G, Miller DP, Christiani DC. Modeling lung cancer risk in case-control studies using a new dose metric of smoking. Cancer Epidemiol Biomarkers Prev 2005;14:2296–302.
15. Gurney JW. Determining the likelihood of malignancy in solitary pulmonary nodules with Bayesian analysis. Part I. Theory. Radiology 1993;186:405–13.
16. Mannino DM, Aguayo SM, Petty TL, Redd SC. Low lung function and incident lung cancer in the United States: data From the First National Health and Nutrition Examination Survey follow-up. Arch Intern Med 2003;163:1475–80.
17. Wahidi MM, Govert JA, Goudar RK, Gould MK, McCrory DC. Evidence for the treatment of patients with pulmonary nodules: when is it lung cancer? ACCP evidence-based clinical practice guidelines (2nd ed.). Chest 2007;132:94–107S.[CrossRef]
18. Ung YC, Maziak DE, Vanderveen JA et al . 18Fluorodeoxyglucose positron emission tomography in the diagnosis and staging of lung cancer: a systematic review. J Natl Cancer Inst 2007.
19. Cummings SR, Lillington GA, Richard RJ. Estimating the probability of malignancy in solitary pulmonary nodules. A Bayesian approach. Am Rev Respir Dis 1986;134:449–52.[Medline]
20. Swensen SJ, Silverstein MD, Ilstrup DM, Schleck CD, Edell ES. The probability of malignancy in solitary pulmonary nodules. Application to small radiologically indeterminate nodules. Arch Intern Med 1997;157:849–55.[Abstract]
21. Swensen SJ, Silverstein MD, Edell ES et al . Solitary pulmonary nodules: clinical prediction model versus physicians. Mayo Clin Proc 1999;74:319–29.[Medline]
22. Spitz MR, Hong WK, Amos CI et al . A risk model for prediction of lung cancer. J Natl Cancer Inst 2007;99:715–26.
23. Golub TR, Slonim DK, Tamayo P et al . Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 1999;286:531–7.
24. Akaike H. A new look at the statistical model identification. IEEE Trans Automatic Control 1974;19:716–23.[CrossRef]
25. Hanley JA McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29–36.
26. Hanley JA McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148:839–43.
27. Sullivan PM, Etzioni R, Feng Z et al . Phases of biomarker development for early detection of cancer. J Natl Cancer Inst 2001;93:1054–61.
28. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37–46.[CrossRef]
Commentary
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |