Non-small cell lung cancer (NSCLC) is a leading cause of death worldwide. The upregulation of the epidermal growth factor receptor (EGFR) due to mutations has been observed in a number of cancers, and tyrosine kinase inhibitors (TKIs), such as gefitinib and erlotinib, which specifically target EGFR signaling, have been used to treat NSCLC patients. The presence of EGFR mutations was previously shown to confer sensitivity to TKIs. In this study, we evaluated the correlation between EGFR mutations and response to erlotinib in Chinese NSCLC patients. We recruited 36 patients with stage IIIB/IV NSCLC who had failed first-line chemotherapy, and treated them with erlotinib. We used immunohistochemistry to determine EGFR expression, and we screened for mutations using PCR analysis. We used Cox regression analysis and Kaplan-Meier curves for survival analysis. We found that 8 patients had exon 19 mutations, while 3 patients had exon 21 mutations. An Eastern Cooperative Oncology Group (ECOG) grade of 2 was a significant negative predictor of overall survival (OS). Patients with EGFR mutations showed a significantly better OS compared to those without EGFR mutations. Additionally, multivariate analysis showed that erlotinib-treated stage IV patients had a significantly longer progression-free survival (PFS) compared to stage IIIB patients. Patients with EGFR mutations also had a significantly better PFS compared to those without EGFR mutations. The overall remission rate (22.2%) and disease control rate (75%) were significantly higher compared to the rates after second-line chemotherapy (<10%). In conclusion, the presence of EGFR mutations could be a marker to predict the therapeutic efficacy of erlotinib and the prognosis in Chinese NSCLC patients.