There is an ongoing debate on the declining semen quality, and unfortunately, existing evidence is inconclusive and inconsistence. We evaluated the impact of sociodemographic characteristics, lifestyle, medical history and work exposure on semen quality. Univariate and multivariate analysis was used to investigate the association between different risk factors and semen quality parameters. Total sperm count (p = 0.041), sperm concentration (p = 0.007), normal morphology (p = 0.002), total motility (p = 0.004) and progressive motility (p = 0.009) decreased in men with varicocele. Sperm concentration increased in tea (p = 0.044); progressive and total motility increased in cola (p = 0.018, p = 0.012) consumers. Progressive and total motility decreased in urogenital surgery (p = 0.016, p = 0.014) and infection (p = 0.037, p = 0.022). However, age, coffee and alcohol drinking, physical activities, sleep duration and cell phone use were unrelated to any of semen parameters. Interestingly, semen volume (p < 0.0001), total sperm count (p < 0.0001) and concentration (p < 0.033) increased with longer abstinence period (>5 days); normal morphology (p = 0.013) improved in men with higher body mass index (BMI > 24), curvilinear velocity (p = 0.042) increased with smoking; semen volume (p = 0.050) increased in manual labourers. This study highlights the importance of sociodemographic characteristics, lifestyle, occupational exposure and medical history and provides time trends in semen quality, its clinical importance and direction for further research.