Cancer is one of the primary causes of death worldwide, and its morbidity and mortality rates are increasing rapidly. However, standard treatment modalities (surgery, radiotherapy, chemotherapy, and immunotherapy) often fail to achieve a satisfactory therapeutic effect. Extracellular vesicles (EVs) are natural nano-sized lipid bilayer vesicles secreted from cells. Owing to their advantages of low toxicity, high biocompatibility, low immunogenicity, and inherent targeting, EVs can be exploited as drug delivery vectors for cancer treatment. In this review, we summarize the research progress of EV-based drug delivery systems in cancer treatment by focusing on four aspects: sources, cargo types, cargo loading methods and modification strategies. Finally, current challenges and future perspectives are discussed.