Exosomes are formally defined as extracellular vesicles, which are formed in compartments with endosomal origin by the inward budding of the endosomal limiting membrane. Recent analyses of the dynamic events within exosome-generating compartments have overturned the dogma that only late endosomal membranes produce exosomes. It is now clear that recycling endosomal, autophagic, regulated secretory, and other organelle membranes contribute to exosome production. In this opinion article, we discuss studies demonstrating the critical roles of membrane origins and mergers, together with intracompartmental microenvironments, in generating intraluminal vesicle and exosome subtypes with diverse physiological and pathological functions, both inside and outside the secreting cell. These findings provide significant opportunities to develop novel strategies that overcome the current challenges of detecting and targeting disease-relevant exosomes.