A native of Dundas, Ontario, Joseph Spencer graduated from McGill in Applied Science in 1874. After postgraduate studies in geology and mineralogy at Gottenberg University (M.A., Ph.D. 1877) and King's College, Halifax (M.A. 1880), Spencer taught at Hamilton Collegiate Institute (1877-1879) and King's College (1880-1882). From 1882 to 1887, he was Professor of geology and mineralogy as well as director of the Museum of Geology and Natural History at the University of Missouri. In 1888 he moved to the University of Georgia, and from 1890 to 1893 combined teaching duties with those of State Geologist. The later part of his career was devoted to independent research.