The area of Wooster was first settled by Alabamans Nathan and Elizabeth Pearson in 1869. The couple purchased just east of the present town after purchasing a section of land on January 10, 1870, first owned under an 1820 patent by War of 1812 veteran Robert Robertson. A store owned by W. E. Adams was sold to J. P. Wooster on October 27, 1881, and the store became both the namesake of the town and its community center. A U.S. Post Office opened on March 20, 1883, under postmaster J. H. Smith. By the 1910s the town had grown to encompass the post office, three stores, a blacksmith shop, doctor's office, and Masonic building, but no church or school. A Baptist church joined the community in 1917.