Born August 8, 1837, on Oakington farm, in Harford County, Maryland, Stump was admitted to the Maryland Bar in 1856. He practiced law in Bel Air, Maryland, and was elected to the State Senate in 1878. Stump was elected to Congress as a Democrat in 1889, and served in the House of Representatives until 1893. In Congress, Stump was a member of the Immigration Committee responsible for drafting the Immigration Act of 1891, which created the US Immigration Service. President Grover Cleveland appointed Stump to serve as Superintendent of the Office of Immigration, then in the Department of Treasury, in 1892. Congress acted in 1895 to elevate the office to the Bureau of Immigration in 1895, and Stump became Commissioner of Immigration in that year. He served until 1897, when he returned to Bel Air and the practice of law. He died at his home, Waverly, on January 9, 1917.