1540s, originally "person hired to remove refuse from streets," a modification of Middle English scavager, scawageour (late 14c.), the title of a London official who originally was charged with collecting tax on goods sold by foreign merchants.
This is from Middle English scavage, scauage (Anglo-French scawage) "toll or duty exacted by a local official on goods offered for sale in one's precinct" (c. 1400), from Old North French escauwage "inspection," from a Germanic source (compare Old High German scouwon, Old English sceawian "to look at, inspect;" see show (v.)).
The scavenger later was charged with inspection and maintenance of streets: Blount's description ("Glossographia," 1656) is "an Officer well known in London, that makes clean the streets, by scraping up and carrying away the dust and durt." The modern general sense of the word "one who collects and consumes or puts to use what has been discarded" evolved through the notion of "collect and dispose of rubbish."
The word came to be regarded as an agent noun in -er, but the verb scavenge (q.v.) is a late back-formation from the noun. For the unetymological -n- (c. 1500), compare harbinger, passenger, messenger, etc. Extended 1590s to animals that feed on decaying matter. Scavenger hunt is attested from 1937. Mayhew (1851) has scavagery "street-cleaning, removal of filth from streets."