A sound found chiefly in words of Old English, Old Norse or Greek origin, unpronounceable by Normans and many other Europeans. In Greek, the sound corresponds etymologically to Sanskrit -dh- and English -d-; and it was represented graphically by -TH- and at first pronounced as a true aspirate (as still in English outhouse, shithead, etc.).
But by 2c. B.C.E. the Greek letter theta was in universal use and had the modern "-th-" sound. Latin had neither the letter nor the sound, however, and the Romans represented Greek theta by -TH-, which they generally pronounced, at least in Late Latin, as simple "-t-" (passed down to Romanic languages, as in Spanish termal "thermal," teoria "theory," teatro "theater").
In Germanic languages it represents PIE *-t- and was common at the start of words or after stressed vowels. To represent it, Old English and Old Norse used the characters ð "eth" (a modified form of -d-) and þ "thorn," which originally was a rune. Old English, unlike Old Norse, seems never to have standardized which of the two versions of the sound ("hard" and "soft") was represented by which of the two letters.
The digraph -th- sometimes appears in early Old English, on the Roman model, and it returned in Middle English with the French scribes, driving out eth by c. 1250, but thorn persisted, especially in demonstratives (þat, þe, þis, etc.), even as other words were being spelled with -th-. The advent of printing dealt its death-blow, however, as types were imported from continental founders, who had no thorn. For a time y was used in its place (especially in Scotland), because it had a similar shape, hence ye for the in historical tourist trap Ye Olde _______ Shoppe (it never was pronounced "ye," only spelled that way).
The awareness that some Latin words in t- were from Greek th- encouraged over-correction in English and created unetymological forms such as Thames and author, while some words borrowed from Romanic languages preserve, on the Roman model, the Greek -th- spelling but the simple Latin "t" pronunciation (as in Thomas and thyme).