Middle English yelwe, from Old English geolu, geolwe, "yellow," from Proto-Germanic *gelwaz (source also of Old Saxon, Old High German gelo, Middle Dutch ghele, Dutch geel, Middle High German gel, German gelb, Old Norse gulr, Swedish gul "yellow"), from PIE root *ghel- (2) "to shine," with derivatives denoting "green" and "yellow" (such as Greek khlōros "greenish-yellow," Latin helvus "yellowish, bay").
In Middle English it also was used of a color closer to blue-gray or gray, of frogs or hazel eyes, and as a translation of Latin caeruleus or glauco. The meaning "light-skinned" (in reference to black persons) is recorded by 1808. It was applied to Asiatics by 1787, though that first reference is to Turkish words for inhabitants of India.
Yellow peril translates German die gelbe gefahr. The sense of "cowardly" is by 1856, of unknown origin; the color was traditionally associated rather with jealousy and envy (17c.). Yellow-bellied "cowardly" is from 1924, probably a semi-rhyming reduplication of yellow; earlier yellow-belly was a sailor's name for a half-caste (1867) and a Texas term for Mexican soldiers (1842, based on the color of their uniforms). Yellow dog "mongrel" is attested from c. 1770; the slang sense of "contemptible person" is recorded by 1881. Yellow fever is attested from 1748, American English (jaundice is a symptom).