pony n.

1650s, powny, "a very small horse" (less than 13 hands in height), from Scottish, apparently from obsolete French poulenet "little foal" (mid-15c.), diminutive of Old French poulain "foal," from Late Latin pullanus "young of an animal," from Latin pullus "young of a horse, fowl, etc." (from PIE root *pau- (1) "few, little") [Skeat's suggestion, still accepted]. Compare, from the same source, foal, filly, Sanskrit potah "a young animal," Greek pōlos "foal," secondarily also of other young animals; Latin pullus "young animal," Lithuanian putytis "young animal, young bird."

A small horse, especially one of a small breed, as opposed to a colt or filly, words which indicate merely young horses. German, sensibly, indicates this animal by attaching a diminutive suffix to its word for "horse," which might yield Modern English *horslet. Modern French poney is a 19c. borrowing from English.

The Shetland breed of ponies are stoutly built, active and hardy, with very full mane and tail, and of gentle, docile disposition. In western parts of the United States all the small hardy horses (mustangs or broncos) used by the Indians are called ponies. [Century Dictionary, 1897] 

Meaning "crib of a text as a cheating aid," especially a translation of a Greek or Latin author used unfairly in the preparation of lessons (1827) and "small liquor glass" (1849) both are from notion of "smallness" (the former also "something one rides," a translation being something that enables a student to "get along fast").

As the name of a popular dance, it dates from 1963. The U.S. Pony Express began 1860 (and operated about 18 months before being superseded by the transcontinental telegraph). The figurative one-trick pony is 1897, American English, in reference to circus acts.

1824, in pony up "to pay," of uncertain origin; similar uses of pony or poney in the sense "money" date to late 18c. OED says from pony (n.), but not exactly how. "Dictionary of American Slang" says it is from slang use of Latin legem pone (q.v.) to mean "money" (first recorded 16c.), because this was the title of the Psalm for March 25, a Quarter Day and the first payday of the year. Latin pone is the imperative of Latin ponere "to put, place" (see position). Related: Ponied; ponying.