"edge, rim," mid-15c., from Old French verge "twig, branch; measuring rod; penis; rod or wand of office" (12c.), hence, from the last sense, "scope, territory dominated" (as in estre suz la verge de "be under the authority of"), from Latin virga "shoot, rod, stick, slender green branch," of unknown origin.
Earliest attested sense in English is now-obsolete meaning "male member, penis" (c. 1400). Modern sense is from the notion of within the verge (c. 1500, also as Anglo-French dedeinz la verge), i.e. "subject to the Lord High Steward's authority" (as symbolized by the rod of office), originally a 12-mile radius round the king's court. Sense shifted to "the outermost edge of an expanse or area." Meaning "point at which something happens" (as in on the verge of) is first attested c. 1600. "A very curious sense development." [Weekley]