1520s, "briefly, in few words," from summary + -ly (2). Meaning "without hesitation or formality" is from 1620s.
Entries linking to summarily
summary adj.
early 15c., "brief, abbreviated; containing the sum or substance only," from Medieval Latin summarius "of or pertaining to the sum or substance," from Latin summa "whole, totality, gist" (see sum (n.)). Compare Latin phrase ad summam "on the whole, generally, in short." Sense of "done promptly, performed without hesitation or formality" is from 1713.
-ly 2
common adverbial suffix, forming from adjectives adverbs signifying "in a manner denoted by" the adjective, Middle English, from Old English -lice, from Proto-Germanic *-liko- (cognates: Old Frisian -like, Old Saxon -liko, Dutch -lijk, Old High German -licho, German -lich, Old Norse -liga, Gothic -leiko); see -ly (1). Cognate with lich, and identical with like (adj.).
Weekley notes as "curious" that Germanic uses a word essentially meaning "body" for the adverbial formation, while Romanic uses one meaning "mind" (as in French constamment from Latin constanti mente). The modern English form emerged in late Middle English, probably from influence of Old Norse -liga.