We've all heard and possibly have used this phrase when referring to the North and South of the United States. However, we've come a long way from what or rather how this phrase came into use. It's from an old surveyors map, one produced by Charles Mason and Jeremiah Dixon from 1763-1767. Below you can see a short explanation of it from the 1884 copy of Houghtaling's Handbook of Useful Information, also I have a link to The History of Mason & Dixon's Line ©1855 for even more information.
Mason and Dixon's Line.
A name given to the southern boundary line of the Free State of Pennsylvania which formerly separated it from the Slave States of Maryland and Virginia, It was run—with the exception of about twenty-two miles—by Charles Mason and Jei emiah Dixon, two English mathematicians and surveyors, between Nov. 15, 1763, and Dec. 26, 1767. During the excited debase in Congress, in 1820, on the question of excluding slavery from Missouri, the eccentric John Randolph of Roanoke made great use of this phrase, which was caught up and re-echoed by every newspaper in the land, and thus gained a celebrity which it still retains.