I lived in "Annapolis" (Downtown, Eastport, Cape St. Claire and Severna Park) for 33 years, then moved to the Eastern Shore. Best thing I've ever done moving over here.
It definitly depends on what you want in a city. I live in southern Carroll and love it. Its close to Ellicot City and Colombia but doesn't draw the bad crowd.