Hop City, while a great beer store, has a moderate homebrew selection. They do beer tastings and sell the used bottles from those tastings for real cheap (especially if you catch them before they've removed the labels). It's in midtown next door to the 5 Seasons.
If you're in the Alpharetta area,
Beer Necessities has a great selection of both equipment and ingredients. They've always had every ingredient I needed on hand.
Where else is there around town?
Threaded Mode