The best thing Atlanta has to offer is unbelievable food and drink. Other than that, our tourism is pretty crap. Personally, I'd skip the tourist mile and go to The High Museum or Botanical Gardens, but those definitely aren't for everyone.
If you're in to craft brew, Brick Store Pub is usually rated #1 or #2 in the nation.
As far as cool things, centinial olympic park is kinda cool, and in the middle of the World of Coke, the CNN Center, Phillips Arena, the GA Dome, and pretty close to the Georgia Aquarium. All of which are pretty neat places to see.
I have to concur...most bang for your buck. Stay downtown...the CNN center, Georgia Aquarium, World of Coke, Underground Atlanta are all within walking distance of each other...
Atlanta Botanical Gardens is a must. Stone Mountain, CNN Center, drinks or snacks at the top of the Westin (the tall round hotel in the center of Atlanta skyline), Georgia Aquarium if its cold out, the Cyclorama next to the Atlanta Zoo if you are a Civil War or history buff, Phipps Plaza and/or Lenox Mall for those that like to shop or just people watch, The Fabulous Fox Theater is a great place to see a band or a play or a musical or just about anything...as well as the Tabernacle for music.
Helen, GA is as touristy as you can get and Christmas is the best time to visit... well Octoberfest is pretty cool too.
That's a little out of range for a day trip to Atlanta, but the area surrounding Helen can be really nice. Some friends got married at a little vineyard nearby, and the view was incredible. The wine was good, too
If you go to Helen in the summer, do the rafting thing down the river. It's $5 or $10 but very relaxing once you get used to the water.