The area is known for tennis and beautiful beaches. If you drive a short ways to the north, you can visit the town of Fernandina Beach. There are lots of little touristy shops, restaurants, and book stores. It's a very walkable town and manages to stay quaint. They often have festivals for various things.
If you go to the South on A1A, you can take a ride across the St. Johns on the St. Johns River Ferry to the old shrimping village of Mayport. The village is slowly being destroyed to make way for cruise ship terminals and lots of modern stuff. However, a shack on the river called Singletons is a good place to eat shrimp or fish. You can also buy fresh off the boat shrimp cheap from the small processing plants in the area. The north end of the town has a boat ramp that has a big parking lot that is perfect for watching sea birds and all the shipping that comes in and out of Jacksonville. On the east side of town (it's about a 3 block town), you can see the Mayport Naval Station and light house through the chainlink fence around the base. It's an active place and you'll see all kind of ships, boats, helicopters, etc.
If you keep going south from Mayport you can go through the Jacksonville Beaches and then on down to St. Augustine. The Jacksonville Beaches have pizza places, ice cream shops, and all the things you need from a city. St. Augustine has a three attractions that kids might like -- The Castillo de San Marcos (a Spanish fortress), the St. Augustine Alligator Farm (where they might get to touch a snake or baby alligator), and Ripley's Believe It Or Not Museum.
If you go west from Fernandina Beach, you can go to the Okeefenokee Swamp and go canoeing or take a tour. I believe that Jacksonville also has a good planetarium and children's museum.