The Museum of Natural and Other Curiosities is a home to odd things and discoveries by painter Joseph Steward. In 1797, it was made opened to public and was located at the attic of the Old State House. In 1808, when the museum's popularity increased, Steward relocated the museum across the street. The collection of odd specimens continued until before his death in 1822.

Included in the collection is an eight-foot alligator, an albino woodland critters, a two-headed pig, a whale vertebrae, and a giant lobster's claw. Steward's original collection has a two-headed calf which was donated in 1996 from a dairy farm in Cheboygan, Michigan.

Dioramas of conventional exhibit includes rare butterflies, different insects, as well as exoskeletons of horseshoe crabs which were as old as the colonial age. Though Steward is more popularly known by his art pieces ? of his paintings and portraits, his museum has made a remarkable identity for him.

In 1880, the museum's admission cost 25 cents wherein Steward gave discounts to patrons of the place.