The Pusey-Brown House serves as a community landmark since 1825. First called the Hill House, it served as a hotel, home, library, boarding house, and a community gathering place. This structure, which gives out a regal aura, can remind anyone of the events that have taken place over the years.

The house, which was dedicated in 1922, was built by John Y. Hill, who arrived in Kentucky before opening a tailoring shop. As a result of boredom, he started building houses and by the time he died in 1859, Hill had already constructed a college, a church, and a lot of other structures in Elizabethtown.

After the ownership of several future Hill generations, the property was leased and operated as the Hill House until 1890. In 1895, it was purchased by A. D. McCann, who sold it to A. J. Bush in 1913. The next owner was J. M. Smith who finally sold it to the Pusey Brothers in 1922. The Puseys bought it to fulfil a dream of establishing a community house in the city.

Today, it houses a library, a museum, and a historical theatre, which encourages the participation in certain community events and activities that help establish a sense of community among the locals.