The Whitehall Museum House is the site of the Whitehall estate, originally owned by George Berkeley and his family in 1729. It was Berkeley?s home when his kids Henry and Lucia were born. It was also where he started his plans on starting St. Paul?s College in Bermuda.

Berkeley moved out in 1731 and donated the Whitehall to Yale University, along with his library. It was then sold and was turned into an inn. This didn?t affect the agreement of Berkeley?s donation to the premier university, since he stipulated that the income of the property should be used to support three Yale scholars.

By 1899, the National Society of the Colonial Dames of America acquired the house through a 999-year lease. They now operate it as a museum while maintaining the whole property, including its garden. It also provides accommodation for scholars who focused their studies on Berkeley.