The town is called San Simeon. The house was built by William Randolph Hearst, the newspaper magnate (and grandfather of the infamous Patty Hearst). The house is owned by the state of California since it was donated by the Hearst Corporation in 1957. The name given to the property by W.R. Hearst was La Cuesta Encantada ("The Enchanted Slope"). It was a well-known retreat for the Hollywood and political elite of the 1920 and 1930s.
If you are going to visit, there is a nice little motel 3 miles south of the gate that my family stays at all the time when we are in the area. It's not a Hilton but it is nice, clean, and inexpensive. TV and cell phone service is very sketchy in the area. Courtesy Inn of San Simeon
And make sure to stop in for some of the best BBQ ribs in California at the Main Street Grill in Cambria just a few miles farther south!