Anson Phelps Stokes, mine developer, railroad magnate and member of a prominent eastern family, built Stokes Castle as a summer home for his sons. After the castle (or the tower, as the Stokes family always referred to it) was completed, in June 1897, the Stokes family used it for two months. Since then, with one possible exception, the structure has remained unoccupied.

Stokes castle is made of granite. The huge stones, raised with a hand winch and held in position by rock wedging and clay mortar. The architectural model for the castle was a medieval tower Anson Stokes had seen and admired near Rome. This building originally had three floors, each with a fireplace, plate glass view windows, balconies on the second and third floors, and a battlemented terrace on the roof.  It had plumbing and sumptuously furnishings.

The structure stands as an abiding monument to the men who built it and to those who helped develop Austin.