Grey Towers was built between 1865 and 1867 for the ironmaster William Randolph
Hopkins, the designer was John Ross of Darlington. Its unsual colour is due to the whinstone used in its construction.

Hopkins, Gilkes and Co were ruined due to their part in the Tay Bridge Disaster and the house went on the market in 1879 for £30,000 but remained unsold.
It became the home of Sir Arthur Dorman, founder of Dorman Long between 1895-1931. After Dormans death the house was bought by Sir Thomas Gibson Poole as the site for Middlesbroughs tuberculosis sanatorium, however due to the 2nd World War it was not completed until 1945.

After the closure of the Poole Hospital in 1989 the house became a target for vandals but has since been converted into apartments