Other sources seem to suggest there is no surviving evidence of the medieval Castleton castle other than the mound it once stood one.
I’m sure an architectural historian could point out the error of my ways immedaitely, but i’ve always though this door lintel adjacent to the site looks like its re-used from something much older.
I agree, looks like something salvaged from a more important building