With the death of John de Somery in 1322 the extensive estates of the Barony of Dudley entered into a period of confusion and disputed ownership.
Only in 1327 did John de Sutton, after imprisonment in the Tower of London and the extraction of a forced disclaimer to the property, eventually
inherit the family estates of his wife Margaret de Somery.
The Castle had a relatively calm succession of the de Sutton lords, all name John.