William of Wykeham, Bishop of Winchester and Chancellor of England bought the house in 1377. It then passed to William?s great-nephew Sir Thomas Wykeham and thence to Sir Thomas?s granddaughter, Margaret, who married Sir William Fiennes, later the 2nd Lord Saye & Sele, in 1448.
Sir Thomas Wykeham obtained a licence to ?crenellate and embattle? in 1406: he added the battlemented wall to the gatehouse, thus giving the medieval house a military appearance ? these changes allowed the manor house to be called a castle.
In 1554 Richard Fiennes completed a major reconstruction. He raised the roof to accommodate two floors above the Great Hall, building two staircase projections to the south and adding ? on the foundations of the medieval kitchens ? two rooms which form the west wing. After his death in 1573 his son, Richard, continued the embellishment of the interior, recording the date 1599 on the plaster ceiling in the Great Parlour.