Nestled against the tightrope neighborhood of Basse-Eau, the Castle has its feet firmly on the ground and a robust and martial architecture. The shape of the buildings overlooking the Pearl Guardhouse, between the Castle Ladder and the Ladder of the Pearl, certainly recalls that in other times, a fortification stood here. Some legends say that a citadel watched over the neighborhood. Others prefer to attribute this layout to a former palace belonging to a merchant who, bankrupt, divided his property into many lots. The now disappeared fortress still survives in the name of the castle and in the military shadow it casts over the Upper Gardens in the setting sun.

Despite this austerity, these buildings attract restaurateurs who see excellent banquet halls and exploit the countless vaulted cellars both for storing provisions safely and for discreetly receiving mysterious clients. The Castle thus attracts an astonishing mix of high society and… the underworld.