By 1474, the castle in Żelowice had already been erected and belonged to the von Tschesch family. After 1706, the estate was in possession of the Weighardt von Hoffmann family. The family's representative, count Franz Ludwig Xaver had planned to found a Cistercian monastery here. For this purpose, he began to reconstruct the building. As a result, the Renaissance ground floor acquired a second storey, and the entire castle assumed a Baroque form. Another reconstruction place in 1858. The building was extended by a third ornamental story. After WWII, the castle's interior was adapted to serve residential functions.