No, you can only destroy the weapons and buildings blocks of the castles, like walls, roofs, bridges, treasure chests and the throne room. You cannot destroy the characters, animals and decoration items. However, the figures disappear when their weapon is destroyed.