The building is now a ruin, towering over the east end of the village. The building of the castle began in 1121 to protect the lawless border areas and over the years changed hands many times. It stands on the south bank of the River Tweed, high above the river, so that the north side is protected by a steep slope leading down to the river. A deep ravine protected the east side and an artificial moat was dug round the west and south sides to complete the protection. The castle had an inner and outer ward. The inner ward stood on a mound and was separated from the outer ward by a moat, crossed by a drawbridge.
The main entrance to the castle was the strongly fortified West Gate leading into the outer ward. There was an additional gate to the south of the outer ward, known as the Sheep Gate.
The inner ward was entered by crossing a drawbridge across the moat and entering through a fortified gate on the west side. The drawbridge has now been replaced by a wooden bridge. On the north side of the inner ward was the bishop’s hall, measuring 60 ft by 30 ft (18.3m by 9.1m), now like the rest of the castle in ruins. To the east side of the inner ward stands the keep, measuring 84 ft by 60 ft (25.6m by 18.3m) and 88 feet (26.8 m) high. The keep is said to have been built by Hugh de Puiset.
It played a major role in the many cross border conflicts until after the Battle of Flodden in 1513 and the eventual union of the crowns resulted in it eventually falling into disrepair.
Norham Castle is now in the care of English Heritage and is open to visitors. It is a subject of many painting especially those of JMW Turner which can be seen in the Tate Gallery in London.