Elgin Castle was once a Royal Castle and was built as a motte and bailey castle in the 12th century. The castle ha been a ruin since the 15th century and very little remains today. But the location still gives a commanding view over the town of Elgin. Map location.