The first documentary mention of the castle of Elmarshausen goes back to the year 1123. The "Lords of Helfenberg" and the "Lords of Gudenberg" began the construction of a castle that then became the possession of Hermann von der Malsburg. He started the reconstruction and building of the castle into the current moated castle, which was completed by his son, Christoph von der Malsburg, in 1563. The original round arch endpieces of the gable were rebuilt to flat triangular gables around 1800, causing the architectural style today to appear mixed. After reconstruction and modernisation work, today the castle is still inhabited by the von der Malsburg family and has a well-known Trakehner horse breeding enterprise.
The moated castle can only be visited from the outside.