The Esbensen residence on Bakken is located in Indrebyen or Inner Kvænby, on the western edge of the Vadsø town centre. This area is characterised by villas from the 20th century. It lies just outside of the town centre area that was bombed and burned in 1944. It is a lovely detailed building in the Swiss style. The main façade is south facing and the entrance is east facing, overlooking the town centre. The terrain slopes downwards from the property towards the sea, and thus provides the site with a view that overlooks the fjord and the harbour. Encircling the house is a garden with a picket fence.

Originally the house was built by the merchant family Nordvi, as were many of the buildings at the trading settlement Mortensnes, further in on the Varangerfjord. Andreas Esbensen bought the site in 1877 and the main building was moved to Vadsø in 1897. The foundation wall still stands at Mortensnes and is a part of the listed cultural monument there.