History of Shadowdale - Elminster Moves to Shadowdale:

It is of note that during the time that Doust Sulwood of the Knights of Myth Drannor assumed the role of Lord of Shadowdale, Elminster took up residence in the area. A semi-regular visitor up to that time, he took possession of a low, abandoned tower at the foot of the Old Skull, and declared himself to be officially in retirement. The nature of that retirement varies from active involvement in local affairs to long-term vacations on other planes. The natives of the Dale have come to the understanding that they cannot always count on the powerful mage being in residence in times of need or danger, but when he is present in these circumstances his aid is usually given.