Lindon was the last mainland remnant of Beleriand following the War of Wrath at the end of the First Age, the rest of the land having been broken or submerged by the tumults. While not exactly clear, there are indications that before the Downfall of the Noldor in the Second Age Lindon was larger than it was during the Third Age.
During the Fourth Age, it was one of the last Elven havens as the remaining Elves of Rivendell and Lothl�n left Middle-earth. In the beginning of the first century, Fourth Age, it experienced a population growth as migrants from the east came to Mithlond. Not all Elves left Middle-earth immediately, many of the migrants made long-term temporary settlements.
Aside from Elves, Gandalf, Bilbo Baggins and Frodo Baggins also went to Valinor from the Grey Havens, and a family tradition held that Samwise Gamgee, having been himself a Ring-bearer, albeit briefly, did likewise, in the year 1484 of the Shire Reckoning, Fourth Age 61.