When Madison retired from politics in 1817, he went back to his plantation home in Virginia. A former slave of Madison's, Paul Jennings, wrote the first White House Memoir. After the death of Thomas Jefferson, Madison was appointed to Jefferson's spot at the University of Virginia. A few years before Madison died, he sold around 25% of his slaves to try to get money to pay for the plantation. He later died on June 28, 1836 at his plantation home. Dolly Madison later sold the plantation. To the right is James Madison's gravestone at the Madison Family Cemetery.