From the Super Mario Wiki
Gramps is Toad's paternal grandfather, and therefore the father of Moldy, Toad's father. Like the rest of Toad's family, he hails from Fungusville. Little is known about his past, except that he was sentenced to serve a sentence of unknown length at Koopatraz State Prison over fifty-four years ago. Gramps likely had Moldy prior to his sentence, implying that he is at least in his seventies, if not older.
Fifty-four years ago, Gramps was sentenced to Koopatraz State Prison, most likely by either King Koopa's Father or his grandfather. It is probable that Gramps did nothing morally wrong, and was deliberately sentenced to Koopatraz out of sheer cruelty, as is the standard for its other prisoners. After twenty-four years, Gramps came to be Koopatraz's only prisoner for a long time.
Thirty years later, Gramps was introduced to his new cellmates, Mario, Luigi and Toad. Upon hearing Toad's name, Gramps asked if he was related to the Fungusville Toads, which led to them discovering that they were grandfather and grandson, and they finally got to hug each other for the first time.
Mario eventually engineered an escape plan from Koopatraz, and after receiving Gramps' advice, he smuggled a few forks out of the mess hall, which the male prisoners used to dig out of their cell with. Unfortunately, this plan led to Mario getting caught by Warden Koopa, who sentenced them all to hard labor. However, Mario was able to come up with another escape plan, which involved fooling a Goomba, whose keys he stole to rescue Gramps and the other prisoners.
Gramps and the other prioners escaped from the guards by going down a storm drain, where they pulled out a drain stopper underneath Koopatraz Island, slowly causing it to sink. They escaped from the sinking island in a boat, where Gramps announced that he was finally a free Mushroom again. Mario asked if he was free to buy them all dinner, and Gramps responded by showing him the steel cup and loaf of bread he'd smuggled out of the prison mess hall beforehand.
Although Gramps was never seen or heard from again, it's likely that he returned to Fungusville to live a more peaceful life.