From the Super Mario Wiki
Save Blocks first appear in Super Mario RPG: Legend of the Seven Stars. When Mario jumps on one, the block takes him to the save screen, allowing the player to save their game. These blocks are scattered all over the Mushroom World. Additionally, Save Blocks can always be found inside inns..
Save Blocks are seen again in the Paper Mario series. This time, instead of jumping on top, they need to be hit from underneath like regular blocks. They also have a vastly different appearance, being rainbow-colored blocks labeled "S" with a star inside rather than the Super Mario Bros.-style blocks with a star on top seen in Super Mario RPG. The rainbow coloring is later used in Partners in Time by the Aurora Block.
In the first two installments of the Mario & Luigi series, Save Blocks are replaced by Save Albums; however, Save Blocks are present in most of the locations of Mario & Luigi: Bowser's Inside Story and Mario & Luigi: Dream Team, also in the form of yellow Emoglobins. In the latter (game), Mario and Luigi have to hit the blocks from below, while in the former, Bowser has to hit the blocks by punching them.
Names in Other Languages