Stamps are collected throughout the game by completing certain tasks or doing jobs for the worlds inhabitants. There are 100 stamps in total to collect. Once all 100 are collected you can present them to YumYum in Cloud Hill who will reward you with the Devil Parts; Devil Engine, Devil Tires and Devil Meter.
Some stamps are easy to do and can be achieved without really trying, such as visiting all the houses in a each city, others can take a lot more work to achieve such as 'Gave a Magazine!', trust me, its harder than it sounds, or 'Became the President!'