Target, for example, has an actual scrapbook section, but if you go a few aisles over to the card section, you will find some cute epoxy stickers...that always seem to be on clearance. You can also get paper clips in a variety of shapes and sizes in the school supplies aisle:
The options are obviously limited, but whose not for saving a little money every now and then!
Wal-Mart, too, has a good selection of scrapbooking basics and some of the best prices we've seen. EK Success Adornaments Scrapbook Fibers are only $5.00 and Vario Tab Refills are less than $6.00 - both $1.00 plus less than what you'll find at most stores.