All of our letters were based on a single sheet of construction paper, so they were close to the same size. I had some poster board in the house, and found that they fit well on the small pieces. I then discovered that the small pieces were exactly a quarter of a full sheet. When I added a front and back cover, I had exactly enough for the pages. I made the covers out of the the card board on the pack of a pad of Melissa and Doug paper. To make them prettier, I lined the inside covers with paintings I did while the kids painted to use up the paint! And, for the record, I used about four glue sticks to stick them all down, and Mike broke the hole punch on the front cover.
Here is Natalie's finished alphabet book: