Paper and weight
A Lately magazine is US Letter with a matte cover and pages, thin enough to read in one sitting and pass around. A photo book is usually hardcover on heavy stock, built to sit on a shelf and come out for occasions.

Both put your photographs on paper. They differ in weight, in who does the designing, in how long they take, and in how often you will actually make one. Here is the honest comparison.

A Lately magazine is US Letter with a matte cover and pages, thin enough to read in one sitting and pass around. A photo book is usually hardcover on heavy stock, built to sit on a shelf and come out for occasions.
A Lately designer edits the issue; you pick the photos and add a caption where it helps. A photo book is designed by you in a template editor, one page at a time.
A magazine takes minutes to start and holds 15 to 45 photos, from $33 with standard US shipping included. A photo book often takes an evening or a weekend, holds a hundred or more photos, and varies widely in price.
A magazine can be monthly, seasonal, or one at a time, and is easy to make again next month. A photo book is one project, one volume, usually for one milestone.
Choose a magazine for an ongoing record, when you would rather pick photographs than lay out pages, or when everyone in it should get a copy. Choose a photo book for one formal milestone you want to design yourself. Or both: the wedding gets the album; the year around it gets the magazines.