Aaron, the good thing about a box made of that stuff is that it will protect the plants from temperature spikes too, it'll warm up and cool down at a slower rate than the surrounding area.
I used this stuff to make my rooting 'hot boxes' and the ballasts from the lights keep them at a balmy 70-75 degrees, even though my unheated basement gets down to the 40's. In the morning, when the lights have been off for 6 to 8 hours, the potting mix is still in the 60's.
It does get stuffy in there, with the moisture from the potting mix rising out of the pots.
Good for rooting, and I doubt you'd have to air it out much, since your stored plants would be kept at a lower temperature, and with a lot less moisture.
How big will you build it (them)?
They will bulge if they're packed with plants, are you considering building a frame to stiffen the walls?