I look for about 38 as a night time low. That gives me some room for forecast error. It's the freeze point I'm worried about, anything above that is all good. If it's not a whole lot above freezing they are not going to grow anyway. To me, if they have broken bud, I want the leaves opening in sunlight so I don't have to deal with stretching and leaf burn/drop later. So, a couple of cold crappy days they will go in the garage, it gets nice they go out. Night time forecasts are below 38 I shuffle in and back out in the morning. I have often seen night time forecasts of 36 (in spring and fall)which thankfully I brought them in, and in the morning when I left for work there was frost.
When the forecast is right around 38 I like to go outside before bed and do the breath test. If I can see even a faint whisp of my breath(which you can see before it is actually below freezing in the mid 30's) I don't trust the forecast and I bring them in.
Of course, another factor is how many trees you have. When you have less than 30 trees, it takes 15 minutes or less, so shuffle moring and night is no big deal.