weijunwang
I don't believe you can predict reliably by the fall colour of leaves to determine which plant need protection and which one does not need.
If you have two similar plants, ported the same way, but one is watered less in the fall and the other is pampered then the less watered one may shed its leaves earlier compared to the other.
If you have space and get temperatures below freezing then protect them all.

weijunwang, sorry but you can not.  I have different varieties and some are more sensitive than others.  You and I are in the same zone.  Just wait till all the leaves fall off, then store them in your garage during winter.  keep the soil moist and don't let them dry out and don't get them soaking wet either.  Your trees need to go through a chill period in order to produce next year.  cheers,

Best to let the trees go thru a dormant(chilled) period.

I have kept my potted trees in an unheated upstairs room and the temp never drops below freezing, probably stays around 50. My trees stay dormant until early spring and then they all struggle to get enough light. Your trees would get plenty of light in a sunroom. I have had some problems with spider mite before I get them back outside. Easy to fix with soap and water but a mess in the house...

Thanks for all the input.  I will put my small potted trees in the sun room for the winter.  The larger ones can go below the sun room (originally my back deck )at the back of my house, where 5 sides are shielded.  It will be pretty dark without direct sun light all day but wamer then other places outside.