The deer are so awful here that I quite literally have to chase them out of the orchard. Soon, I expect they will begin trying to eat the groceries as we take them from the car to the house.
But other than a crazy buck attacking a small bare fig tree last fall (he likely thought it was another buck because of the antler like shape of the branches; I said he was crazy), we have never had deer eating the figs.
I suspect they don't like either the smell of the figs or the latex from the leaves and stems.
You are safe with the figs.