I asked the same question not too long ago.
I think the consensus has been to remove them once they are about the size of a BB just to make sure its a fig you are pulling off and not a bud.
I have been doing that with this years batch and I have noticed once I removed the small figs, the cuttings put forth more buds and leaf growth (which my gut tells me is good).
Good luck