Last year I left two small figs on one of my small potted trees. They were bigger than fig embryos but not very large -- not nearly to the point where they reached the size where they were "stalled out" and just waiting to swell and ripen. They were much smaller than that. Anyway, one dried up and fell off over the winter, but one held on. It went on to become my first ripe fig of this season. It was excellent.
I'm sure the advice to knock off the big ones is spot on. If you have a couple of trees and want to experiment a little, you could try to leave some very, very small ones on.
I originally got the idea from this video (link below), which someone posted in a thread over 2 years ago. I didn't save the link at the time, but I just happened to come across it today!!!!
Cheers,
Jim