Hi jdsfrance
It doesn't need the wasp, but yes for me as well it is prone to drop figs, however the ones it holds on to are good.
I have a second year plant, which was a cutting from my original Rouge and this is also ripening a few figs now, so it fruits early pot grown.
Yours should benefit from a sunnier spot so probably worth another chance.