I personally wouldn't let it stay on. That's just me. No hard wood, I'm guessing the root system isn't that well established. Fruit takes a lot of energy to completely form. There's a solid chance that fruit will never even form, or it will rot, or it will be small and totally lacking flavor. The chances you will get a nasty piece of fruit are huge; this means that your chances of allowing your plant to waste a lot of energy are high.
I would pluck it off. Odds of anything beneficial happening are miniscule, odds of detriment are very high.