My opinion is the tree is not old enough to keep the fruits.
I had 50 fruits on my 6 years old and 30 developed normal while 20 never grew but stay very small and incipient turned blackish just like yours and fell out on 1st of May.
Of course another cause could be that the embryos were damaged during winter,and they fail to develop later,and fell out.
Last Year it had 10 fruits only 4 grew and 6 fell out just like yours.
So it is the strength ,of the tree that makes it drop or not,in one instance or the Embryos were partly damaged by frost in the winter and fell off later,in the second instance..
And in pot is not easy to keep a tree in maximum strength shape.
That is why Adriano keeps his Desert King in ground in Canada.
Of course it is near to impossible to keep the Embryos alive,trough the winter,on an inground tree in Canada, but :::HE DOES.
Here is my Desert King,6yrs old in pot .
It has fruits in stagnant stage the size of a golf ball now.
This is in my opinion the best Breba fig there is,but:
All breba figs are difficult to fruit in cold climates.
That is why I only grow it as an experiment and not a base fig.
My base figs are early main crop figs.