Hi don_sanders,
You can do it now; I've done it to suckers that I didn't want to keep.
I potted some and most are doing good. The others will likely get back from the dirt - I didn't want to keep them anyway. So just playing.
I wouldn't use a shovel as you may damage the tree of your ( soon to be former unless) friend.
Simply grab a stem at dirt level with your two hands and pull it. Some may come without roots, but then just try a second one.
Choose a middle sized shoot. Pull it and remove all but the last 2 or 3 leaves.
Keep her in a shady spot for one week, water every day and then in a half shady spot.
Good luck.
By the way, looks like "hardy Chicago" for me ...But without ripe fruits ...