My solution is to carry a pocket knife in my unlocked compartment where my phone is plugged in. The trick is to insert the blade in the correct location by the retainers and twist the blade and not pry with it. Same technique works with a small screw driver. Twisting works better than prying for me.
I agree. It’s too hard without assistance from a blade or some other device.