It can be done, I've seen it happen. The usual gig is to wait till you come to lights and wrench it off.Sounds like a very rare and difficult athletic trick to perform
I've had to kick out at a few cars, more a kind of tap with the outside of the foot than a kick. When they decided to change lanes as i was parallel to them, but nothing too serious. Just a friendly reminder to check properly.
What is easier to do is when at the lights you open the passenger door. They hate that or fold in the mirror. but actually damaging it I wouldnt do.