I too struggle with this and it can be extremely painful at times. While I was born with this, it has intensified as I have integrated PRWMS.

Interestingly enough, I am in the middle of going through Anna Wise's The High Performance Mind. She attributes this to too much Delta. I'll just quote what she says:

"If you can learn to expand yourself while you are receiving these stimuli, you will not need to reject them or defend yourself against them ... Rather, you experience the stimuli, acknowledge them, honor them, and let them pass through you without letting them stick or feeling responsible for fixing them."

Hope this helps.