When to let things let go and when to fight for them?

I find myself in conflict with the Buddhist idea of letting go. If someone insults you, is rude to you, insults a loved one, harrases you or a loved one and so on. According to Buddhism, we should forget about it. Past is past. But then on the other hand Buddhism also mentions that we should be courageous. And sometimes being courageous can lead to further pain. Say that you confront people who insult a loved one and then they stab you. I'm having a problem to decide when to confront and when not to. For now, I always let go and I feel like a doormat sometimes. Thank you.