If someone causes me pain but I don’t lament it, feel sorrow about it- what circumstance, if any, warrants acting against the person causing pain?

Step 1: Some person causes me pain Step 2: let’s say I stay neutral about pain, pain does not cause me sorrow, and I don’t feel even slightest feelings of ill will towards that person — all because I know pain is impermanent and it is not self Step 3: so at that point, should I take action against this person, as matter of cause and effect - or not - or what would Buddha do in similar circumstances