Impermanent self

I hear people saying this a lot regarding Annatta - "if something is impermanent then it cannot be self." But doesn't this only apply if you're coming from the view that a 'self' must be permanent? Why can't the self be impermanent? If I have the view that there is a self and it's impermanent then something impermanent can be self.