Midrash su Levitico 21:4

15) (Vayikra 21:4) ("Let a husband not become tamei among his people to become profaned.") "Let a husband not become tamei among his people to become profaned." What is the intent of this? Because it is written that he makes himself tamei for his mother and his father, his son and his daughter, his brother and his sister, whether they are or are not fit (for the priesthood), I might think that he does so also for his wife." It is, therefore, written "Let a husband (who is a Cohein) not become unclean (for his wife." There is a husband who becomes unclean and there is a husband who does not become clean. How so? He makes himself tamei for his wife who is kasher, and does not make himself tamei for his wife who is pasul (i.e., who makes his children challalim, this being the thrust of "to become profaned").

16) "among his people": when they act as his people do, and not when they have strayed from the ways of the congregation. "to become profaned": If he (the strayer) follows this course, he (the Cohein in question) becomes chullin ("mundane," and unfit for the priestly service.) I might think that he remains chullin forever; it is, therefore, written "beamav." As long as he (the strayer in question) is "with it" (imo [i.e., as long as he persists in this course]), then he (the Cohein) is chullin. Once he departs from it, he (the Cohein) reverts to his original sanctity.