Nachschlagewerk zu Bamidbar 22:42

But He was not angry during all those days… You might ask: Since he could only curse them at the moment when Hashem was angry, and Hashem was not angry during all of those days, why did Hashem tell him not to curse the people? Surely the curse would not have taken effect. The answer is that Hashem knew that they were destined to stumble with Baal Pe'or and that a plague would come upon them, thus if he has cursed them, people would have said that the plague came because of the curse.