Kabbalah zu Schemot 1:14

Similarly, in the days of Joseph, who had his father's image, exile was avoided because he protected them throughout his life. But when he died, exile immediately befell them, as it is written, "And Joseph died" (Shemot 1:6) which is followed by, "Come, let us deal wisely with them" (Ibid. 10), and "they made their lives bitter with hard bondage, in mortar, and in brick..." (Ibid. 14).