Commentaire sur La Genèse 44:17

It would be degrading for me. I do not wish to be the instrument of chastisement for your earlier crimes because Hashem uses the wicked for that purpose; see I Shemuel 24:14.

חלילה לי מעשות זאת, "It would be a disgrace if I were to do such a thing." Joseph meant that even if it were true that G'd was using this opportunity to punish the brothers for some other sin, he was neither authorised nor competent to administer such retribution. He was only authorised to deal with guilt which was obvious to him. He was entitled to judge the person in whose possession the goblet had been found. The brothers were free to go home, however. Joseph thought that by allowing them all to go and by keeping only Benjamin as a slave he had appeased Yehudah. He felt certain that Yehudah had accepted the verdict. How was he to know that Yehudah would immediately begin to lambast him as is evident in the following verses. According to our view Yehudah reasoned with Joseph as long as he felt that they were all being punished for something they had done in the past. When he realised that innocent Benjamin was being singled out for punishment whereas they, the guilty ones, were allowed to go free, he realised that he did not confront divine judgment in the person of Joseph, but that Joseph was a capricious ruler who had framed Benjamin for reasons of his own. There was therefore no cause for the brothers to submit to what they had previously considered as divine retribution.

אל אביכם. Who is waiting for you to bring back food.