Commentaire sur La Genèse 42:14

הוא אשר דברתי THAT IS IT THAT I SPAKE UNTO YOU — The statement which I made — that you are spies — is true and certain. This is the meaning according to the literal sense of the words. A Midrashic explanation is: he said to them, “And if you find him (your brother) and they demand of YOU a large sum, would you redeem him?” They answered him, “Certainly!” He asked them, “If they tell you that they will not restore him to you for any sum of money, what would you do?” They replied, “That is what we have come for — to kill or to be killed”. Whereupon he retorted:הוא אשר דברתי אליכם THAT IS JUST WHAT I SAID TO YOU — you have come to kill the people of this city. I have divined by my goblet that two of you destroyed the great city of Shechem (Genesis Rabbah 91:7).

הוא אשר דברתי אליכם, the very one of whom you have said that he is no longer, whereas you refused to be specific about what happened to him, is the one who went back to report all that you have seen here, or that you have decided to stay a while in order to engage in spying.

הוא אשר דברתי, "This is precisely what I have said (you came to spy)." Joseph meant: "I do not retract a single word from what I have said before." The word לאמור here means "I continue to say what I have said all along."

הוא אשר דברתי, if you really spoke the truth that ten of you came as brothers, why did the eleventh not come also? As far as your claiming that he stayed home in order to attend to the needs of your father, what about your father’s wives and your own wives? Are they not able to cope with these chores during your absence?

The statement that I made, that you are spies... Rashi is answering the question: Does the verse not imply that the brothers, too, said they were spies? For we find Yoseif responding to them, “That is just what I said... you are spies.” Yet this cannot be true, for they said, “Your servants have never been spies.” Rashi answers: The word “that” (הוא) belongs at the end of the sentence. [Accordingly,] the verse means: “What I said to you, saying you are spies, that is the truth.” Some ask: What was it that led Yoseif to accuse them of being spies, causing him to say, “That is just what I said... you are spies”? Another question: How would bringing Binyamin verify that they were not spies? The answer is: First he said to them, “You are spies, for you entered through ten different gates of the city,” as Rashi explained. They replied, “We are all the sons of one man, and a man does not send all his sons together to spy, lest they be caught and all killed.” He said to them, “According to this, you surely are spies. Would a man send all his sons [to buy food] without leaving one or two to serve him? Perforce, your father sent [some of] you to spy the land, but since [each of] you was afraid to be caught and did not want to [be the one to] take the chance, your father was obliged to send all of you. Furthermore, he did this so that you could take counsel from ten, because “Salvation results from much counsel” (Mishlei 11:14). To this they replied, “Your servants are twelve brothers, and behold, the youngest one stayed with our father to serve him, while the ten of us went to search for our lost brother. None would do this as faithfully as his own brothers. We risked our lives to bring him back to his father, and entered through ten gates.” Yoseif replied, “That is just what I said to you. You are spies. You claim your brother is missing for many years, but you did not search for him until now! Surely you are lying, and you came to spy. And it is also a lie that you left one brother with your father to serve him. Therefore, you shall be tested in this manner. If your brother comes with you, it is questionable whether you are spies. But if he does not come, you definitely are spies, for the reasons I stated.” This answers all the questions. In the name of Maharshal I found somewhat similar to this.

Das eben greift Josef auf. Weil euer Vater weiß, dass euer Vorhaben ein gefährliches ist, schickt er den Jüngsten nicht mit.

It is also possible that Joseph meant: "When I accused you of spying this is exactly what I had in mind. You came to look for something you had lost. You yourselves have only confirmed it. The Midrash there describes the discussion about what the brothers were going to do if they located their brother and the owner would refuse to release him The word לאמור is an allusion to what the brothers had said they would do in such an event.