Commentary for Genesis 42:38

לא ירד בני עמכם MY SON SHALL NOT GO DOWN WITH YOU — He did not accept Reuben’s offer. He said, “What a fool is this oldest son of mine! He suggests that I should kill his sons. Are they his only and not mine also?” (Genesis Rabbah 91:9)

My son. Yaakov refers to Binyamin in this way because he was the only son left from his primary wife [Rochel].

כי אחיו מת והוא לבדו נשאר, "for his brother is dead, and he alone remains, etc. Jacob felt the need to explain how he evaluated the certain loss of Shimon who was presently a hostage in an Egyptian jail- if Benjamin were not to go to Egypt, compared to the possible loss of Benjamin if the latter were to go to Egypt. Jacob explained that if he were to lose Benjamin he would not have a son of Rachel left at all. If he were to lose Shimon, however, he would at least have other sons of Leah left to him. Besides, Jacob described an accident which would happen to Benjamin as practically certain; this is why he spoke about וקרהו אסון in the past tense instead of יקראהו אסון, something in the future. He added the words בדרך אשר תלכו בו, "on the journey you will undertake." He hinted that what happened to Joseph happened to him although he travelled on a route which he was familiar with and had travelled many times without encountering anything dangerous. Jacob was afraid that something similar could happen to the brothers even on a route they considered safe.