וַיֹּ֕אמֶר לֹֽא־יֵרֵ֥ד בְּנִ֖י עִמָּכֶ֑ם כִּֽי־אָחִ֨יו מֵ֜ת וְה֧וּא לְבַדּ֣וֹ נִשְׁאָ֗ר וּקְרָאָ֤הוּ אָסוֹן֙ בַּדֶּ֙רֶךְ֙ אֲשֶׁ֣ר תֵּֽלְכוּ־בָ֔הּ וְהוֹרַדְתֶּ֧ם אֶת־שֵׂיבָתִ֛י בְּיָג֖וֹן שְׁאֽוֹלָה׃
И сказал он: 'Сын мой не пойдет с тобою; потому что его брат мертв, и он остался один; если он постигнет вас путь вреда, то вы сокрушите мои седые волосы от скорби до могилы.
Rashi
לא ירד בני עמכם MY SON SHALL NOT GO DOWN WITH YOU — He did not accept Reuben’s offer. He said, “What a fool is this oldest son of mine! He suggests that I should kill his sons. Are they his only and not mine also?” (Genesis Rabbah 91:9)
Radak
כי אחיו מת, not a statement of fact but an assumption. Yaakov reasoned that if Joseph were alive he would have heard from him at some time during the over twenty years that he had been missing.
Sforno
My son. Yaakov refers to Binyamin in this way because he was the only son left from his primary wife [Rochel].