Midras Yosua 15:8: Pirkei DeRabbi Eliezer & Sifrei Devarim

What did the men of Jebus do? They made images of copper, and set them up in the street of the city, and wrote upon them the covenant of the oath of Abraham. When the Israelites came to the land (of Canaan), they wished to enter the city of the Jebusites, but they were not able (to enter), because of the sign of the covenant of Abraham's oath, as it is said, "And the children of Benjamin did not drive out the Jebusites that inhabited Jerusalem" (Judg. 1:21). || When David reigned he desired to enter the city of the Jebusites, (but) they did not allow him, as it is said, "And the king and his men went to Jerusalem against the Jebusites, the inhabitants of the land; which spake unto David, saying, Thou shalt not come in hither" (2 Sam. 5:6).

We are hereby taught that the Temple is built in the portion of Benjamin and like the head of an ox extends from the portion of Benjamin to that of Judah, as it is written "and between his (Benjamin's) shoulders (in a slight depression on the highest part of Benjamin's land) does it (the Temple) rest." As to its being written (Bereshith 49:10) "The scepter shall not depart from Judah," that refers to the chamber of hewn stone (in the Temple), which is in the portion of Judah, viz. (Psalms 78:67-68) "And He rejected the tent of Joseph and did not choose the tent of Ephraim. He chose the tribe of Judah, Mount Zion, which He loves." But the Temple was in the portion of Benjamin, viz. (Bereshith 35:19) "And Rachel died, and she was buried on the way to Efrath, which is Bethlehem."