Halakha Imamat 19:30: Mishneh Torah, The Chosen Temple, Chafetz Chaim, Kitzur Shulchan Arukh & Mishneh Torah, Positive Mitzvot

The Temple is not to be built at night as it says "and on the day the Tabernacle was erected," i.e. we erect it during the day but not at night. We may work on the construction from dawn until the stars come out, and everyone is obligated to build helping physically and financially both men and women just like the sanctuary in the desert. However, we do not close the schools in order [for the children] to build the Temple nor does building the Temple override Shabbat or the festivals.

(7) And all this, even if not in the house of study. But if one speaks lashon hara and rechiluth in the house of study or in the house of prayer, he also transgresses (Vayikra 19:30): "And My sanctuary shall you fear." (Our house of study is included in the category of the sanctuary, as explained by the poskim.) And we have been commanded by this verse to fear the One who resides there, wherefore we may make there only reckonings of mitzvah, such as [those pertaining to] the tzedakah fund and the like. How much more so is it forbidden to engage there in laughter, joking, and idle converse. And this is a categorical prohibition, as explained in Shulchan Aruch, Orach Chayim 151:11. And even more so is it forbidden to speak there lashon hara or rechiluth, out of fear of the Blessed L–rd who dwells there, aside from the grave issur in itself [of speaking lashon hara]. In speaking thus, one shows himself not to believe that the Holy One Blessed be He reposes His Shechinah in this house — whereby he has the audacity to speak in the house of the King against the will of the King.
And even those who study Torah regularly in the house of study, where they are permitted to eat and drink (as explained in the aforementioned Orach Chaim, section 1), in any event, if they go astray in the issur of laughing and joking or lashon hara and rechiluth in the house of study, they transgress the positive commandment of "My sanctuary shall you fear," aside from the issur itself. As the Magen Avraham wrote (151:2): "Are the Torah scholars not exhorted in respect to fear of the sanctuary? All that was permitted them, perforce, is eating and drinking because they learn in the house of study. If they had to eat and drink outside the house of study, their studies would be interrupted." (As far as converse in general which is not talk of levity for Torah scholars in the house of study, see what we have written with the help of the L–rd in Part Three.)

The sanctity of a synagogue and Beis Midrash is very great, and we are commanded to fear the One Who dwells in them, blessed be His Name; as it is written, "Revere My Sanctuary."¹Leviticus 19:30. A Synagogue and Beis Midrash are also called "Sanctuaries," as it is written: "I will be to them as a miniature Sanctuary"²Ezekiel 11:16. and it is explained that this refers to the Synagogue and the Batei Midrash. Therefore, it is forbidden to engage there in idle talk, or to make calculations, except those that pertain to a mitzvah, such as the calculations of charity funds, or the like. Such places must be treated with respect, they must be kept meticulously clean, and candles must be lit in them to show our respect. You should not kiss your small children there, because it is not proper to demonstrate any other love except the love of Hashem, blessed be His Name.