When I first moved to Brooklyn, there was a synagogue whose building had burned down and was temporarily -- for a few years -- located in a shack built in someone's yard. Because a fruit tree stood in the middle of the yard, the structure was built around it, with the tree in the building surrounded by wood so it could not be seen. The reason for this, I was told, was that because the tree was fruit-bearing it could not be removed due to a biblical prohibition (Deut. 20:19-20). However, was this necessary?
R. Ya'akov Kamenetsky as quoted in R. Ya'akov Jacob, Bi-Mehitzas Rabbenu (p. 223):
Our master ruled that it is permissible to cut down fruit trees "for the need of building" (regarding a new building of Yeshiva Beis David), based on the words of the Taz (Yoreh De'ah 116:6). Even after they told our master that the gentile worker was injured in his leg during the cutting, he still did not change his position and was only interested in whether the yeshivah had insurance for this.Perhaps, though, since the building for this synagogue was intended as temporary (even though it remained for a number of years), the conclusion might be different.