Lashon Ha-Ra about "Outsiders"

R. Binyamin Yehoshua Zilber (Responsa Az Nidberu, vol. 14 nos. 60,69) discusses whether one is prohibited from speaking lashon ha-ra (negative gossip) about gentiles and non-religious Jews. From a technical legal perspective, the prohibition only relates to those who "perform the acts of one's nation," i.e. observant Jews. However, R. Zilber (a haredi halakhist from Israel) writes that one may still not speak lashon ha-ra about such people because it can cause negative feelings and creates negative character traits within a person.