Unless the public building is toward a certain religion (church, Christian store, ect.), then yes. No matter where you live, someone will always have a diverse religion than what you do. Public places need to give the impression that no matter what religion you are, you are welcomed and deserve their services.
as a christian i would like to see all symbols of other faiths taken down and christian symbols everywhere even on public buildings but considering this would never be agreed to i would say that all of them should be taken down from public buildings so that it is equal