When you step away from religion, nothing you do has to be about religion unless you make it that way. I don't mind if the religious are inspired by their religion to do good things. When they glorify it or do it for religion's sake, or think that their personal beliefs apply to everyone else, I doubt their honesty.
I've been invited to churches for social gatherings. It doesn't bother me to go and act a little, and people try to help me feel like I'm a part of it even if I'm not. I wear my TOOL hat and my Nine Inch Nails shirt, even there - not out of defiance or disrespect, but simply because that is how I choose to dress. I used to go to gothic cathedrals or castles in Germany just to admire the architecture and the atmosphere, but I saw nothing wrong in the fact that I was not a Catholic or a Templar.
None of this will stop me from going to see the pyramids, whether they are in Egypt or Peru -- though they, also, are religious. Religion is part of history and culture, as is music. There's no "establishment" you're betraying by continuing to enjoy and take part in your personal taste in music.
It's not about Christianity or Atheism. It's about YOU.