I read that as they were your friends but no longer religious.

I would've kicked them out or messed with them enough to the point that they would want to leave.