Honestly, I've noticed this happen to me and other people--and I really think it's a confidence thing. Sometimes I come off as shy, and that is a turn-off for a lot of people. Have you ever had somebody speaking to you who was super-quiet or mumbly or not making good eye contact? I think people subconsciously notice these cues and respond by not being as friendly or engaged in the interaction. I'm acting more confident now and people respond to me differently, though I'm just as nice as before.

Of course you may just have bitchy coworkers. Either way you're not a loser.