You're used to having someone around you all the time. And now that he's not, it's like a culture shock. You'll get used to it eventually. If not, find a replacement.