I can understand both points of view. He's spending all his other time with you and wants to have a break to see friends without you being ever present, but then you feel like you're stuck at home and you want to go out and feel like you haven't got anybody else you can do that with.

Think it comes down to you needing some friends where you are so you're not so dependent on him for company