There's no reason to storm off if all you did was casually ask him about it. His overreaction makes me think he's feeling guilty -- not necessarily because he's done anything, but just because he knows he's mentioned it too much, so you noticing too made him defensive