Sounds like my grandfather. I didn't speak to him for a good 10 years of my life and he lived 15 minutes from me. As an adult looking back, I just think he was an asshole, but I'm not scarred for life or anything.