To be fair, when you write a song that publicly chews out your ex and then make a video for it full of imagery pertaining to her... people will continue to link the song and said ex. You set a strong image like that and it'll stick - in a weird way, people still linking the song with Britney is a testament to the impact he made with it and the video. Same way people still want to know who Carly Simon wrote You're So Vain about... these things leave impressions.

That said, it is seriously overstretching it to think that five years later every ad lib is still calculated to be a barb. At the very worst he's being a bit cheeky (and that's probably putting it too strongly), and I doubt either he or Britney cares at this juncture so what the hey. It fits the song, he's having a bit of fun and it's really not that serious