Personally, I delete comments to my own profile after I've thanked the commenter & I don't need to do any followup. On profiles I manage, I ALWAYS delete the comment proposing a merge after the merge is completed. If any pertinent information was included in the comment, I incorporate it into the biography. I also usually delete comments that I've made (mostly in connection with a merge) once the comments are no longer needed or relevant.
Because comments CAN be deleted (and once deleted, they're totally gone), I believe any comment worth leaving posted is worth incorporating into the biography, where it will forever be findable through the Changes tab. Once it's incorporated into the biography, it doesn't need to be left, so I delete it when the profile is one that I'm the only manager for.
I don't delete all comments. I leave discussions about the profile's data (e.g., spelling/styling of name, date and location debates, etc.), although ideally such discussions would have taken place in G2G, which is better designed for them. I also don't delete a comment that's needed to make sense of another comment. And I tread very lightly on deleting my comments from high interest profiles and from profiles with multiple managers - especially since I know at least one manager who feels strongly that ALL comments should ALWAYS stay on the profile because it shows the level of collaboration.