Yes! While I think thanks should be given if someone holds a door for someone else, when men comment on our appearance and it's unsolicited, it's often not welcome and we don't owe them anything.
I say in my book that when men (especially at work) comment on your appearance, one way to get them to stop is to either ignore it or as you say, agree with them. It knocks them sideways! If they're trying to make you feel embarrassed, it clearly didn't work.
And can I add - We certainly don't owe them "please" but sometimes "thank you" does the trick. If someone is being sexist, use the words "Thank you" instead of please. It conveys an expectation that your request (or command) will be delivered!