Ugh, so many women will relate to this.
Your point about not stopping to shut it down is very important. Guys like this (and in many situations) are testing the waters, seeing how far they can push things. And until you shut it down they will keep trying.
If you ignore him and keep walking, he will continue to ignore your discomfort and convince himself that since you haven’t blocked him, you’re ok with what’s going on. (You may even be teasing him, egging him on.)

Golden rule — no matter how “nice” he is (or says he is) he clearly isn’t bothered about how ill at ease you are. Don’t so something just to “be nice” or to spare his feelings. We are conditioned to put other people first.

He doesn’t care about your feelings so you have to.