You did the right thing. There wasn't a lot else you could say to them in the moment; they held all the cards, and all the power to deny and retaliate if you'd tried to report them.

Although I advocate standing up to sexism in all forms when you can (otherwise it's going to continue), personal safety (both physical and mental) should always come first.