Just tell him "some people are stupid", and remind him of the word assume. (I've known people assume P D James was male, just because of the genre of writing, plus use of initials.)
When we record females named Wallace, and males named Evelyn, and there are well known persons such as Glenn Close, why should anyone be surprised that there are names that can be either sex? Just because there are different spellings for males and females for some names, it is not/was not always so. I have a niece Stacey (supposedly the "feminine" form of Stacie/Stacy). There's at least one male actor named Stacy. The famous (or, at least, he used to be) writer Evelyn Waugh. Names such as Ashley/Ashleigh, Leigh/Lee, Lesley/Leslie, Alison/Allison, Vivian/Vivien, Beverly/Beverley, Anne/Annie, Lindsay/Lindsey, Joyce, Meredith, and more, have always been either/or.
Some such names, too, have been markedly male for a number of generations, then switched to being female for generations.
Names that come from surnames can be male or female, but some will always try to "force" them to be one or the other. At least we're more open to the possibilities! :)