It is hard for me to piece this together without concluding that you and your match are half-first cousins, each sharing your maternal grandfather. (There is an outside chance that you could be full first cousins, but that is a very low probability if you share 440cM.)
The aU number suggests that the match is around 1C1R or equivalent, i.e., a half-first cousin., or maybe a 2C. But I see no way to get you that X-match number at second cousin level except assuming that multiple females had children with no crossover on the X-chromosome during meiosis. I don't know how common it is to have no X-crossover, so if it is more common than I think then this analysis will be incomplete.
Anyhow, if grandpa had two daughters, he would have given his entire X-chromosome to each of them.
If each of those daughters has a child, the X-chromosome she gives that child is recombined, so is half her paternal X-chromosome and half her maternal chromosome. So, the children of these two sisters are expected to share only 1/4 of their X-chromosomes, i.e., 25%. That's already too small for your match.
Because you believe that you and your match both got this X-chromosome through your mothers, both would have the chance to recombine. So you can't insert a male in the possible connections and still reasonably expect to share fully half of an X-chromosome.
So I guess the question is -- do you know *anything* about your match's mother's adoption circumstances? Was your maternal grandfather alive and an adult at that time, and was she in the general vicinity where your maternal grandfather might have been?
The other question is -- is it possible to test your mom and/or hers? That would give further clarity.