Hi, Edna. The short answer--which Lynda stated--is that yDNA testing, by itself, won't pinpoint a particular person. Unlike autosomal testing, there's no degree of relationship inferred by yDNA testing. Results are presented as a probability of generational distance, an estimate of the number of generations to a shared male ancestor.
That said, if the standard cost of the Family Tree DNA Y-37 test ($149 if purchased through a surname project) is not an issue, I would strongly recommend ordering the test. A few reasons:
1) A set of autosomal DNA results, by itself, can't differentiate between the DNA contributed by the father from that of the mother. Even at close relationship distances like 2nd or 3rd cousins, not having a paper trail for your father's line means it will be more difficult to understand whether the match is on your grandfather's branch, or on your grandmother's. A solid yDNA match to a male cousin when combined with a close autosomal match is bankable, and may give you a touchstone to help begin classifying paternal and maternal lines...especially if that match is a 1st or 2nd cousin.
2) Autosomal DNA is going to give you powerful evidence out to the 1st cousin level, or your dad's grandparents; very strong evidence out to 2nd and 3rd cousins; usable but weaker--and more complex to deal with--evidence at 4th and 5th cousins; and more than likely unusable evidence at 6th cousins and beyond, putting a ceiling of functional use for most people at the 4g-grandparents (my opinion only and some certainly disagree). If your AncestryDNA results are showing no matches closer than 3rd cousin, a yDNA match to one of those more distant cousins can provide important additional evidence. It may not validate the actual relationship, but it would confirm you're on the right track and on the right branch of someone's family tree.
3) Last, and perhaps most importantly, FTDNA is the only major testing company that stores submitted samples for future testing. You'll get two vials and two cheek swabs with the Y-37 test (swab thoroughly, than swab some more!); FTNDA will only open one vial if the sample is adequate to complete the test ordered. The unopened vial goes into long-term storage, but the opened vial is also stored and can be used for subsequent tests.
I have a sample submitted in 2004 that's had 16 separate tests run against it, the most recent last year. FTNDA hasn't yet told me that sample is defunct...though it must be pretty close to it. Larger-scale tests like the NextGen Big Y that sequences most of the Y-chromosome requires more material, but that unopened vial would be enough. Bottom line is that I feel it's important to, whenever possible, get our elderly parents' samples into storage. You can use the initial test results to help with the immediate requirements, and still be able to order a new test in, say, 2030 if you want to. As quickly as genetic genealogy has advanced the past decade, that DNA "time capsule" could prove invaluable to your research and your family's legacy.
A tip: you can also download your raw DNA results from Ancestry, and then upload them to FTDNA to give you more possible matches and additional comparison tools. Same sort of process you went through on GEDmatch. I believe the fee to "activate" the results for matching is $19. If your dad's yDNA test and autosomal results are on file at FTDNA, potential matches will see both and be able to better understand how the two types of DNA correlate.
Good luck with the search!