If you haven't already seen it, this publication might provide a clue for your search:
Cook, B. T. (2005). ''Brown families of the Virginias: Descendants of James Brown Sr. and Esther Goff Brown of Tazewell County Virginia''. Cape Coral, Florida: B.T. Cook.
It says that James Brown was born about 1755 in Virginia. He married Esther Goff in 1775 (age 20). They had eleven known children. He died in 1814. Their descendants and relatives lived mainly in Virginia and West Virginia.
This may be a one-off, private publication, since the only library I've found that has a copy is the LDS Family History Library, 35 NW Temple, Salt Lake City, UT 84101, (801) 240-2584.
Good luck with your search!