It appears that there is a James B Walker, born about 1861 in TN, who has a wife named Mary, and they lived in Knoxville, TN at the time of the 1900 census. But his brothers were Alex and John (according to the 1870 census), he had 3 daughters, Zola, Trula, and Zelda (1900, 1910, & 1920 census), and he died on 2 Sept 1925 in Knoxville, TN, so he doesn't appear to be your James Butler Walker.
There is another James B Walker, born about 1861 in TN, listed in the 1870 census in Cookville, Putnam County, TN with an older brother William J Walker, but it appears that he married a woman named Emily, and lived in TN after 1940.
Have you found his death record, or a marriage record for James Butler Walker and Mary Elizabeth Knoy? Does it list where either of them are from?