According to the 1891 census James became a Grocer. The Geddie & McPhail Genealogy says he was "a groceryman in Dufftown, a village in Banff, Scotland"  In the 1911 census he and his wife Mary and daughter, age 8, Marie are living at 26 Conval Street in Mortlach, Dufftown  The record indicates that James and Mary had had no other children by 1911
Have you taken a DNA test? If so, login to add it. If not, see our friends at Ancestry DNA.