I cannot find any documentation that links a "Mary Brown" with parents Joseph and Rachel Brown, or a Mary Brown from Appleton, Maine who married a Dyer.
In the 1820 Census, Joseph and Rachel were living in Appleton and they had two sons under the age of 10. I presume those two sons were James and Levi, born in 1816 and 1818 respectively. In the 1840 Census, James and Levi were (presumably) included as adults, along with their parents, for a total of 4 adults. There were 9 children listed in the 1840 Census for a total of 13 people residing at the Brown farm in Appleton, Maine.
In the 1850 Census, only six of the eleven children are listed. James and Levi and Nathaniel had all moved out. That leaves two unidentified children who were living with Joseph and Rachel in 1840, but were not living with them in 1850. Any assistance will be greatly welcomed.
1820, 1840, 1850, 1860 U.S. Census records for Appleton, Maine, available on Ancestry.com.
1870 U.S. Census record for Pembroke, Maine, available on Ancestry.com.
Have you taken a DNA test? If so, login to add it. If not, see our friends at Ancestry DNA.