I *think* that Richard might be an illegitimate child of Mary Armitage. He's Martha's grandson, and in the 1861 census he's Frances' nephew. And Mary was 19 when Richard was born, while her siblings (that I know of, at least) were too young. Hard to be certain, though, with what I've got.
So, you're saying that Edward doesn't belong to this family, since he appears to have been born quite close in dates to Joe, who is in my family, correct? In that case my next step might be to find everything I can on that family as well, and see if I might at least eliminate some possibilities that way. Thanks!