This Samuel's wife's name is Betsy, so perhaps not a match:
United States Census, 1850
||Nunda, Livingston, New York, United States|
|Birth Year (Estimated)
"United States Census, 1850," database with images, FamilySearch(https://familysearch.org/ark:/61903/1:1:MCYY-PXQ : accessed 12 April 2016), Samuel Spencer, Nunda, Livingston, New York, United States; citing family 190, NARA microfilm publication M432 (Washington, D.C.: National Archives and Records Administration, n.d.).