Could the following be the family you are searching for?
1860 Wayne County, Indiana Census, Wayne Twp., Richmond P.O.
June 23
Page 712
1295 Dwelling House
1303 Families numbered in the order of visitation
Armstrong Grimes 42 M Indiana Farmer
Judith A. Grimes 41 F Virginia
William S. Grimes 16 M Indiana
Peter Beard 76 M Pennsylvania
June 23
Page 711
1290 Dwelling House
1298 Families numbered in the order of visitation
George Grimes 60 M Kentucky
Mary Grimes 52 F Indiana
David Grimes 32 M Indiana
Martha J. Grimes 23 F Indiana
Mary E. Grimes 16 F Indiana
Clinton Grimes 16 F Indiana
George F. Grimes 16 M Indiana
James M? Grimes 12 M Indiana
Lourena Grimes 10 F Indiana
Nancy D. Grimes 8 F Indiana