He was mulatto, so his father was white and his mother was a slave. If he lived in the house with the white family, he was what was called a child of the plantation, meaning his white father didn't leave him in slave quarters and maybe regarded him as a son. His birth, however, would not be recorded the way white children's were.
My gg+ grammy, Letha, the wife of Wilson Lee, was a child of the plantation. Her daddy/owner was Benjamin Roach. Mr. Roach, contrary to the law then, educated Letha and even tried to send her to medical school.