It appears that he is likely the son of Gilbert P. Milligan and Julia Glendora Baker. The son Charles B on the 1880 census in NC has a brother Creswell. In 1900 census Gilbert and Julia have moved to TX (prior to 1888) and Creswell is living with this couple. Julia has 6 total children, 5 living, but only 4 are in their household. Charles being the oldest may have moved out on his own? Can't find him again until 1901 marriage to Lillian.
On both Julia and Gilbert's death certs - the informant is Charlie or CB Baker of San Marcos, Hays, TX - which is where Charles is living with his wife Lillian Virginia Bennett.
Charles B Milligan
death 07 Nov 1963 : Austin, Travis, TX
Male White Widowed
death age: 87 years 10 months 2 days
birth date: 25 Dec 1875 : NC
occupation: Carpenter
residence: San Marcos, Hays, Texas
cemetery: San Marcos City
burial place: San Marcos, Texas
burial date: 07 Nov 1963
Texas, Marriages, 1837-1973
https://familysearch.org/pal:/MM9.1.1/F6B6-P41
Groom C. B. Millegan
bride L. V. Bennett
marriage date: 20 Mar 1901 Hays County, Texas
U S Census, 1880 Catawba, Catawba, NC
https://familysearch.org/pal:/MM9.1.1/MC6F-J2K
self Gilbert P. Milligan M 26 NC
wife Julia G. Milligan F 22 NC
son Charles B. Milligan M 4 NC
son Creswell Milligan M 1 NC (Oscar Creswell on other records)
U S Census, 1900 ED 70 Justice Precinct 6, Guadalupe, TX
https://familysearch.org/pal:/MM9.1.1/M3GJ-9XJ
head G P Milligan M 47 Oct 1853 NC married 26 yrs
wife Julia Milligan F 43 Dec 1857 NC 6 children 5 living
son Oscar C Milligan M 22 July 1878 NC
dau Clara Milligan F 20 Aug 1880 NC
son Collie Milligan M 12 Oct 1888 TX
dau Myrtle Milligan F 10 Oct 1890 TX
North Carolina, Marriages
https://familysearch.org/pal:/MM9.1.1/F8XR-QLS
groom Gilbert P. Milligan
groom: 1853 birthplace: Of Catawba Co.
age: 21
bride Julia Baker
birth date: 1857 birthplace: Of Catawba
age: 17
marriage date: 23 Dec 1874
marriage place: Catawba Co., North Carolina
Texas, Deaths, 1890-1976
https://familysearch.org/pal:/MM9.1.1/JNSB-JK8
Julia Glendora Milligan
death date: 04 Dec 1934 Austin, Travis, TX
Female White Widowed
death age: 76 years 11 months 13 days
birth 21 Dec 1857 NC
spouse G. P. Milligan
father Henry Baker
burial place: Oakwood Cem.
burial date: 05 Dec 1934
Informant Charlie Milligan, San Marcos, TX
Texas, Deaths, 1890-1976
https://familysearch.org/pal:/MM9.1.1/JXXW-BQX
Gilbert P. Milligan
death 07 Aug 1916 Austin, Travis, TX
Male White Married
death age: 62 years 10 months
birth 07 Oct 1853 NC
father G. A. Milligan birthplace NC
occupation: Carpenter
cemetery: Oakwood
burial place: Austin, Travis, Texas
burial date: 08 Aug 1916
Informant C B Milligan, San Marcos, TX