In 1850 and 1860, he is attending school, probably in or near Boston, Massachusetts.
His half-brother Charles was also a porter.
1890: 4 James Place, Boston, Massachusetts (US Veterans Schedules)
1880: 64 Putnam Street, Cambridge, Massachusetts with his wife Alice, daughter Harriet, step mother Caroline and half-brother Charles
1863: Boston, Massachusetts (military enlistment)
1860: Boston, Massachusetts with his father, sister, and 44-year-old Jane Bachanan
1850: Boston, Massachusetts with his parents, sister, the Bassett family (Caroline, George and J. A.), 74-year-old Virginian Julia Jackson, 30-year-old Virginian Ann Muton, Mary & Thomas Collins, and the Lavinia Johnsons (58 and 13 years old, respectively)
First Sergeant in Company A of the 54th Massachusetts Volunteer Infantry of the Union Army, the first regiment in the United States made up entirely of enlisted men of color. He was about 18 years old, single and working as a laborer when he enlisted on 12 February 1863 in Boston, Massachusetts. He was wounded in action on 18 July 1863 at the Second Battle of Fort Wagner. Mustered out 20 August 1865.
The highest rank attainable for soldiers of color at this time was Sergeant Major. The promotional path was Private -> Corporal -> Sergeant -> First Sergeant -> Quartermaster Sergeant -> Sergeant Major.
"United States Civil War Soldiers Index, 1861-1865," database, FamilySearch (https://familysearch.org/ark:/61903/1:1:F9TM-WJ9 : 4 December 2014), Burrell Smith, Jr., First Sergeant, Company A, 54th Regiment, Massachusetts Infantry (Colored), Union; citing NARA microfilm publication M544 (Washington D.C.: National Archives and Records Administration, n.d.), roll 37; FHL microfilm 881,906.
1850 United States Federal Census Year: 1850; Census Place: Boston Ward 6, Suffolk, Massachusetts; Roll: M432_336; Page: 399B; Image: 157
1860 United States Federal Census Year: 1860; Census Place: Boston Ward 6, Suffolk, Massachusetts; Roll: M653_521; Page: 914; Family History Library Film: 803521 (as Burrell Smith)
1880 United States Federal Census Year: 1880; Census Place: Cambridge, Middlesex, Massachusetts; Roll: 544; Page: 259B; Enumeration District: 444
1890 United States Veterans Schedules for Boston, Massachusetts