There was a Joseph Mathurin Oliver born 22 June 1888 who married Alice Bouchard.
Joseph died on 2 May 1972 at Vancouver, BC, Canada
There are several men named Joseph Olivier who left Marseille, Plouezec, Kerity or other parts of France and came to New York to Ellis Island. Maybe one of these is your Joseph Olivier.
If you can provide birth, marriage or death dates regarding Joseph or his parents, it could improve the liklihood of obtaining some answers to you search for information.