While Melanie's explanation of Ó and Mac/Mc is OK in broad outline, the pedant in me bristles at the little errors, so pardon me while I nitpick:
Ó does not mean "descendant of". It means plain old "male descendant". Similarly, mac does not mean "son of". It just means "son". In all cases, you get the "of" by using the genitive (possessive) form of the ancestor's name: Ó Dubháin means "male descendant of Dubhán". (Nobody was called Dubháin. That'd be like calling someone "Dwayne's".)
Ní is a contraction of inghean Úi "[daughter] [of the male descendant]", and nic is a contraction of inghean mhic "[daughter] [of the son]", i.e. neither one means "daughter of". They were the particles used for women's names in Gaelic, but this is seldom relevant in genealogy, because the English-speaking clerks invariably imposed English naming assumptions on everyone (just as WikiTree tries to impose North American naming assumptions on all of us).
On the question of whether Ó was ever used interchangeably with mac: obviously not in Gaelic, because the one was used for clan bynames, the other for patronymics. But by the 1800s, most people used inherited family names, so it hardly mattered whether the person referred to in the name was a distant clan ancestor or a many-times-great-grandfather. Thus I wonder whether any family was ever recorded alternately as O'Whatever and MacWhatever.