I am not familiar with the family but after looking over the records, this is what I would guess as being the family that arrived from Ireland between 1829 and 1835 (based on reported birth places and years of children) and resided in the Leeds Co. Area:
John Ivey b. ~1796, Ireland m. Hannah Donahue b. ~1798, Ireland
I have not been able to find an earlier record than the 1861 Lansdowne Twp. Leeds Co. P. 40 transcribed here: http://ontariocensus.rootsweb.ancestry.com/transcripts/1861/1568-8.html
As you can see from the above record, the spelling is already alternating between Ivy and Ivey and it does so a few more times over the following decades (in addition to being mistranscribed at one point as Tuey)
From the various records for the children it is likely the above family lived for a time in Yonge Twp., and Elizabethtown Twp. in addition to the Front of Leeds and Lansdowne Twp. (name varies over census years) where they are enumerated from 1861 onwards.
I have not run across any Elizabeth Ivey's, although I have seen mention of a Catherine (I did not include her above because I saw no actual records with her named)
Also of interest there is a record of a Patrick Purcell marrying a Bridget Ivy in 1842 at a location that is associated with a number of other records from the above family (Saint Francis Xavier, Brockville, ON) unfortunately there is not enough other detail to draw any useful conclusions from this transcription Ref: https://familysearch.org/pal:/MM9.1.1/XLP2-F6X This would have been a year or two before the birth of the eldest child given for Michael O'Grady and Bridget Ivy - despite her very young age, it is a possibility that needs to be considered.
If this is the Ivey family you are focusing on here are some more relevant records:
But before I list them a possible !!!!Eureka?!!! moment
I found on Family search a record for Michael O'Grady marrying Elizabeth Ivy in Jan 1842 (https://familysearch.org/pal:/MM9.1.1/F2VG-5P1). In May 1842 there is the record I already posted previously for Patrick Purcell marrying Bridget O'Grady. Since all the census records (and birth records for children) give Michael O'Grady being married to Bridget O'Grady perhaps the church records (or the transcriber) have flipped Elizabeth and Bridget's names. Perhaps getting copies of the records that originate at Saint Francis Xavier Roman Catholic Church in Brockville will solve the mystery. Details on the source are here: http://www.archivescanada.ca/english/search/ItemDisplay.asp?sessionKey=1143412449030_206_191_57_196&l=0&lvl=2&v=0&coll=1&itm=262303&rt=1&bill=1
Anyway, the other records are:
1911:
John Ivey (Jr.) and Elizabeth: http://data2.collectionscanada.gc.ca/1911/jpg/e001999395.jpg
Bridget O'Grady ALSO a Patrick Ivey http://data2.collectionscanada.gc.ca/1911/jpg/e001999456.jpg *Note this record gives Bridgets year of Immigration as 1829.
1901:
Michael Ivey and Mary: http://data2.collectionscanada.ca/1901/z/z002/jpg/z000078351.jpg
Bridget O'Grady http://data2.collectionscanada.ca/1901/z/z002/jpg/z000078350.jpg
1891:
John Ivey (Jr.) and Elizabeth http://data2.collectionscanada.gc.ca/1891/jpg/30953_148152-00159.jpg
Michael Ivey and Mary http://data2.collectionscanada.gc.ca/1891/jpg/30953_148152-00197.jpg
1881:
Michael Ivy (widowed?) ALSO James Ivy and Jessie with mother Hannah Ivey ALSO Bridget O'Grady http://data2.collectionscanada.gc.ca/e/e327/e008171756.jpg
John Ivy and Elizabeth http://data2.collectionscanada.gc.ca/e/e327/e008171746.jpg
1871:
John Ivey (Sr.) and Hannah with son James ALSO Michell http://data2.collectionscanada.ca/1871/jpg/4396307_00668.jpg
John Ivey (Jr.) and Eliza http://data2.collectionscanada.ca/1871/jpg/4396353_00021.jpg
Bridget O'Grady http://data2.collectionscanada.ca/1871/jpg/4396307_00667.jpg
Aside from Census records there are:
Christening: Patrick Ivey https://familysearch.org/pal:/MM9.1.1/F2H3-TQX
Christening Ellen Ivy https://familysearch.org/pal:/MM9.1.1/F2H3-T5J
Christening James Ivy https://familysearch.org/pal:/MM9.1.1/F2HQ-9ZR
1881 Marriage Michael Ivey to Mary Ann Grennon https://familysearch.org/pal:/MM9.1.1/FMJ2-ZS6 ***Michael married twice, both times to a Mary - The other one was Mary L. Loftus; I didn't find a marriage record but some of the childrens birth records reflect this - and it makes sense with the 1881 census year where Michael has no wife, as the marriage was 1 June and the enumerators started collecting data on 4 April.
1918 Death Registration of John (Jr.) Ivey https://familysearch.org/pal:/MM9.1.1/JXDP-FWL
1862 Marriage of John Ivey and Elizabeth McDonnell https://familysearch.org/pal:/MM9.1.1/F2LV-VY2
1873 Marriage of James Ivey and Jessy Ann Myers https://familysearch.org/pal:/MM9.1.1/FMFB-LW7
1861 Marriage of Mary Ivy and William Gorman http://homepages.rootsweb.ancestry.com/~maryc/leeds61.htm
There are other records I ran across (such as Patrick Ivey's two marriages in Montana), but I need to get on with some other things - hope this all helps you make the connection you are looking for,