I've yet to find a contemporary document calling it Plymouth Bay Colony. As for "New Plymouth", yes, but that was the town or settlement, not the larger colony or county entity.
And if you're anal about it, Jim, so am I. I do change profiles that name "Massachusetts, USA" prior to 1776, and change them to the colony names prior to 1691. I try to be as accurate as possible. But then dealing with place names in Maine... that's frustrating, too.