Incidentally, some people have said that it doesn't make sense to say someone was born or died in a place that didn't exist at the time. That's true, except that it may not be quite right to say that the place didn't exist, but only that a particular name for the place did not exist.
For example, the continent of North America clearly existed before Columbus happened to stumble across it. It just wasn't called by that name. (Columbus didn't call it that, either.)
It's definitely a much more precise identification to say "West Virginia" or "Ohio", than it is to say "Virginia" simply because the time referred to might predate the other states. If someone was born in what we now call West Virginia or Ohio, the place obviously did exist. It just may not have been called that yet -- but we can know what place is meant.
From looking at U.S. censuses, I'd say it hasn't been uncommon for people born outside the U.S. to identify their birth place (or a parent's birth place) according to what it was called at the time of the census. That wasn't always the same as what it was when the person was born.
If the goal is an understanding of where the person was born, it may actually be more meaningful to give the contemporary name -- along with, perhaps, an explanation of what the name was at that time.
Again, though, giving the contemporary name for a place is not the same thing as naming a place that doesn't exist. The town of Williamsburg, Pennsylvania, existed when it was Williamsburg, Bedford*; Williamsburg, Huntingdon; and Williamsburg, Blair. Using a county name before the county existed doesn't change that -- we still know what place is being referred to.
[*I think I got a little carried away. Huntingdon Country was created from Bedford County a few years before the establishment of Williamsburg, so I don't believe Williamsburg would ever have been in Bedford. Still, "Williamsburg, Huntingdon" and "Williamsburg, Blair" both refer to the same place. But if it's before 1846, Williamsburg, Blair, would be anachronistic. However, Woodbury Township, Bedford, and Woodbury, Township, Blair, are not just different names for the same place, but different places. (Although it's possible that Blair's Woodbury Township have might once been connected to Bedford's Woodbury Township.)]