The structure of London just before the 1666 Fire can be made out on this Bill of Mortality.
The top block is the 97 tiny but jam-packed parishes of "London within the walls".
But as soon as there were walls, there were people living outside the gates. So the City was always keen to extend its power, while reserving its privileges.
At an early date the City extended its power over the 16 parishes of "London outside the walls" which are the 2nd block on the Bill (with double the population of the walled area). This includes Southwark, south of the river, which had been a borough.
Later the City took control of the 12 "out-parishes", the 3rd block, which includes Islington. But for national government purposes, these parishes were still covered by the Sheriffs of Middlesex, not the City sheriffs.
The 4th block is the 5 parishes of the so-called "City" of Westminster, which wasn't controlled by the City but was often lumped with London for convenience.
Then in the 19th century, "London" was often defined as the territory of the Metropolitan Board of Works, which organized sewers and stuff and eventually morphed into the London County Council.
For other purposes, "London" was the Metropolitan Police area. (Pedants will point out here that the City doesn't run the Met, but has retained its own separate City police force)
Then there was the London Postal area, where parishes and counties were dropped from postal addresses in favour of "London SE15" etc.