Say I’m in a fair-sized city. How far down would I have to dig to no longer be in that city?

Legally, you’re still in the city at any depth this side of the center of earth.

If you’re looking for undisturbed soil:

Suburban – houses: 5′.

Suburban – streets: 20′ in most cases, but at low as 60′ for large deep utilities.

Suburban – commercial buildings: 20′ in most cases, but as low as 40′.

Urban – general: 20′ in most cases, but as low as 60′.

Urban – city center – streets: 40′-60′ in most cases, but 120′-240′ is subways are present.

Urban -city center – skyscrapers: 60′-120′ are common, but extend to 300′-400′ below the lowest floor for some caisson foundations.