Jaywalking could become a critical issue. Pedestrians and pranksters, knowing that the cars are programmed to yield to any in their path, could bring traffic to a halt. Outfitting the cars with facial recognition technology could help identify violators, but that raises its own tricky issues.
Drivers lose their shit over speed cameras but, hey, sure blanket cities with facial recognition devices to bust jaywalkers (and much of what people consider "jay walking" is, at the moment, perfectly legal generally if not everywhere).
At least I've noticed more urbanist types who thought self-driving cars would be great for urbanism are starting to lose the faith a bit.