Evidence suggests we subtly believe that going north is more time-intensive than going south:
[A] number of subjects were asked to estimate the travel time for a northbound versus southbound bird. The majority of respondents believed traveling north from the equator would take longer than the reverse. “A lifetime of exposure to the metaphoric link between cardinal direction and vertical position,” they write, “may cause people to associate northbound travel with uphill travel.”
This little effect, the study found, has a number of potential implications. People were more likely to think it was cheaper to ship things to southern destinations than northern destinations, and the directional “framing” in retail advertisements.
Over time something called “feature accumulation” happens:
“Feature accumulation hypothesis” — i.e., the more information there is to be observed about a journey, the longer it will seem. With each journey, [one grows] more familiar with the route — every last billboard, fast-food restaurant, interesting natural feature — and thus also more familiar with the time remaining. …. Feature accumulation is one reason people seem to inevitably overestimate the duration of a walking trip — and underestimate a trip by car. There’s simply more to see, more to take note of.
This is why traversing New York seems faster than traversing Venice, for instance. It’s “enlarged by its complexity.” If, of course, you walk with your head down like a good New Yorker, it could skew the results further.