I figure a lot of the humans are like the Skywalkers- descendants of slave labour brought in by other races luke the Hutts [who probably can't afford to leave if free to do so], or there for otherwise not-good reasons.

As in since it's a backwater, desolate planet, it'd be the spot to go about your illegal business, so I imagine some of the human population is there for assorted immoral purposes. Smugglers, gangsters etc.