I have been thinking about this issue for years too and don't really have a handle on why. In my city, we are 52% Caucasian, 20% Asian, 2% Black/African-American and most of the rest of the population is Hispanic. When camping, I see primarily Caucasians and Hispanics, but almost no other subgroup. Camping is a poor man's vacation so it ought to bring out lower socioeconomic groups. When hiking, however, I see primarily Caucasians unless I'm hiking in the San Gabriel Mountains, and in particular, Mt. Baldy, where the mix is probably 60 Caucasian, 35% Asian (a large percentage of which greet me with heavily accented English) and maybe 5% Hispanic. When I go climbing, the population is nearly 100% Caucasian.
I haven't figured out why the Hispanics who I see camping, don't hike and why the Asians I see hiking don't camp.
I'm no demographer, so these statistics are surely flawed, but they seem to represent my own observations.
