You should be fine. I wouldn't stress it.
I'm not sure where you saw the numbered sites. If they were in the trees, close to the HSC apparatus, those are the tent cabin slots.
The backpacker's camp at Glen Aulin is another 100 odd yards to the north, more in the open, on a little granite bluff east above Conness Creek. You hike through the HSC to get to it. A quick skip to the loo when your kid needs to do the business.
I remember some numbers up there, but it was more Camp 4 than Upper Pines. Like you, I was there off-season. It was real nice, but I had it all to myself.
I remember going to the can. Struck me odd it was open after the HSC was packed up. I don't think it was a flush toilet - that wouldn't make sense. More like one of those Little Yosemite composting set-ups. Since I can't remember, it probably wasn't too gross. Otherwise I would have pooped in the woods.
"A more private spot a few minutes hike away" is kinda against the rule book. If you have a Glen Aulin pass, that's where you are allowed to camp. There's another "pass-through" pass that doesn't have that perk. The spot is real popular, they want to minimize damage. There are some other nice spots in that area, but you need to go another half mile or so and I ain't ain't posting that all random on the interwebs.
It's a nice fun hike, hope both of you enjoy it.
