It sounds like you're mostly interested in elevation, so if the summit didn't pan out, just drop down about 100' in elevation (towards LA). There's any number of good spots, mostly shielded from the wind, particularly if all you need is a bivy site.
I was up there yesterday - it was still rather breezy. Some poor buggers were wearing shorts, no gloves, etc.
