You could start about 10:30 or 11 PM and get to the wilderness boundary at Lone Pine Lake at midnight, where your day hike permit requirement technically kicks in.
I have been told otherwise by the rangers. They stated that dayhikes begin at the trailhead. Your permit is trailhead to summit to trailhead. Therefore, you cannot start on the trail until midnight (even though you can hike as far as the boundary anytime). Sounds contradictory, doesn't it? At any rate, four of our five dayhike summits had 10PM starts. There are no rangers on the trail that late looking for early starting scofflaws.