Does everyone in the group need to start the hike at the same time?

Those that are planning on summiting want to start at 3am. The others don't want to start until 5 or 6.