It’s best to stay together, but there is no rule requiring it. If you meet a ranger on the trail, and the leader and the permit are elsewhere, it would come down to whether the ranger believes your story. I don’t think they would issue citations unless they thought you were trying to pull a fast one.
A story that may be instructive: Many years ago, I climbed Mt. Shasta with a group of friends. Marek thought the permit/pass system was B.S., and didn’t bother getting one. Everybody else played by the rules. A ranger showed up while we were hanging out on the summit. He promptly asked Marek for his permit/pass. Marek gave the ranger some lip. The ranger gave Marek a ticket. The ranger never asked anyone else for his permit/pass. Someone asked the ranger why he didn’t ask anyone else for his permit/pass. The ranger said that Marek was the only one who looked guilty.