.... and you only deal with the Mt. Whitney Trail crowds on the way out.
The Trail Crest Exit Permit is required to use the Mt Whitney trail to exit via Trail Crest and Whitney Portal. It is the only place you need a permit to use a trail to exit, because lots of people were entering on other trails and all were exiting that way.
If you enter on ANY Inyo National Forest trail, the exit permit is required. It is only people coming in from the west side and Yosemite that can exit without the exit permit.