Welcome, ARK. Posting here is welcomed. Being this is Sunday, it is not clear what date your permit is for.
Unfortunately, Inyo National Forest has all sorts of rules in place to prevent scalpers from getting reservations and then selling them to highest bidder. And part of that includes rules that the person named on the reservation must pick up the permit. So swapping with someone else isn't really feasible.
If you really want to hike later, there are usually unused permits due to no-shows and people dropping out of groups. Mid-week are more available, but even on weekends, you can get them. Check the
unused permits pages for prior years.