Discretion being the better part of valor, you did the right thing. I've had to make that exact same call - turn around a mile from Whitney's summit - due to conditions that I didn't think were safe for my wife and daughter. It's a tough, tough thing to do, especially when you're feeling fine. As the saying goes, the summit is optional but the parking lot is mandatory. Good job!