"If a man's public record be a clear one, if he has kept his pledges before the world, I do not inquire what his private life may have been."