"Inference is founded upon obvious reasons. Regard to reputation has a less active influence, when the infamy of a bad action is to be divided upon a number, than when it is to fall singly upon one. A spirit of faction . . . will often hurry the persons of whom they were composed into improprieties and excesses for which they would blush in a private capacity."