"Of course, if one ignores contradictory observations, one can claim to have an "elegant" or "robust" theory. But it isn't science."

Next Quote

The more experiences and experiments accumulate in the exploration of nature, the more precarious the theories become. But it is not always good to discard them immediately on this account. For every hypothesis which once was sound was useful for thinking of previous phenomena in the proper interrelations and for keeping them in context. We ought to set down contradictory experiences separately, until enough have accumulated to make building a new structure worthwhile.

— Georg C. Lichtenberg