"The Republican Party was just as scared of the Tea Party as the Democrats were. We just didn't know it at the time."