Bankers Abandoned Obama — But the Rest of the Rich Held Surprisingly Strong