So, as promised, I wanted to make a few quick points in defense of Summers:
First, with regard to his speculation that innate gender differences may partly explain why women are underrepresented in elite science and math positions, it was unquestionably a boneheaded thing for the president of Harvard to say, and not a particularly constructive hypothesis for him to muse about. (Even if true, then what?) It's worth mentioning, however, that Summers' comments were motivated by his interest in alleviating gender inequality in these fields, which is how he prefaced them at the time. The whole point was to initiate a conversation about why women might be underrepresented.
If anyone doubts Summers' commitment to gender equality--as opposed to his effectiveness as a spokesman on the subject--they need only look to his tenure as World Bank chief economist, where one of his main crusades was expanding girls' access to education. As a 1995 Washington Post article put it:
In 1991, with the backing of Bush administration officials he had helped campaign against in 1988, Summers was named chief economist at the World Bank. He pushed market-oriented development policies, championed the need for privatizing state-owned enterprises and was a tireless advocate of investing more funds to educate young girls.
Which brings us to the next line of attack on Summers: The infamous World Bank memo in which he made the case for some transfer of pollution from developed to developing countries. The economic logic of the idea was pretty airtight; the moral logic was pretty abysmal. But that's the whole point. Summers says he intended the memo as a way of provoking questions about certain environmental policies, not as a policy proposal in itself, which is why it was only distributed internally (until it was leaked, of course). This strikes me as highly plausible, since Summers came from a background--academic economics--where it's extremely common to raise analytic points by way of far-fetched thought experiments. (These kinds of hypotheticals were everywhere during my two years of grad school.) And since, as Summers said at the time, "No sane person favors dumping toxic wastes near where anybody lives, or thinks the places could be made better off with more toxic wastes."
But, of course, there's no need to take Summers' word for it. Anyone concerned that he'd start shipping toxic waste to sub-Saharan Africa were he to become Treasury secretary need only examine the year-and-a-half when he actually was Treasury secretary, during which his record was remarkably free of this and other non-sane policymaking.
The broader point is that, if you don't like what Summers actually did at Treasury, or what he's actually urged other policymakers to do, then, by all means, question and criticize. (Though bear in mind that some of his thinking has evolved on that stuff, too.) But to bash him for things he never intended to propose, or, in the case of the gender comments, for a ham-fisted discussion of a problem he and his critics both want to address, seems to miss the point. Given the economic hole we're in, it'd be a shame to rule out one of the most brilliant policymaking minds around on the basis of these side-issues.
Update: A commenter wants to know more about Summers' policy evolution. The place to start is here, where you can read his most recent Financial Times columns. I'd also recommend this excellent column from last year about wage inequality.
Be sure to read Jonathan Cohn's thoughts on Larry Summers here.