Have Pollsters Become Too Powerful?

The last time Peter Kellner forecast the result on the eve of a major British vote, he was accused of costing U.K. taxpayers tens of billions of pounds. For much of the Scottish referendum campaign, the outcome had seemed a foregone conclusion. Yet on September 7—eleven days before Scots voted for or against independence from the rest of the United Kingdom— a poll by YouGov, of which Kellner is president, suggested that the outcome was too close to call: The "Yes" camp was ahead, by the narrowest possible margin. YouGov put its lead at 51 to 49, well within the pollsters’ margin of error. The 307-year-old Union seemed in doubt. David Cameron, Ed Miliband, and Nick Clegg cancelled their commitments and travelled to Scotland bearing inducements designed to halt the Yes surge.

It has since been calculated that the promises the three party leaders made to Scottish voters—by way of amendments to the Barnett formula for allocating public spending to Scotland, Wales, and Northern Ireland—would cost the U.K. £45 billion over the next decade. And the Daily Mail blamed Kellner, Britain’s pre-eminent political pollster, for helping to run up the bill. Other polling companies had detected a shift in the public mood, but it was the YouGov poll that became the subject of debate and the focus of irritation for those who believe that polling has too much influence on politics.

On March 29 YouGov and Kellner again stirred up expectations when its poll for the Sunday Times, the first to come out after the Cameron/Miliband televised question-and-answer session, put Labor four points ahead of the Conservatives. If the six-point swing that the finding represents were reflected across the country on May 7, Labor would win “enough seats to come close to an outright majority, even if it loses badly in Scotland,” Kellner wrote.

The poll was out of line with that of YouGov’s competitors, however; a ComRes survey the same day put the Tories four points ahead. And YouGov polls later that week put the two parties neck-and-neck again. Yet the Labor surge, though brief, was “real,” Kellner told me. “There has plainly been a rise in Ed Miliband’s ratings [since the first leaders’ television event].”

Kellner is confident in his firm’s methods and in what its findings tell us. On the day of the Scottish referendum, YouGov went back to the people they had surveyed earlier in the week and asked how they had voted. This showed a slight but consistent shift from Yes to No. Kellner went on television at 10:30 p.m., saying he was 99 percent certain that the Better Together campaign had succeeded. It was brave to make a call so early, and a leading psephologist dismissed it out of hand. One joke went that if Scotland voted for independence, it would be a race to see who resigned first—Peter Kellner or David Cameron. Kellner was proved correct.

It is hard to know what impact the Scotland poll that showed Yes narrowly ahead had on the final result. Kellner acknowledges that YouGov’s figures sent the No camp into “a blue funk,” but argues that there is no way of assessing whether the party leaders’ response made a difference or not. The late swing back to No was not unexpected—people often “look over the cliff edge and then draw back in referendums,” he said.

Kellner, who is 68, with glasses and receding white hair, gave me another example of the way in which polling influences politics when I met him at YouGov’s offices near Old Street in east London, before Christmas. In the summer of 2013, when it became apparent that President Bashar Al Assad of Syria had used chemical weapons against his own people, Barack Obama asked David Cameron to support a U.S. attack against the Syrian regime. Parliament was recalled on August 29 to debate the proposal. Three days before the Commons vote, Kellner added a question to a YouGov poll asking whether people would support or oppose Britain’s participation in a military campaign. Fifty percent of respondents were opposed to British planes enforcing a no-fly zone over Syria or, if necessary, joining in the conflict.

The Sun published the results on a Wed­nesday. “That poll went round the world,” Kellner said as we sat in his office, a small, glass-walled cell in the corner of a much larger (and entirely empty) room. “It was the only poll ahead of the Thursday debate, in which it was accepted as a non-contentious fact that the public did not want the British government to take part in the operation.”

The government’s motion asking for MPs’ support for military action was defeated by 13 votes. “The Commons voted against, and Obama pulled back,” Kellner said. “So the Commons vote prevented, indirectly, the air strikes going ahead.

“Here’s the question I ask myself: Was our poll a key part of the chain that led to the attacks not taking place—and if it was, is that improper?” His answer is no.

(A similar question about political influence had been asked three years earlier, in 2010, when one of YouGov’s polls showed that Ed Miliband had erased a large deficit and overtaken his brother, David, in the race for the Labor leadership. Some on the left believe the survey swung the momentum in favour of Ed.)

Kellner’s allegiances are well known—he has been a member of the Labor Party for 40 years and is a former political editor of the New Statesman— and yet, as a pollster, he aspires to political neutrality and insists that he personally does not want power or influence. “If people take notice of what I say, it’s because of the public attitudes I analyze, not because of my personal opinions,” he said in an email exchange following our meeting. Such modesty is faintly unconvincing: Kellner is frequently on air, interpreting the data in memorably pithy ways. His friend Andrew Cooper, the Tory peer who co-founded the polling company Populus in 2003 and served as Cameron’s director of strategy between 2011 and 2013, said Kellner is still the best political commentator in Britain.

He is also well-connected in Labor circles, being married to the Labor peer Catherine Ashton, whose term as the EU’s high representative for foreign affairs and vice-president of the European Commission ended in November. He concedes that his contributions have some influence on political debate, though he expresses this in a characteristically measured and thoughtful way. He describes himself as a believer in Edmund Burke’s notion of representative democracy—which does not regard an MP as a delegate required to enact his or her constituents’ will but rather as a representative entitled to interpret it according to his or her judgement and conscience—and he thinks that polling data can “enrich” the conversation between politicians and those they serve. George Gallup, the father of modern polling, believed that it would create “a truer democracy” by making governments “more efficient and responsive,” and Kellner reckons that the data is now available in sufficient quantities to realize Gallup’s aim.

Immigration is a case in point. “Plainly, a big majority of people think we have far too many immigrants,” Kellner said. “But once you start digging into it, you find that belief is related to a whole range of other things—it’s to do with the recession, insecurity, worries about the future—and a lot of it is nonsense.”

A recent Ipsos MORI poll showed that the average Briton believes that immigrants make up 24.4 percent of the population, when the figure is 13 percent. Kellner notes, too, that people think the proportion of the population on Jobseeker’s Allowance is “vastly higher” than it is. The confusion informs the mainstream parties’ illogical response to the challenge of Ukip. “As someone once pointed out, the mainstream parties are saying: ‘Ukip is right—don’t vote for them.’ They should be saying: ‘Ukip is wrong— don’t vote for them.’”

More sophisticated use of polling data, Kellner argued, would allow politicians to “dig into the roots of people’s hostility” and address the causes, rather than symptoms.

Andrew Cooper of Populus takes a similar view. He says that politicians use polls in the way the CEO of a big business might: “to understand in detail what people think about the world, to be clear about the problems and concerns in their lives and to understand the most effective way they can make their case.” No politician, Cooper told me, is governed by the data that tracks shifts in public opinion—not least because it shows that “most people, on most issues, don’t have a settled view.”

“If you did try to figure out what to do with reference to public opinion, you would end up completely becalmed because voters think nothing of holding two contradictory opinions, or changing their mind on a dime,” Cooper said. Besides, the public wouldn’t respect politicians who seemed to respond to their own mood too closely. What Kellner calls the “paradox of polling” shows that people want it both ways. “They want politicians to listen to the voters and do what they say, but they also want politicians who know their own minds,” he said. “They want them to be tough and sincere, but they want them to do, toughly and sincerely, what they want them to do.”

Kellner explored the subject of people’s perception of politicians in more detail in a 2012 booklet, Democracy on Trial, which collected polling data that shows what voters “really think of parliament and our politicians.” It is not a story of universal contempt: Only 15 percent of voters think Westminster succeeds in “representing the interests and wishes of people like you” and 62 percent believe that “politicians tell lies all the time”— yet 63 percent agree that, “for all its faults, Britain’s democratic system is one of the finest in the world.”

It is a sentiment that Kellner appears to endorse. In 2009 he published a book called Democracy: 1,000 Years in Pursuit of British Liberty, consisting of annotated extracts from celebrated speeches on freedom, from the laws of William the Conqueror to recent considerations of liberty and the constitution. It was dedicated to his father, Michael, on the grounds that it “contains some of the reasons why he chose to become British.”

Michael Kellner was an Austrian Jew, born in 1920, who emigrated to Palestine with his family in 1938, after Kristallnacht. “Most kids of that age in Palestine at the time either joined the Irgun and fought the British or joined the British and fought the Germans,” Kellner said. It is one of many binary propositions that punctuate his conversation— the product, perhaps, of a habit of seeing the world through the prism of either/or questions. His father had never been to Britain, but he joined the British army and was sent to Greece, where he was wounded and captured in the German invasion of 1941. Michael Kellner spent the rest of the war in a German POW camp, and when it was liberated he went to Britain, where he stayed for the rest of his life. “Britain’s tradition of liberty and tolerance was important to him,” Kellner said. “He became a British subject out of choice, because of what Britain stood for.”

He then became a Labor Party councillor in County Durham. Kellner says that growing up in a politically interested household shaped his career. He won a place to study mathematics at Cambridge but switched to economics and statistics because it had more practical applications. His first job, as a “specialist statistician,” was with the Sunday Times and he stayed there for eleven years before joining the New Statesman and working as a columnist for the Independent, the Evening Standard, and the Sunday Times. Later, he set up a “polling relationship” with the Independent and Newsnight, and his interest in data and statistics has remained constant throughout his career. “I have never enjoyed or been very good at the lobby gossip side of politics,” he said.

His own political career was brief; he stood in elections for Westminster City Council in 1978 although, given he was a candidate in Belgravia, he did not have “great hopes” of winning. Yet his party links were an advantage when YouGov was set up because its two founders had strong connections with the Tories: Nadhim Zahawi is a Baghdad-born businessman who became a Conservative MP in 2010, and Stephan Shakespeare set up the ConservativeHome website, which he later sold to the pollster Michael Ashcroft. It was “useful” to have someone known to be involved with Labor, he says. The Lib Dem peer Navnit Dholakia was another early recruit, for the appearance of neutrality is essential to any polling organisation—and especially one pioneering a new way of measuring public opinion.

When I arrived at the YouGov offices Kellner handed me a copy of the front page of the News Chronicle, dated Wednesday July 4, 1945. It reported that Gallup’s second “Election Poll” had “tested the voting intentions of a representative sample of men and women voters covering 195 constituencies” and found that Clement Attlee’s Labor Party had a six point lead over Winston Churchill’s Conservatives. Churchill’s status was such that many dismissed the poll. Even the News Chronicle, which had commissioned the research, had its doubts. Yet it proved correct: In fact, it underestimated the margin of victory—Labor won 47 percent of the vote as Gallup predicted, but the Conservatives polled six points lower than expected, and Labor won a momentous victory with a majority of 146 seats.

It was the first time that polling had been available in a British general election, and Gallup was the only company providing it. Polling had originated in the United States in the early part of the nineteenth century. In 1824 a newspaper in Pennsylvania asked people which presidential candidate they preferred, and on the basis of their replies it predicted that Andrew Jackson would win more votes than John Quincy Adams.

The first national poll came nearly 100 years later: In 1916, a magazine called the Literary Digest mailed out millions of cards to its readers asking which way they intended to vote and forecast Woodrow Wilson’s victory on the basis of the returns. The Literary Digest also picked the right candidate in several subsequent elections, though it failed to anticipate Franklin D. Roosevelt’s victory in 1936 because a disproportionate number of its readers were Republican voters. It was during this very election that George Gallup conducted the first demographically representative survey.

Since then, opinion polling has become an accepted part of political and commercial life. Kellner offers a summary of what he calls the “disasters:” Pollsters failed to detect the late swing that caused Thomas Dewey to lose to Harry Truman in the 1948 U.S. presidential election, and most did not predict that Ted Heath’s Tories would overturn a Labor lead in the 1970 general election—the honorable exception being MORI, the company set up the previous year by Robert Worcester. But the most egregious failure came in 1992, when John Major won a majority of 21 seats despite trailing in the polls throughout the campaign.

That result is usually attributed to the “shy Tory factor,” which suggests that people were ashamed to admit they were planning to vote Conservative, but Kellner believes there was a more substantive problem. “All the pollsters used the same target data for social class—and it was wrong. The election had the misfortune to take place a few weeks before the 1991 census data became available, and when it came out, it showed that Britain in the 1980s had moved far more from working class to middle class than anyone thought.” Reweighting the raw data to the right class makes a significant difference. Kellner believes that Labor was never in the lead: The Conservatives were ahead from the day John Major became prime minister in November 1990.

Regardless of the causes, the failure forced the pollsters to change their methods. Until then, most polls had been conducted face to face. During the 1990s, telephone polling became the norm but by the time YouGov set up in 2000 a new form of communication was available. “The internet was the game-changer,” Cooper said.

Given that approximately half the population had access to the internet, people wondered how you could get representative samples, and Kellner couches his explanation in another neat formulation. “We did two things—one that worked and one that didn’t. The first thing we did was explain in detail how we did it. That didn’t work: People’s eyes glazed over. And the second thing was we kept getting things right.”

YouGov’s early successes included the 2001 election, when it estimated Labor’s lead more accurately than most other companies, and the first Pop Idol final in 2002, when it predicted that Will Young would beat Gareth Gates by 53 percent to 47.

Even so, online polling was still facing “very real problems” when Cooper and Michael Simmonds set up Populus in 2003. Kellner may have worked out how to make the weightings and adjustments that would secure representative samples, Cooper said, but that didn’t mean that anyone else had. “Some big companies spent a long time trying to replicate it, but couldn’t figure out a way to do it. The same basic theory and weighting used in telephone polling didn’t work when applied to online polling—which left people wanting to move into it staring at a blank sheet of paper.”

The debate about the merits of respective methods still goes on, even now, when the proportion of people online is higher than the proportion of people with a fixed-line telephone and the response rate for some telephone polls has fallen as low as 20 percent. “The argument about online panels is that to some extent they are self-selecting, according to who signs up, but telephone polls are now on the way to becoming self-selecting,” Kellner said. Ipsos MORI, which produces a monthly Political Monitor survey for the London Evening Standard, says it is “methodology-neutral,” meaning that it tries to use the method most suitable for its clients’ needs. It does a lot of online work but still maintains that, for surveys that need to “cover as wide a sample as possible, for example of older people, who are more likely to vote,” telephone polling gives “the most consistent results with less need for extra weighting.” Yet all the pollsters agree on one thing: No approach is without drawbacks. “Different polls give you different answers, and you can find examples in both camps of the more or less accurate,” says John Curtice, professor of politics at the University of Strathclyde and Britain’s most prominent psephologist.

Ipsos MORI, which was formed in 2005 from the merger of Robert Worcester’s MORI and the British branch of the French company Ipsos, remains the U.K.’s largest research firm. But YouGov is growing fast; it expanded into America in 2007 when it bought the U.S. company Polimetrix and now has 25 offices worldwide, in continental Europe, Asia, and the Middle East. Its ambition is clear, and both its strapline (“What the world thinks”) and mission statement (“It is our ambition to supply a live stream of continuous, accurate data and insight into what people are thinking and doing all over the world, all of the time, so that companies, governments and institutions can better serve the people that sustain them”) have a slightly cultic air. The firm now has a “panel” of more than 600,000 members in the U.K. and three million worldwide, from whom it selects a weighted sample of respondents for each poll.

Given its aims, I was surprised by how quiet its headquarters seemed—the building near Old Street looks more like a fashionable block of flats than an office. Yet I later realized that the unattended desks in the room beyond Kellner’s office were further evidence of YouGov’s ambition: It had just taken over two more floors in the block, partly to accommodate future expansion.

Kellner estimates that 90 percent of YouGov’s work is market research and only 10 percent politics, though in his particular case the numbers are reversed. He used to have a 1.7 percent stake in the firm which, the Financial Times reported in 2007, was “worth about £2.3 million.” He sold the shares on the advice of the cabinet secretary, however, after his wife became leader of the House of Lords in Gordon Brown’s first government in 2007. John Humphrys, the BBC journalist and presenter of the Today program, who was also given shares in YouGov when it was set up, remains a shareholder and still writes a column on its website. Kellner calls himself a “wage slave” although, if the FT’s estimate is correct, he would have become a millionaire when he sold his shares. “Few journalists have moved into new territory with quite such spectacular results,” said a profile in the Independent in 2006, under the headline “Ten out of ten hacks really envy him.” It is not he who makes the company money, he adds: but he will not reveal what newspapers pay for a poll, saying only that it’s less than it was 20 years ago. Industry insiders suggest a figure of between £500 and £1,000 per poll if commissioned as part of a long-term contract. Indeed, political polling is often a loss-leader, and many companies do it less for profit than for publicity.

When I spoke to him before Christmas, Kellner made few predictions, beyond saying that he expected the turnout to go up again in the May election, as it did in 2005 and 2010 after the collapse of 2001 when the inevitable Labor victory generated little tension or ideological argument. Now, when we speak again, he still believes that we “shall still end up with a Conservative lead in the popular vote”—without the party securing an outright majority—“unless some specific event or events give Labor a boost.”

Yet in a race this close, small margins can have a big impact: The spike for Labor in late March briefly raised the prospect of a minority single-party government, like the one Harold Wilson formed in 1974, leaving the other big parties with the choice of allowing it to govern or risking public displeasure by voting it down, triggering a long period of uncertainty. When we first spoke, Kellner suggested that “the ten days after the election might be as tense and unpredictable as the ten days before.” Now it seems that the uncertainty could last for months.