Polling Observatory campaign report #3: All changed, changed utterly

DipticBy The Polling Observatory (Robert FordWill JenningsMark Pickup and Christopher Wlezien). You can read more posts by The Polling Observatory here.

This post is part of a long-running series (dating to before the 2010 election) that reports on the state of the parties as measured by vote intention polls. By pooling together all the available polling evidence we can reduce the impact of the random variation each individual survey inevitably produces. Most of the short term advances and setbacks in party polling fortunes are nothing more than random noise; the underlying trends – in which we are interested and which best assess the parties’ standings – are relatively stable and little influenced by day-to-day events. Further details of the method we use to build our estimates can be found here and here.

With just over a week to go until the general election, the campaign continues to take surprising twists and turns, not least the jaw-dropping projection of a hung parliament from YouGov for The Times.

When we first reported, shortly after the snap-election was called, the Conservatives held a commanding lead and Labour seemed to be meandering towards electoral oblivion. This remained the scenario at the start of May, though the scale of UKIP’s collapse in the polls was starting to become clear – with the Conservatives seemingly the main beneficiary. Just four weeks on, the change in Britain’s political landscape is remarkable – even if the likely outcome of the election (a decent sized Conservative majority at least) remains much the same. Labour has surged in the polls, now standing at 35.5% (up over eight points from 27.8%), while the Tories have fallen back from their initial bump after May called the election (at a still impressive 44.5%, down from the high of 45.6%). UKIP have continued to lose support at a rapid rate, with our estimates putting them at just 4.0% – less than a third of their vote in the 2015 general election just two years ago. The Liberal Democrats’ have also fallen back to just 7.8% (which would be below their catastrophic performance at the last election). Barring an even larger polling miss than occurred in May 2015, the political landscape of Britain looks like it will be redrawn in unexpected ways. There continue to be good reasons to be cautious about what the polls are currently telling us – due to the wide range of Conservative leads being shown by different polling houses and the possibility that Labour’s votes may stack up in seats in cities among younger and educated voters, where they tend to already hold large majorities, while falling away in marginal seats elsewhere.


UK 29-05-17 anchor on average

One of the features of our method is that it enables us to estimate the ‘house effect’ for each polling company for each party, relative to the vote intention figures we expect from the average pollster. That is, it tells us simply whether the reported vote intention for a given pollster is above or below the industry average. This does not indicate ‘accuracy’, since this will only be known on June 9th. It could be, in fact, that pollsters at one end of the extreme or the other are giving a more accurate picture of voters’ intentions. Indeed, in contrast to the 2015 election where there was convergence of the pollsters around the Conservative-Labour margin of zero, the most recent set of polls have shown Conservative leads ranging from as little as 5 points to as high as 14 points – outcomes that would have vastly different results in terms of a parliamentary majority for Theresa May.

In the table below we report the ‘house effects’ towards or against each of the parties for all polling companies who have recently conducted surveys. We, of course, estimate separate effects where the same company uses different modes (i.e. where Survation have fielded polls using both online and telephone surveys). We also (where possible) create ‘new’ polling houses where pollsters have implemented significant changes to their method and weighting procedures, though these are not always easy to date precisely. Nevertheless, the estimates give a picture of which pollsters tend to show higher numbers for which party, and thus are a handy guide for reading the latest polls with a dose of caution.

Our estimates reveal a range of house effects – and some interesting patterns too. It is first of all apparent that ComRes and ICM stand out as tending to report higher numbers for the Conservatives (+1.6 points and +1.4 points respectively) and lower numbers for Labour (-1.6 and -1.4 points). In contrast, ORB, Survation (online) and SurveyMonkey are at the other end of the spectrum — in tending to show support for the Conservatives lower and Labour higher than the industry average. Interestingly, Ipsos MORI and Panelbase show both parties higher – due mainly to their tendency to put UKIP much lower (in the case of Ipsos MORI this is a substantial 4.5 points).

House effects, by pollster

Pollster Mode Turnout filter Con Lab Lib Dems UKIP Green
YouGov Online Self-reported -0.3 (0.2) -1.0 (0.2) +0.9 (0.1) +0.4 (0.2) -0.6 (0.1)
ComRes Online Turnout model +1.6 (0.3) -1.6 (0.3) +1.3 (0.2) -0.6 (0.3) -0.3 (0.1)
Ipsos MORI Telephone Self-reported +1.2 (0.4) +2.0 (0.4) +1.5 (0.2) -4.5 (0.3) -0.3 (0.2)
Survation Online Self-reported -2.2 (0.5) +0.5 (0.5) +0.6 (0.3) +1.0 (0.4) -1.1 (0.2)
Survation Telephone Self-reported 0.0 (0.8) -0.4 (0.8) -0.3 (0.5) -0.9 (0.5) -0.3 (0.3)
Panelbase Online Self-reported +1.2 (0.7) +0.4 (0.7) -0.1 (0.4) -0.9 (0.5) -0.2 (0.3)
Kantar (TNS) Online Turnout model -0.6 (0.6) -3.2 (0.6) +1.4 (0.4) +0.7 (0.4) +1.5 (0.3)
ORB Online Self-reported -1.3 (0.6) +1.7 (0.6) -0.4 (0.4) +1.6 (0.4) +0.2 (0.2)
SurveyMonkey Online Unknown -0.8 (0.7) +0.9 (0.7) -1.9 (0.4) +1.1 (0.5) +1.6 (0.3)
Opinium Online Self-reported +0.1 (0.5) +0.7 (0.5) -0.5 (0.3) +0.4 (0.3) -0.5 (0.2)
ICM Online Turnout model +1.4 (0.3) -1.4 (0.2) +0.3 (0.2) +0.8 (0.2) +0.1 (0.1)

Pollsters have made many methodological changes since 2015, making it tricky to discern the causes of variation in these ‘house effects’. One notable feature of the methodology used by ComRes and ICM is the use of demographic turnout models to predict the propensity of individuals to vote. This has the consequence of down-weighting those respondents who have been less likely to vote in previous elections – giving rise to considerably lower Labour vote shares due to their current reliance on younger respondents and previous non-voters. In contrast, other firms such as Ipsos MORI and Opinium use self-reported likelihood to vote, giving rise to slightly higher vote shares for Labour. We will only know which of these adjustment procedures (if either) has been effective on June 9th, however.

While the Conservatives still hold a sizeable lead, the differences across pollsters could represent the difference between a huge working majority in parliament for Theresa May and an election that delivers few gains to the Conservatives contrary to all expectations.  Only time will tell who has got closest to the result.


Robert FordWill JenningsMark Pickup and Christopher Wlezien

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s