The Polling Observatory Forecast #1: Lessons for 2015 from polling history

With a year to go, the Polling Observatory team launch their forecast for the 2015 general election…

Starting this month, the Polling Observatory team is joined by a new member: our old friend and colleague, Christopher Wlezien of the University of Texas at Austin, who will be helping us to produce a long term forecast for the 2015 General Election, using methods we first applied ahead of the 2010 election. Our method involves trying to make the best use of past polling evidence as a guide to forecast the likeliest support levels for each party in next May’s election, based on current polling, and then using these support levels to estimate the parties’ chances of winning each seat in the Parliament. In this first post, we introduce the poll-based element of this model; in later posts we will introduce and explain the seat-based element.

The past is, of course, an imperfect guide, as voters and parties change and each election is, to some extent, unique. However, this does not mean past polling tells us nothing. On the contrary, as we have shown in previous research (non-gated version here), careful analysis of past polling reveals common underlying trends and patterns in British public opinion. It is these trends and patterns which we use to estimate our forecast of the likely path of public opinion. In 2010, our method fared relatively well against the alternatives on offer, getting the overall outcome of a hung Parliament with the Conservatives as the largest party correct, and coming quite close on all three parties’ seat totals. The forecast performed well relative to others forecasting models published by colleagues, notably beating celebrated polling oracle Nate Silver’s prediction for the British election.

The method works in the following way. Thanks to the Fixed Term Parliaments Act, we know how many days remain until the next general election. For any given day, we can use all the polling data from past general election campaigns to estimate two things: how closely the current polling is likely to reflect the election outcome, and which direction public opinion is likely to move in between now and election day. We do this for each of the three main parties separately, seeing what polling history can tell us about their respective fates.

This is one of the simplest possible ways of forecasting how elections will turn out, and it leaves out an awful lot. We do not look at the impact of leader approval ratings, the objective state of the economy, or public economic perceptions – things which other models have used as forecasting tools. We simply take the best possible estimate of where public opinion is today (an estimate constructed using our poll aggregation method) and ask: How informative does history suggest this estimate will be as a prediction of the next election? Where does history suggest public opinion will move between now and election day?

Our method starts by considering the systematic and predictable ways in which the public’s intention to vote for parties varies over the election cycle – based on past evidence. Some shifts in public opinion are impossible to anticipate, such as in reaction to shocks or events. Other dynamics may be more predictable, however; for instance that pre-election poll leads tend to fade or that parties may benefit from ‘campaign effects’ (such as due to increased attention during the official election campaign). To forecast the election day vote share, we need to know the relationship between vote intention t days before the election and the vote share in past elections. Therefore, the first step in our forecasting procedure is to estimate the relationship between vote intention and vote share through a series of regressions – for each of the main parties – for each day of the election cycle. To do this, we use all available polling data since 1945 (more than 3,000 polls) – across seventeen elections. This allows us to determine both how well the polls predict the final outcome on a given day (unsurprisingly the polls become more predictive the closer we get to the election), and to determine whether support for a party is above or below its long-term equilibrium level – and is likely to gain or fade as the election approaches.

Our past work, using this very simple method, suggests it has some useful lessons to teach us. Firstly, we find that the predictive power of polls evolves differently for different parties: polling of Conservative support becomes steadily more predictive from over a year out, while for Labour and the Liberal Democrats, the main improvement in accuracy comes in the last six months. Secondly, we find that support for Labour, the Liberal Democrats and the Conservatives tends to “regress to the mean” – if current support is above the long run average, it tends to fall; if it is below, then it tends to rise.

While the daily regressions teach us a lot, there is a fair bit of “noise” in the regression estimates, as each regression is based on 17 data points (elections). However, although each regression is based on only 17 data points, we have a separate regression for each day and we know that the estimates for one day should only differ so much from the estimates from the days immediately before and after it. Therefore, the second step in our forecast procedure is to reduce the noise by smoothing the regression estimates over time. The procedure we use to produce the smoothed regression estimates is similar to the procedure a sound engineer would use to remove static from a sound recording.

The third step in our forecast is to use the smoothed regressions estimates to produce forecasts by plugging our daily estimates of vote intention into the smoothed regression equations. These vote intention estimates come from the same poll aggregation method we use in our monthly Polling Observatory updates – a Bayesian averaging of the polls.

In the final step, we pooled the forecasts over 30 day intervals, so that the new pooled forecast on each day is a Bayesian averaging of all forecasts up to 30 days prior to that day.

The predictions we get by applying these methods to current polling since March 2011 are shown in the figure below. Our forecast model has consistently predicted a very close result – the Conservative vote share is expected to recover from its current level of around 32%, rising to around 36%, within half a percentage point of Labour, whose poll share is not expected to change much from current levels. In vote share, the result is close to a dead heat – the Conservatives are currently forecast to have 36.1%, and Labour 36.5%. The Liberal Democrats are forecast to recover some ground from their current polling position, but still put in their weakest performance in decades, with a forecast vote share of 10.1%.

Forecast 01-05-14 cropped

Forecasting vote shares can only take us so far, however. Westminster elections are won and lost in 650 separate battles for constituencies up and down the country. The aggregate vote shares are only an imperfect guide to the likely distribution of seats – our current forecast of a 0.8 point Conservative lead in vote share, for example, would be likely to produce a Labour seat lead. In future posts, we will employ the second part of our forecasting model to translate these vote shares into seat shares.

The seat-based section of the forecast also provides us with a mechanism to examine two of the big unknowns in our forecast – how the Liberal Democrats will perform after their first term in government for generations, and the impact of UKIP. Neither has any historical precedent, so we cannot model these effects in the historical part of the model. However, we can take an alternative approach, taking the baseline predictions from our historical model and applying different scenarios in the seat based part of the model. This will give us some sense of how sensitive the likely outcome is to changes in the fortunes of the two smaller parties. We will explore such scenarios in future posts.

Robert FordWill JenningsMark Pickup and Christopher Wlezien.

15 thoughts on “The Polling Observatory Forecast #1: Lessons for 2015 from polling history

  1. […] new British Election Study site here and the Polling Observatory’s new election prediction here. The election prediction by Rob Ford, Will Jennings, Mark Pickup and Christopher Wlezien takes a […]

  2. Interesting fact: on a uniform swing, that would mean Labour would be just 2 seats of an overall majority.

  3. Are those 95% confidence intervals? I’d take even money for each party that their final vote is outside the range indicated.

  4. I like the bottom up data approach – but it is also worth looking top down.

    The 70 year trend for the two main party combined share of the vote has consistently been down. The only difference in the 2015 election is that we now have an incumbent third party which might help the two party share of the vote (this effect may be annulled though by UKIP). It is a fair assumption that 88% of Total UK vote will go to the two main parties plus Liberal Democrat and UKIP.

    No incumbent has increased its share of vote since 1974 (and that was a six month Government not a five year). Labour is now the natural party of government scoring 40% share as recently as 2001 (after the Iraq War and credit collapse their share fell to 29% in 2010). If we assume there is no increase in the two party vote: 66% there is 22% available to Liberal Democrats and UKIP. A good guess would be UKIP 10%, Liberal Democrats 12%.
    The 2015 election is different, because there are two incumbent parties, but I think it is inconceivable that the two main parties will increase their combined share of the vote by more than 3% (all polling indicates significant numbers of voters unimpressed with the two main parties). Labour will recover votes lost in 2010 if only because of fall in Liberal Democrat share. This makes it very difficult for the Conservatives. The best scenario is Conservatives getting 35% and Labour 34% a net gain for the two main parties of 3% (leaving 19% for the 3rd/4th parties). Where this 19% goes is the interesting issue and how it will translate into Seats. Liberal Democrats highest share would be 14% with UKIP getting 5%. On universal swings this would result in +16 gains Tory and +12 Labour but tactical voting will mitigate Liberal losses to max 15 seats (25% total) and may elect maximum 2 UKIP seats.

    If this scenario is right, and it seems reasonable, there are only approximately 13 seats available as net gains for the two main parties combined. Let’s assume the Liberals hold 42 seats and lose 15. Based on the most marginal seats 8 of these would go to the Conservatives and 7 to Labour. A wipeout for the Liberals would only benefit the Conservatives over Labour by a few extra seats which means that the main battle is between the two main parties. This is where it gets problematic for the Conservatives: they will need to hold seats against an inevitable increase in Labour vote. The best scenario is them losing 39 seats to Labour (using with a gain of 8 from the Liberals this would still leave them with a net loss of 31 seats leaving Labour as the main (if not majority party).

    The top down approach points to only two possible outcomes: Labour as largest party (with majority) or Labour as largest party (without). A bottom up approach may suggest there is a third possible outcome but this flies in the face of political reality.

  5. Francis Maude is having a terrible time from some of his core supporters because of the possibility of fracking in his Horsham constituency. I think there are another 35 or so Tory MPs similarly affected by the problem. Has anyone done any analysis on how fracking might impact the GE result? Ditto for the Heathrow matter.

  6. What are the error bands meant to be? 50% confidence intervals perhaps?
    There is no way that any prediction of votes at the GE can have 90% confidence intervals of less than +/-5% at this stage.

Leave a Reply to CastIron Cancel reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s