Libor Mk2? Were the betting markets on the EU Referendum fixed by The City to manipulate FX markets?

Wild political and economic times breed paranoia and conspiracy theory.  Right now, people are saying the Sunday Times is sniffing around about a big story.  A gigantic story.  Elements in the City attempted to rig the relatively illiquid betting markets on the day of the EU Referendum.  They wanted to send Foreign Exchange markets the wrong way – i.e. strengthen the Pound – so they could sell it at high levels and make a killing when it collapsed with a Brexit vote.  Their losing outlay on Betfair was modest compared to what they could win on the vastly more liquid FX markets.  Is this true? 

Were city traders political villains?

Allegations of market manipulation at Betfair in particular had been circulating on Twitter for weeks prior to the vote.  Arch peddler is a bête noire of this blog, Professor Leighton Vaughan Williams (@LeightonVW), Director of the Betting Research Unit and Political Forecasting Unit at Nottingham Business School  – although he suggested manipulation was in the other direction.  He insinuated Leave were being ‘bigged up’ in the markets, and merrily revealed how he was hoovering up value bets on Remain. 

Just hours before the Brexit result became known on June 23^rd, 'Remain’ traded as short as 1/10, a 90.9% probability that Britain would stay in the EU.  Were the public sucked into believing that Remain were winning, hands down?  Was the social media narrative of a failing Leave campaign - on the crucial day - a bogus construction of some pernicious speculators?  Like lemmings, did punters place losing bets on Remain, thinking it a slam dunk certainty?  One £100,000 bet was reported at 1/10 from a member of the public. 

Far more seriously, did the actions of some corrupt traders influence how people voted?  Was the dark hand of the City attempting a more audacious plot, beyond winning a few quid and queering capital markets, to send voters the wrong way?  News of the odds were public as voters made that thoughtful trudge to the polling booth.  With a Remain victory looking like a near certainty, perhaps voters either wanted to be on the winning side, or felt some new assurance in a Remain choice, given their fellow voters were for staying in, in the very hours of casting their ballot.  If the suggestion of betting market rigging is true, this could be a Libor Mk2 scandal with political knobs on.  The vote for Leave might have been larger.  Our political future could have been deranged.

For all the naysayers, betting markets have a credibility in the public mind.  There is good reason for trusting those bookies.  This blog believes that markets should usually be our most reliable guide to future political outcomes, in fact any sort of future outcome. 

Unlike answering an opinion poll question, the act of staking personal cash can be a tax on stupidity - if you lose.  Bettors pay more attention.  The betting market is like a vast information storage system.  It draws information from every nook and cranny of the public conscious.  Every speech, every interview, every slip, every piece of punditry, every economic indicator, every pub conversation, every grating piece of’ Vox Pop’ from the BBC.  In it goes into the mixer.  And yes, dear old polling goes in too.  It’s trusted.  But maybe less so now.  The verdict on polling's value gets mashed, whisked, sieved and weighed with all other indicators.  A mass of punters attempt to sort the wheat from the chaff, motivated by the power of self-interest.  This is the fullest and most reliable aggregation of our public collective wisdom. 

No one television commentator could compete with it.  During the campaign, Laura Kuenssberg et al. should have just read out the moving odds for Remain and Leave, with their implied probabilities to make it really simple, and let it rest there.  That would be our best guide to what was going on.  Unfortunately the professional journalists don't talk odds much, particularly the lofty TV ones, it usually gets in the way of their own take.  They also loathe betting as a dirty business, probably thinking they have a responsibility NOT to mention it.

But what if this actually beautiful process of public judgement making, political betting, was being buggered by nefarious capitalists from the City?  Did it happen?

The short answer is that only Betfair and the bookmakers will know who was staking what.  I worked at Betfair and others.  They know.  Just as they have been excellent in addressing match-fixing questions, the onus is on them again to tell the public what was going on.  Prior to the investigation however, I will make two points in defence of betting markets.

Firstly, this is indeed a mad and paranoid story.   The problem with it – as a conspiracy – is why would a small group of traders waste probably hundreds of thousands of pounds, probably millions, to provide a indivisible benefit for many other City speculators aware of what was going on.  How was it all organised?  And how could they be so sure that Leave would win?  The final polling was dire for Leave.  Word has it the City ran its own exit polling, but decent exit polling on a one-off Referendum is beyond the capabilities of even the best British political scientists; not just hard but hideously expensive too.  Any investigation needs to examine whether there really was exit polling conducted privately, and what the results showed.

Secondly there is a lovely and even more paranoid theory to counter the market-rigging one.  Bookmakers have been getting increasingly peeved off about the Murdoch press, particularly the heavyweight Times and Sunday Times, running endless stories about problem gambling.  23 year old accountants jumping to their death after losing money playing online poker, and communities being destroyed by fixed odds betting terminals.  It's bad copy.  I’ve heard rumours – and they are just that – saying Rupert Murdoch is trying to weaken the existing industry prior to introducing his very own betting service in association with The Sun – “Sunbets”.  As yet, Betfair has largely escaped the campaign, but this new story isn't great for them.  

Personally, I don’t believe this particular conspiracy for a moment.  If the Sunday Times run the story they will believe it’s true.

Without pre-judging any possible future inquiry – and leaping forward several steps into the future, this is what I think actually was wrong with the betting markets – although I do not know for sure until Betfair and bookmakers reveal more information.  ‘Wrong’ is also a matter of degree.  Just because Leave drifted to 8/1 on polling day doesn’t mean that the market got it wrong, any more than the hugely liquid Grand National market got it wrong when Rule the World romped home at 33/1.  Markets aren’t wrong just because an outsider wins.  They are only wrong when outsiders win repeatedly and the fair share of favourites fail to win in their turn, at a rate implied by their probability expressed by odds.

To assess the value of political betting as a predictor of political events (against polling), the researcher must use long-term data, i.e. a large number of occurrences.  A large body of global academic research that shows that betting wins in the long run, and is indeed devastatingly accurate, material which I will gladly share.

But a nagging trend has indeed developed of outsiders winning political betting contests of late.  It is interesting.  Think of the 2015 General Election, not just the surprise Tory Majority, but the extraordinary performance of the Scottish Nationalists.  Ladbrokes may have lost up to £2m on that election, and William Hill reported a £1m loss at their official results.  Think also of the remarkable victory of Corbyn, once priced at 1000/1 on Betfair.  And Trump.  All losers for the bookies.

Bookmakers seem to be increasingly embarrassed – looking weak.  They have started to produce some proper hogwash to explain these results.  Worse, they have turned on their punters, their very own customers, to excuse their prices.  This is the official Ladbrokes explanation for the failure of their markets on the EU Referendum – and it's sad a once great company has descended so far:

 “there’s a huge amount of wishful thinking going on in people’s brains when they’re trying to assess the probabilities of these results”. 

So yes, their punters are stupid.  Ladbrokes’ founder, Cyril Stein, would be horrified.  Unfortunately, this sad explanation does not help explain why Remain was as short as 1/10 on the day of voting – assuming that their wishful thinkers were the more emotional (less rational) Leave punters.   Another now more complicated explanation now comes into play from Ladbrokes, equally damning of their own markets.  The problem Ladbrokes says, was a majority of (“wishful”) punters bet on Leave, but the really big bets came in for Remain, and the majority of the staking.  Again, Ladbrokes present a picture of their markets working badly and their punters unable to react correctly to prices.  We are also not told the vital information:   What was their company position on the main EU Referendum result, each day and overall?  Of course we would expect more money to be staked on an odds on favourite (Remain) than an outsider.  Despite this, Ladbrokes could still be taking a position with Leave, offered at longer odds.  

Were they with Leave or Remain?  Can the pricing of the Referendum better be explained by the bookmakers than the punters, including from the alleged nefarious City types? 

One thing is for sure, we know that Ladbrokes fancied Remain, revealing in the Times Red Box expert survey a heady prediction for the ‘inners’ of 57.01% (they got 48.11%).  I am guessing that much of the explanation why Leave was always too long in the betting – as I repeatedly mentioned during the campaign @mincer and see this blog passim – was that the William Hill and Ladbrokes wanted to bet against Leave, themselves.  Not content to eek out a nice little earner on a vast turnover by jobbing the market (i.e. balancing an equal profit on Leave and a profit on Remain), they wanted a little gamble of their own.  When the money came in from Leave punters, they didn’t move the price commensurately, as a neutral market operator would, but kept Leave longer and more attractive than it should have been.  When some lumpy bets came in for Remain on the day – they collapsed the price to unrealistic levels, really bad jobbing probably.  

Assuming the role of bad political scientists rather than proper bookmakers, they didn’t want to lay this money for Remain.  Graham Sharpe, Head spokesman at Hills, confirmed to me during the campaign that his company was taking a position with "Leave".  It wasn’t the “City Traders” who were taking the position – it was the bookmakers – just has they had wrongly against a Tory Overall Majority in 2015, Corbyn and Trump.  If there was “City money” entering the market on the day, the bookmakers accentuated the effect of it on the market price.  This may have had an effect on the Betfair market, because of course they are linked.  Any arbitrage between the two will be filled in. 

Only complete openness about their trading positions and P & L from Ladbrokes, Hills and others and other trading information from Betfair will resolve why the EU Referendum betting market on Thursday 23^rd June behaved so oddly.  It is extraordinary that in such as tight two-horse race, at one stage, the betting markets suggested Remain had over a 90% chance of winning - as I pointed out on my Twitter account at the time. 

Of course this would have been an important indicator to global financial markets.  The question remains:  was the price of 1/10 a reflection of the weight of genuine money from the public for Remain, money from the City for Remain - either legitimate hedging or attempting to rig the market, or the traditional bookmakers taking fright and protecting their position - which favoured a Remain outcome?  

And was there any secret exit polling conducted,as alleged by city institutions?   The allegations circulating are completely unproved, as our my suggestions that the bookmakers were the real reason why the odds were all wrong, not the massive crowd of punters.  There may have been some big bets for Remain, from whom we do not know.  At the moment, there are more questions than answers, and openness from bookmakers is needed.

Bet "Leave" at 11/4

In recent days, several people have asked for my view on the EU Referendum result.  I have spent three years on a PhD looking at EU support in Britain and a lifetime in political betting, on and off as a political odds-maker.  I've also taken some serious hits.  The worst was a £16k loss on the 1997 General Election opposing the Lib Dems.  As a spread bet, for every seat they won over 30, I lost £1,000.  They got 46, including winning Winchester by one vote.  The first one vote winning margin since 1867.  Even today I wince as I drive past that place on the M3.  So here we go again.

This blog believes in 'crowd wisdom' as expressed by betting market indicators.  On a liquid market like the EU Referendum, we should expect the market to be a reliable guide as to what may happen.  Taking it on with a bet, together with the bookmaker margin, is unadvisable.  Unless you have a strong view that is.  And I think there are two things badly wrong with the market at the moment.

The first is bookmakers are taking positions with 'Remain'.  It is likely that Ladbrokes and William Hill have £1m+ liabilities on a 'Leave' win.  There are plenty of Leave backers out there, but generally the layers are not prepared to make commensurate changes to their odds when accommodating them.  This means their prices don't actually represent market sentiment, but their own trading floor views.  We've seen this before with the General Election, Corbyn and Trump, and each time they have come unstuck.  They think they know better. 

From talking to the current odds-makers, and seeing them operate, it's clear they are being 'advised' by political experts - academics and the like.  Matthew Shaddick, the political man from Ladbrokes, likes pitching up at academic conferences.  In an unholy alliance, he denounces the value of his customers opinions in front of an appreciative audience of expert political scientists, who also loathe betting as a predictive indicator.  At one conference before the 2015 GE, he stated that "political markets are not a good guide to what may happen".  So better take the expert view instead, the complicated academic models and frigged polling methods, with thousands of highly breakable whirring parts.  Received academic wisdom tends to think that the 'Government cue' is still strong, like it was in 1975, and that voters are uninformed and uninterested on Europe and will fall into line.  My own research of British Election Study data from 2008-2012 shows the opposite.  Voters are increasingly independent minded (volatile) and have responded to the crisis in the Eurozone (something Leave should be focusing more on).  Since 2000, EU referendums in Holland, France and Ireland have all gone against the pro-EU Establishment, much to their surprise.  Who is to say it won't happen likewise in more Eurosceptic Britain?

The academic experts and even pundits like politicalbetting.com are also poor oddsmen.  Firstly they are inherent favourite backers.  That is what betting unsophisticates do.  They think 'what is most likely to happen?' and go for it.  A bird in the hand is worth two in the bush.  But betting is about assessing chances against odds of reward, in short, value.   

Secondly, all the noise you hear on Twitter is either from fruit-bat 'Leave' evangelicals or reasoned liberals.  That is the image anyway.  Academia - showered in EU money - is pro-remain, and hangers on to their output, particularly those noisy on Twitter, like politicalbetting, are well known liberals too.  In short, 'informed social media' doesn't want Leave to win, and doesn't want to back something it despises.  The same happened with the Tory victory, Corbyn and Trump, whilst the betting public felt differently - and won.  Ladbrokes lost £2m on GE 2015.  An extraordinary failure when they had so much two way business to eek out a profit.  This was a golden opportunity for bookmaking to prove its worth against polling, instead it just followed it, breaking the golden rule that the market knows better than a handful of traders.     

Lastly, I think a bubble for Remain is developing.  This is most evident in the Times Red Box Survey, where the great and the good of British Politics (and the public) are asked to submit their predicted vote percentage for Remain (average is 54%).  It's revealing.  For example, Matthew Parris thinks 62.5% which proves he is unhinged.  But surely, people are going to be influenced by what others have entered, even Parris and a phalanx of other liberal insiders, kindly marked up with an asterisk on the site so we can pay them special attention.  The exercise lacks one key criteria for a 'Wise Crowd' - independence of judgement.

Then there is Betfair.  People will say that this is a perfect betting market and it doesn't involve bookmakers taking positions, advised by their academic advisors and powered by their own egos.  This is just wrong.  The exchange market reflects the strong traditional fixed odds market.  Arbitrages between the two are soon filled in.  And the old world bookies are far less likely to move their prices.  This is why they build up their liability positions.

Set against the noise of this betting event, and structural reasons why bookmakers may be underestimating Leave chances in their prices, there is some solid evidence to examine.  Pollsters are all over the place and have lost credibility.  But one thing about their averaged results can be relied upon - they will be consistently wrong over time.  We may not know just how well Leave and Remain are doing at any one moment including now, but over the days and weeks we can see how things are moving for the two camps (assuming no methodological changes by the pollsters).  Here there is a clear trend.  Leave are winning this campaign.  Look at the remorseless rise of the red Leave line to now challenge the blue Remain line.  This trend matters hugely, because unless there is some fundamental change to how the campaign is being conducted, it is likely to continue.  And that means - from the graph below (courtesy of Prof. Harold Clarke) - Leave could soon start to overtake Remain.  (Update 13/06/2016 - And they now have).

(Updated 13/06/2016)

So what is the advice for someone who wants a bet?   Look at Oddschecker.com.  11/4 is generally available about a Leave vote.  Have a go and pick up a free bet at the same time.  These odds suggest there is only a 26.7% chance of Leave winning, and 73.3% chance of Remain winning.  It is worth checking these percentages against the full gamut of prediction percentages on the excellent https://electionsetc.com/. 

Academic unanimity in favour of EU membership is to be expected, but derogatory views about public attitudes are less acceptable

Nigel Farage has called a campaign for the impartiality of British academia in the forthcoming EU Membership referendum.  His argument is that academics are so widely and heavily funded by the EU they cannot be properly free-thinking.  On this he is surely right.  In July, Universities for Europe and Universities UK (which represent 133 higher education institutions) came out with unequivocal backing for the campaign for Britain to remain in the EU.  How can this be reconciled with the idea that universities should be bastions of open-mindedness and free speech?  How can any academic speak out without fear when the 'company policy' is against him? 

I've spent four years studying public support for the EU (or lack of it) at a British University.  I know about the pressures to conform to the orthodox wisdoms of academics. (My two excellent PhD supervisors are most certainly blameless here, as is Matt Goodwin who I worked for on his latest Ukip book - published next month).  The tacit rule is don't question the EU funding arrangements for research.  And never criticise the holy grail: the €14.7 billion Erasmus+ programme.  You will have an irate Erasmus exchange student sitting right next to you.  Sometimes I have felt like a dangerous heretic.  Not once have I heard a qualified academic speak out in favour of leaving the European Union at a public event, such as an academic conference.  

A survey before the last election of 1,019 academics for the Times Higher Education Supplement found only a lonely 4 who supported the Eurosceptic Ukip. Reason enough you might say, to return to the days post 1603 of the 'fancy franchises' when Oxford and Cambridge Univerisites had two MPs each of their own. My arriviste institution, the Univeristy of London, was granted one through the Second Reform Act of 1867.   Nowadays however, it's generally accepted that the more cerebral do not necessarily make better voters.  They like the rest of us have their own interests.  And these, as much as their unique insights explain how academia has become so embarrassingly detached from the collective wisdom of the average voter.

Armed only with one vote like the rest of the public, academics can still exert considerable influence on our politics through their research.  As I show below, it's one central idea from academia that frames how the media report the public debate on the EU, and unsurprisingly, it's a favourable idea for the 'remain in the EU' lobby.

The idea actually stems less from decent research than a prejudice academics hold about the wisdom of public opinion on EU matters.  My research work has challenged the central 'EU friendly' rule that dominates European Studies departments and colours voter behaviour research on the EU issue.  It's this: the public cannot be trusted to make sensible decisions on EU integration.  They don't understand its complexities.  They are uninterested in its remoteness. And, most seriously, they are led by their emotions (when not by the political class).  The intellectual justification is in part poor empirical research and part suspect psychological theory.  More specifically in Britain, the public, so the argument goes, see immigration and economics as separate concerns within the overall EU debate, but too often their rational economic judgements get infected with more visceral (non-rational) feelings of national identity that drive immigration attitudes. To use the academic vernacular, EU support is structured two-dimensionally, and these dimensions are 'cross-cutting' or 'orthogonal'.  Simply put: the two types of support which make up the overall 'remain or leave' judgement in the public mind are separate but conflicting.  Rational thoughts can't be squared with visceral feelings.  How could voters make head or tail of it? 

The derogatory stance of academia towards the voting public on the EU is an all important subtext to the forthcoming referendum.  Whilst the 'remain' lobby would not be so unwise as to say explicitly the public are confused and can't be trusted, the Europhile camp believe it.  They will use the 'two dimensions' idea in a more subtle way to frame the debate.  They want arguments about immigration and economics detached so they can play on economic fears, whilst branding the immigration side of the debate an emotional side-show.  So far they are succeeding.  In the public debate, the idea of independent economic and immigration aspects of the question are independent is all pervasive.  When you hear the ubiquitous words 'soft' (economic driven) and 'hard' (identity driven) Euroscepticism you are hearing a narrative of divided, incompatible attitudes.  Even opinion pollsters measure interest in immigration as separate from EU membership (and then conclude there's not much interest in the EU question).  But the public don't see it this way.  They have no difficulty reconciling their economic and immigration views.  For my PhD, I have researched different types of attitude towards EU membership, particularly on immigration and economic questions.  Having analysed a range of survey data, I have found no empirical evidence suggesting economic and identity attitudes on the EU question are anything but one-dimensional.  Economic and identity concerns are tightly correlated - i.e. they face the same way, either against or for membership.  They only form observable separate dimensions when survey indicators are 'cherry-picked' to conform to the beloved two-dimensional theory (E.g. Boomgaarden et al., 2011, Mapping EU attitudes: conceptual and empirical dimensions of EU support.  European Union Politics, 12(2) 241-266).

Some of my research is displayed below.  The table shows a range of 'items' or survey question responses from British voters (vertically) to the 'Intune' survey of 2009 (random-probability sampling).  Some of these questions ('Diffuse Identity items') tap voter feelings of attachment and belonging to Europe and represent deep-seated emotional ties - 'hard support' if you like.  The second set of questions are more specific and are meant to tap rational (economic) evaluations of membership and EU performance, such as whether it benefits me personally or the country. This group of items represents so called 'soft support'.  According to the received wisdom of academia, these two sets of attitudes should not load onto one dimension, but load onto two dimensions.  In other words, the set of hard identity items should correlate with each other, and so should the set of soft performance evaluation items.  But the two groups should not correlate together as one because they are different dimensions.  This is not the case.  The one dimensional model is a better fit to the data than the multidimensional model (denoted by a higher Bentler-Bonnet test score and a lower Standardised RMR test score).  This means it more accurately reflects public attitudes to Europe.  The public see the questions of identity and economics as one.  The idea of two dimensions is a fiction, according to my research. 

Imaging British public attitudes as one-dimensional helps explain why support changes in response to new information - and it does. If British EU opinion is a 'great concrete block' of deep-seated unconscious prejudice - as Tony Blair's pollster Philip Gould* said in 2002 - why should it move in a meaningful and measurable way?  But it does.  You will have to wait for my PhD to be published for more on the structure of EU support, and why levels of British public support go up and down.  Suffice to say now, the main implication of my work is that we need to take the wisdom of the voting crowd far more seriously than the existing academic literature would suggest. 

For campaigners, this is yet more reason to adopt a bottom-up approach where the public are engaged by the respective sides and take part in the debate.  Don't think they will be influenced by elite arguments like they were in 1975.  The public know a lot more about the EU now, and elites are less trusted.  In my PhD I also provide multivariate modelling of time-series data, showing that the power of politicians to influence public attitudes has reduced sharply, particularly during the Eurozone crisis between 2008-2012.  The public are increasingly drawing their own conclusions, armed with new information on EU performance.  They now have sufficiently well structured views to be repelled by patronising politicians who they don't trust much.  They are far more likely to listen to the rest of the public, which could make crowd sourced 'bottom-up' campaigning a decisive tactic, if anyone knows how to do it.  And don't imagine that their judgements won't change sharply in response to new information on the immigration issue, on EU or domestic economic performance, or moving evaluations of the party leaders.

I don't believe that either side of the debate (particularly remain) are ready to really trust the public with a crowd-sourced campaign.  The expensive PR agencies of central London will be in heavy demand once again, producing winning arguments and clever slogans led by often suspect opinion polling.  Inside their lavish offices the detached members of the political elite will devise plans to manipulate the conscious thoughts and unconscious desires of the public.  The public will never be granted ownership of the campaign with a buzzing and gamified online strategy.  By this I mean voter / activist / spokesmen profiles and fund-raising; grass-root campaigning leaderboards and prizes; cartoon, blog, and vlog competitions; online ratings of politicians or arguments and prediction tournaments on what might happen.  If you haven't heard of this stuff, its because it has never happened.  Instead the public will be cajoled as a plaything, not treated as players - never granted ownership of the campaign, for example on which spokesman should get to appear in the media. Excellent spokesman could emerge from the general public providing a greater voltage to the debate, perhaps having been voted as a winner within an online competition.  Who is to say there is not a brilliant communicator out there amongst the general public who has yet to be unearthed?  Such a campaign would not be devoid of PR spin. Any campaign that could show authenticity in listening to the public would have the right image.  The voters would feel part of their own movement.

The blog below on the pernicious impact of Sigmund Freud and family discusses why our intellectual culture has been anti the 'wisdom of crowds' idea - that the many are smarter than the few.  It's the underlying reason why academics justify crowd stupidity and is most prevalent on the more established 'remain' side of the EU debate.  The remain lobby has never believed in the wisdom of crowds since the days of Monnet.  But their elite-centric view of the world of grand top-down plans and big centrally coordinated groups, is fast being pummelled into a flat expanse of rubble by the Internet.  If there is a long term underlying cultural change that will see 'remain' lose, this is it.  The crowd will understand it faster than the planners and Europhiles. The future is theirs.

     

Sigmund Freud and family have a lot to answer for - a rant against Freudianism in theory and practice

Apart from advocating cocaine as an anti-depressant, Sigmund Freud first suggested that our behaviour owes more to emotions and desires lurking in the unconscious than the rational assessments we make consciously.  Western civilisation represses these base instincts Freud maintained. Subsequently, such repression would either be considered bad (as the Nazis thought, along with the lunatic Wihelm Reich, Herbert Marcuse and a school of modern day Marxists); or for good (as the generation of post-war paternalists believed).  What united both the fascists, Marxists, lunatics and the post-war interventionists, was a believe that the unconscious desires of the masses were there to be either let loose or managed by elites who always knew more.  Effectively, by giving a pseudo-scientific account of the dangerous capacity of the masses, Freud had legitimised any amount of subtle controlling by our leaders, benign or not.  ‘Nudge theory’ is merely the latest incarnation of the supposedly benign variety.  Blair was perhaps the most pernicious recent exponent of mass manipulation of the public into support for his just causes, such as his liberal interventionist foreign policy and support for EU membership. Without compunction, he and Alistair Campbell happily 'sexed up' the dossier justifying war in Iraq in 2003.

Before the Internet, and the emergence of a new faith in ‘crowd wisdom’, twentieth century business followed the political manual of Freud, thanks in part to Freud's nephew, Edward Bernays. He pioneered techniques to deceive the public into rampant consumerism (helping to contribute to the Wall Street Crash of 1929).  He was the first to talk of 'public relations', a disingenuous term for as practice in 1920s America of tapping public emotions over reason. One of his first jobs was to break the taboo on women smoking in America by shoving cigarettes in the mouths of suffragettes (who were then photographed) to connect the activity with female emancipation.  The Freudians understood cigarettes at the time to be represented in the subconscious mind as penises, so suddenly women had willies.  After WW2 he worked for the American government stoking up the emotions of Americans against a perfectly legitimate new government in Guatemala, the original 'Banana Republic', that threatened US banana interests and was made to pose a communist threat to US security through his falsehoods.  If you haven't watched Adam Curtis' documentary, 'The Century of the Self' (BBC, 2002), please do. ('TheMayfair Set' by him from 1999 is brilliant too – spot a similar theme).  Bernays’ favourite word was stupidity.  In fact he called most people around him stupid.

Naturally enough, the alternative position to believing in 'crowd wisdom' is considering the public stupid and advocating paternalism.  They are really two ends of the scale.  Perhaps the greatest exercise in paternalism since 1945 has been European integration. Think of the dominant narrative of European Studies departments about British voters and the EU: that European integration is objectively beneficial but remote and complicated, beyond the cognition of generally uninterested voters.  Instead those voters are falsely conscious of their interests.  They are obsessed with the threat posed to British identity by EU governance and EU immigration.  (See my critique of Hugo Young's 'This Blessed Plot' on this website - May 2013).  In his final interview as Prime Minister, Blair offered a frank summary of the false  consciousness theory in respect to EU integration: 'The British people are sensible enough to know that, even if they have a certain prejudice about Europe, they don't expect their government necessarily to share it or act upon it' he said. 

More broadly, this narrative on public support for the EU is incompatible with a whole branch of classical political science rooted in the assumption that man is capable of pursuing his own interests, or as Anthony Downs puts it (1957:2) “that there is a pattern – or ordering of political behaviour that is rational, i.e. ‘reasonably directed towards the achievement of conscious goals’. This is fundamental to the study of voter attitudes, because ‘only if human actions form some pattern can they ever be forecast or the relations between them be subject to some analysis’. The alternative approach, that has the currency in European Studies departments, elevates psychological, social and emotional factors over economic ones in voter decision-making and is sometimes called 'behavioural economics'.

My PhD thesis (summarised in my blog from April 2015) argues that the whole study of public support for EU membership has become deeply confused and unhelpful because it is now dominated by this pessimistic approach to voter motivations, incorrectly specifying two distinct and conflicting dimensions to public EU support: soft (cognitive / economic / rational) and hard (emotional / identity - from the unconscious). No consistent theory is supplied about how these dimensions might be related and when each component becomes more important, and hence no rational theory is possible for what causes over time change in attitudes. It is really a failure to find a third way approach between believing in the wisdom of crowds and believing in their stupidity. 

This link is an easy to understand exposition of the behavioural theory from Prof. Richard Thaler, given in a lecture a few months ago at the LSE. (University of Chicago, co-author of ‘Nudge’ - that attempt to intellectualise paternalist government).  I dislike the sneering tone of Thaler, seemingly a common trait with most behavioural economists. The lecture is full of tiresome jokes about how daft (other) people are. The most ghastly behavioural economists (like Thaler) love anecdotes deriding their opponents and are deeply patronising of the public.  Paul Dolan, a professor at the LSE also talks briefly after Thaler (at 39:40). He says one preposterous thing. Before GE2015, he was a huge fan of Betfair as 'a really brilliant prediction market' in politics - suggesting, somewhat inconsistently, belief in the predictive power of 'crowd wisdom'. However since the election in May his view has undergone a 180 degree volte face, because the market got it wrong.  This is just one example of how people who try and occupy the middle ground between believing in crowd wisdom and psychological theory are just confused.  Of course, the informative value of a betting market on a future outcome cannot be judged on one case, however important that case may be.  Favourites do not always win, they only win a percentage of the time. Therefore the predictive quality of markets must be judged probabilistically, on multiple cases over time.

One of Thaler's buddies is Cass Sunstein, who wrote a largely admirable book on crowd wisdom called 'Infotopia' (2007).  Unfortunately he is also a 'nudger' like Thaler, believing that failures in crowd wisdom are caused by groups becoming 'cocooned' by 'information cascades' (essentially herding), and need to be corrected by the great and the good - the nudgers like him.  But when do we know that such action is required?  And who can we ever trust to nudge?   What do you think?  Listen to the podcast and don't waste £11.99 on Sunstein and Thaler's book.


* The late Philip Gould was well meaning.  He thought through his focus groups New Labour could actually listen and measure what the public wanted.  This was an improvement on Labour in the 1980s, but the 1997 Labour election victory was not 'an end to the elitist politics that has governed Britain for the last 100 years' - see here at 3.46:00.  In government Blair maintained the old paternalism.

One pager from Dominic Cummings on "Why do I want Britain to leave the EU?"

I've added this short polemic to my blog because it represents modern British Euroscepticism at its most cogent (and concise).  It's a positive economic case that doesn't get heard enough.

The EU suffers a combination of huge debts, mass unemployment, a rapidly ageing population combined with unsustainable pension obligations, and an anti-entrepreneurial and anti-technology culture. It has created a euro that damages prosperity, undermines democracy, and encourages extremism. Its dysfunctional bureaucracy is manipulated by corporate interests who like to use the EU machinery to crush competition (just as people from Adam Smith to the democratic left have warned about big business). From public procurement to international trade, our membership undermines good government and sensible policy and wastes billions annually. It is so bureaucratic and slow-moving that it cannot adapt quickly to challenges and is the opposite of the sort of agile institution necessary to cope with contemporary and imminent global challenges – for example, it is so slow moving that it remains stuck with agricultural subsidies dreamed up in the 1950s and 1960s that raise prices for the poor to subsidise rich farmers while damaging agriculture in Africa.

Meanwhile, we need new forms of governance to cope with the spread of markets and technology. We need global cooperation on many issues including profound technological changes such as genetic engineering and robotics. Such cooperation is undermined by the dysfunctional and parochial EU.

Whether they abandon the euro, muddle on, or make a great leap forward to the long planned political union, Britain will do best for herself and Europe and by removing itself from this experiment and showing an alternative path. We could help strengthen international cooperation on the biggest issues facing humanity. By demonstrating the success of a different approach, we could have real influence, rather than the chimera of influence vainly chased by the Foreign Office from meeting room to meeting room since the 1970s – a chase that simply led to concession after concession rather than influencing Brussels to change path.

A NO vote will force a profound rethink of how we organise politically and enable us to develop new systems based on decentralised cooperation and distributed decision-making. This is vital given all the problems the world faces. My 2013 essay and this blog (here) described many of these problems and suggested some ideas about how Britain could place education and science at the heart of its national policy instead of EU membership. This would be far preferable to our current behaviour –  petulant and embarrassing whining and obstructionism on the euro-federalist project, combined with a complete lack of useful alternative ideas born of post-Suez Whitehall defeatism.

We can do much better…


Originally published on Dominic Cummings' blog - June 19th 2015.  See (bottom of)  https://dominiccummings.wordpress.com/2015/06/19/on-the-referendum-4-an-exploratory-committee-and-the-beginning-of-a-no-campaign/

Opinion polling on Scottish independence and EU membership referendums

Pollsters have had a good independence referendum.  However, improved explanations about why voters have swung to ‘Yes’ would be helpful.  This blog discusses public attitude change and how it might be better explained during a future EU referendum campaign.

Opinion pollsters have added insight and direction to an engaging campaign.  Without the polls we would be unaware just how close the contest had become, lost in a fog on punditry.  We would be subject to endless claims and counter-claims from each lobby about who was really speaking for the people.  Polling keeps politicians honest, telling us about the evolving credibility of each side’s message.  Interviews and focus groups with voters may have their own value, but they leave us little wiser about the overall outcome of the event.  By contrast, polling forecasts have moved financial markets, changed campaign strategies, perhaps even caused the No lobby to concede ‘Devo Max’ to the Scots.  Given the increasing demand for polling predictions, and the internet technologies that make the supply of polls cheaper, we should expect increased polling activity if and when an EU Referendum is called.  But the product itself can be improved.      

Whilst the Scottish experience has been great news for the pollsters, it has also given them a couple of headaches.  Firstly, it may go horribly wrong by Friday morning.  Any celebrations of their impact could prove premature if the actual result reveals an industry-wide miscalculation.  Given that the independence ballot is a one-off, methods cannot be tweaked from previous experiences to correct for non-random (systemic) error.  Polling companies are therefore less sure than usual about the accuracy of their surveys, particularly those using online panels that rely upon heavy weighting to the initial samples.  The +/- 3 point ‘margin of random sampling error’ may not tell the full story of polling reliability.  To the pollsters’ credit, they have been both honest and humble about their methods, taking every opportunity to raise their own self-doubts about their figures.  An excellent discussion of referendum polling accuracy can be heard on the BBC Radio 4 More or Less programme, here, and the matter is well reviewed in a blog by Stephen Fisher here and Anthony Wells here.  The general feeling seems to be that, if anything, pollsters are more likely to be overestimating rather than underestimating actual support for Yes.  Even if they get it wrong this time, they deserve credit for sticking their stake in the ground, being open about their methodology so it can be improved upon in future.

Assuming the pollsters have told us correctly what will happen, to be confirmed tomorrow, they are less impressive at telling us why it is happening.  Ultimately, what really matters is change in public attitudes because it’s this change which could split the United Kingdom apart.  Understanding why support has changed requires a fuller range of survey indicators than the simple and less reliable self-reports by voters of their own motivations – the questions which are asked currently.  The public need to be probed in more detail about their national identities, partisanship, economic evaluations and levels of political trust that are all key causal variables.  Also such indicators need to be analysed over time rather than as one off measurements, to pick up on trends.  Very little of this over time analysis has been conducted on attitudes towards independence, in part through lack of data.  This may explain why it took a shock poll from YouGov, showing Yes in front for the first time, to jolt public interest and the No side into serious action, less than two weeks prior to the ballot.  Whilst traditional polling based on individual snapshots of opinion did not see a close race coming, it was in fact predicted by researchers at Southampton University, using a rare (Bayesian) model of vote intention based on trends and probabilities.  Better partnerships between public facing polling companies and academics versed in time-series methodologies could improve the offering and its overall impact.

Understanding underlying trends in support is crucial to framing the right campaigning strategies on both sides, shedding light on what is working and what is not.  Misunderstanding of why support has moved, could explain the poor performance of the No campaign.  In a matter of weeks, opinion is now split close to 50/50 between Yes and No, having been 60/40 against, a shift described by Anthony Wells of YouGov as a ‘real and sustained large change’.  Polls may be overestimating or underestimating support at any one time, but on the question of over time trends they are unequivocal.  As the ‘poll of polls’ below shows, this growth in support for independence has actually been going on for months rather than weeks:

Poll of Polls – 12/09/14  (ScotCen SocialResearch)

Pollsters and political scientists who specialise in public opinion tend to be conservative about shifts in attitudes, particularly short-term ones.  As the Polling Observatory make clear, these can often be nothing more than random noise, artefacts of measurement instruments rather than a significant signal of attitude change.  However, long-term change as evident in Scotland cannot be so easily dismissed. 

The question of explaining attitude change is also a theoretical one about how voters form their opinions, extending beyond practical questions of data and polling methods.  The problem here is a lack of clear underlying theory why individual voters have actually shifted their attitudes.  Within some quarters of academia, there is outright denial that voters possess the capacity to change their views in response to campaign information, at least in a meaningful way.  Dr Rob Johns, writing on the British Election Study website, warned in July – just before the No campaign started to implode: ‘do not expect major shifts in opinion in the run-up to 18 September and that ‘the millions to be spent on persuading voters between now and polling day will be largely wasted’.

What appears in hindsight as irresponsible advice, reflects a gloomy appraisal of the voter.  Rather than behaving in a largely economic (or ‘rational’) fashion by consciously weighing up the costs and benefits of Yes and No, Johns argues voters develop their preferences in an irrational or psychological manner, working against change.  Preferences are deeply rooted in ‘early upbringing and personality’.  This idea dates back to the seminal ‘Michigan School’ studies of the American Voter in the 1950s, an account of electoral behaviour that stressed the role of ‘enduring partisan commitments in shaping attitudes’, formed early in life and remaining stable through adulthood (Campbell et al. 1960).   Despite concerted attempts in voter studies to revise the Michigan School approach with more dynamic ‘rational choice’ type models, the older approach has never really gone away.  Indeed national identity attachment is ingrained in academic literature as the dominant variable for explaining support for the EU (Hooghe and Marks, 2009).  The question is, when significant opinion change does happen, as it did during the Eurozone crisis in respect of attitudes towards EU membership, and now again during the Scottish Referendum campaign, is the Michigan School theory up to the task of explaining it?

Under the Michigan model, when the voter is confronted by new information, he is likely to become polarised in his existing opinions because he assimilates news through the prism of his long-standing biases: ‘the individual is more a rationaliser (of his existing prejudices) than a rational decision maker’ (Lodge and Taber, 2013:26).  For example, in the Scottish case, the voter who watched the televised referendum debates, if a nationalist, would tend to dismiss Alistair Darling’s arguments whilst agreeing with Alex Salmond, without judging the competing arguments on their merits.  The unionist would behave similarly, blindly agreeing with Darling and rejecting the words of Salmond.  Hence, not much change in the polls would result.  If one side enjoys a short-term bounce, it is likely to be superficial, dissipating as voters powerful long-term attachments kick back in. 

There is another specific reason why this psychological, non-rational model theory of the voter is popular in explaining opinion on the Scottish referendum.  Independence is an issue akin to the question of EU membership, long considered difficult for the public to appreciate in a thoughtful way.  Academics like to argue that because the economic and constitutional questions of EU membership (or Scottish independence) are so complex, an essentially lazy and ill-informed public cannot engage with them.  Instead, EU membership or Independence stirs passions that are not reconcilable with rational discourse – so the argument goes.  These passions are based on national identities and populist distrust of the governing elite, which combine in a heady cocktail of emotion.  Whenever we hear the media framing the debate as ‘hearts versus heads’ this line of thinking is in play.  The assumption is the public cannot reconcile competing (affective-cognitive) dimensions in their support, meaning their evolving attitudes are unstable, and usually constructed externally of them.  In the simplest terms, rather than the voter consciously and reasonably weighing up different aspects of the debate and making informed choices himself, the different dimensions in his attitudes become mobilised by the media and politicians instead.  And this becomes the explanation for changing preferences to Yes – the Yes campaign could effectively mobilise voters key underlying attachments against Westminster, together with Scottish nationalism.  The No campaign was always likely to struggle to mobilise Scottish voter’s economic calculus against separation by highlighting its risks.  If the Yes campaign wins, it will be said that hearts have triumphed over minds.  The result was somewhat inevitable and No’s big mistake was allowing the referendum to happen in the first place.

There is an alternative explanation of why attitudes have changed in Scotland which gives more credit to the voter.  It does not deny that Scottish voters’ beliefs are strongly influenced by their national identity and that many have a passionate distrust of the British political establishment, nor does it take a view on the worthiness of these attitudes.  It simply maintains that these emotional aspects of their support are more closely related to their economic evaluations than the psychological theory of voting insists.  Rather than voters’ support for independence being structured along two unrelated dimensions, attitudes are simply a mix of the voters own prior beliefs and new evaluations.  Voters update their prior beliefs as a ‘running-tally’ of their new evaluations of information, changing these prior beliefs when they consider the new information to be sufficiently valuable to affect their interests – and ignoring it if they consider it invalid (Fiorina, 1981).  The voter isn’t so conditioned by his background and psychological processes, rather he has subjective views of his own and is capable of making choices.  The problem with the No campaign wasn’t necessarily the dry emphasis on economic and political costs and benefits, but that the public either didn’t receive sufficient information early enough; they judged the message too disorganised and confused; or discounted the information because they distrusted the source.  If No loses, it was not inevitable, their arguments weren’t necessarily wrong but their presentation was.  In particular it needed to come from more trusted sources than the declining Westminster mainstream elite – perhaps the Scottish voters themselves via a more grass-roots orientated campaign.  (The importance of grass-roots campaigning and the weakness of mainstream political parties in influencing their voters is a major lesson for EU referendum campaigners.)

The running-tally explanation of changing attitudes demands that voter attitudes are modelled over time, to reflect the probability calculation that voters make between the value of their previous beliefs and the worth of new information.  It requires long-running time-series data with the same questions asked in each poll, such as by the monthly British Election Study Continuous Monitoring Survey.  New data, new methods and revisionist theories about the capabilities of voters to make informed choices could all contribute to make polling on a future EU referendum even more impactful than the polling on the Scottish independence referendum.

References:

Campbell, A, Converse, P, Miller, W and Stokes, D.  (1960).  The American Voter.  University of Chicago Press.

Fiorina, M. (1981).  Retrospective Voting in American National Elections. New Haven: Yale University Press.

Hooghe, L and Marks, G. (2009).  A Postfunctionalist Theory of European Integration: From Permissive Consensus to Constraining Dissensus.  British Journal of Political Science, Volume 39, Issue 1, pp 1-23.

Lodge, M and Taber, C. (2013). The Rationalizing Voter.  New York: Cambridge University Press.
My planned PhD thesis is summarised here.  It uses the ‘running-tally’ theory and time-series methods to explain why public attitudes have moved so sharply both for and against EU membership in the last ten years, in response to record EU immigration and financial crisis.