The tyranny of statistical determinism

Among the myths circulating amidst the ether of modern American discourse is the stubborn fallacy that political forecasts and models are deterministic. Regardless of whether a poll measures support for presidential candidates in the hopes of predicting the outcome of an election or gauges the public’s attitude towards hot-button issues before Congress, the media and political scientists like to cite these numbers as Absolute Facts.

Viewed this way, they influence everything from how politicians lobby for and defend policy to how media personalities cover stories and question candidates.

Perhaps the most prominent example of the entrenchment of statistical determinism exists in the way a 2013 Washington Post-ABC poll conducted in the wake of the Sandy Hook shooting was touted by the president and his sycophantic followers in the media as being concretely supportive of his proposed gun control measures.

One question asked whether respondents supported a law requiring purchases at gun shows to be subject to background checks. 91% of respondents expressed support, 82% strongly so. The poll has a margin of error of 3.5%, so what the polling data actually says is that a range of 87.5 to 94.5% of respondents agreed supported a law mandating background checks for purchases made at gun shows and that this result will be true 95% of the time. (The polling methodology does not include a confidence level, so this is assumed as it is the most commonly used.)

This, however, did not interpret the poll within the confines of the data but instead extrapolated the results, claiming that 90% of all Americans supported the president’s entire slate of “common sense” gun control measures, which meant only stubborn, self-interested politicians who didn’t care about the will of the people opposed them.

Leave aside the dubious ethics of the Post relying on numbers compiled by their own polling company as the major piece of evidence in their argument, and leave aside the broader context of laws which already exist to regulate gun shows which the president and the media ignored: the fallacy of statistical determinism, and the ability it has to warp national issues becomes clear.

Polling companies do attempt to tailor their results, using Census data, to achieve a result that roughly mirrors the national polity in terms of age, race, party identity and other social markers.

However, this is still just a model and one constrained by past data and behaviors. Algorithms must assume that the behavior patterns of the past will remain constant in the future, an assumption which is not only mathematically irresponsible but contrary to the fundamentals of human nature.

The tyranny of statistical certainty which such absolutist interpretations of polling data by politicians and the media is problematic not just because it lends a false sense of inarguable factuality to their claims, but because of the potential such posturing has at shaping the national dialogue.

There is nothing wrong with utilizing polling data, so long as there is an understanding that there are constraints built into statistical models which limit how results can be interpreted.

By the time polling is conducted, finalized and reported, layers upon layers of interpretation and speculation have been built into the hard numbers.

The media’s lip service to the margin of error does not even gloss the surface of how the ever-present issue of bias affects polling results.  Bias exists not only in the way the poll is structured- in phrasing, in question order, in the way the conductor interprets responses- but also in how respondents react. People feel pressure to respond within the parameters of what is considered popular consensus. Then, there is the bias of non-attitudes. People do not want to appear uninformed or uncaring, so if they do not know the particulars of an issue or do not have strong feeling, they simply make up an answer.

The length of time over which a poll is conducted and oversampling from one group in order to make the sample fit the broader demographic can also skew polls.

In short, all sorts of uncertainty enters the picture. And it is magnified the more the data is extrapolated beyond the parameters of the original results.

This is why statistical determinism in the media and politics is troubling. Polling, forecasting and modelling are not concrete. They are a useful frame of reference, but they are not established certainties. To pretend that they are, and to use them as a primary argument, is to effectively ground national dialogue in quicksand. It is not so solid as it appears. And it is impossible to avoid the disaster of venturing onto such unstable ground.

Also published on Medium.

All content protected by copyright. The Politics of Discretion, 2016.
%d bloggers like this: