George Walford: Surveying the Surveyors
When discussing the relative sizes of the major ideological groups, one commonly encounters the suggestion that an opinion survey would settle the question once for all. Why undertake all this argument? Why not simply count them? The Chair of a MENSA meeting addressed by an s.i. speaker once put this forward.
The people who have studied this method of investigation, and the reliability of the results it produces, offer little support for such confidence. Commenting on a recent Mori survey, Robert Harris notes the improbability of some of the results. 88% claimed not merely to favour but to belong to a denomination of the Christian church. Only 6% believed in shirking and only 1 in 10 were unhappy in their work. Over 50% claimed to read 10 or more books a year, and 64% to have read one during the past month. 27% claimed to have visited a library or museum during 1988, and 26% of Tory supporters said their ideal society would be mainly socialist. He suggests that people answering surveys tend to give what they believe to be the socially acceptable answer, telling the statisticians “a lot of what we non-scientists call lies.” [1]
Daily contacts also suggest that some of these figures need to be treated with reserve, and these impressions can claim academic support. Michael Billig makes the general statement: “There is considerable evidence of frequent discrepancies between people’s attitudes, as revealed by standard questionnaires, and their actual behaviour,” and supports it with six references. [2] William L. Miller provides an instance, noting that in the British General Election of 1987 no less than 20% of the voters “changed their intentions during the campaign and did not vote in June 1987 for the party they had indicated a preference for in the previous March.” [3] Studies giving results which change by up to 20% in four months have little value either for our understanding of society or for long-term prediction.
Other complications come from relationship between interviewer and interviewee. Polls do not originate in some abstract realm of pure objectivity; they display motivation and they function in a social context. Reviewing a clutch of surveys of Britain John Vincent comments: “Polls measure the interests of those who design the questions. They also measure huge but momentary fluctuations… The environment was a major issue for 5% in December 1988, for 9% in January 1989, 14% in February 1989, 35% in July and down to 18% in December 1989… Polling is the beginning, not the end, of research into meaning and values.” [4]
In the past IC has perhaps been too ready to quote the results of surveys unquestioningly. (Though never basing any substantial argument upon them). In future, greater scepticism.
References
1. Sunday Times Books 22 July 90.
2. Billig M. 1978 Fascists; a social psychological view of the National Front. London & NY: Harcourt, Brace, Jovanivich 44.
3. Mill W. L. 1990 How Voters Change; the 1987 British election campaign in perspective.
4. TLS 21 Sept 1990.
from Ideological Commentary 53, Autumn 1991.