Quantitative versus qualitative methods

Online Foresight Guide

Foresight usually draws on both quantitative and qualitative approaches as they provide distinctive inputs to the analysis of the problems being dealt with. Quantitative data are often given a great deal of weight, but they should not be allowed to dominate.

Quantitative methods

Quantitative methods place greatest reliance on representing developments numerically. Numerical data, of many types, are useful in thinking about longer-term developments, and to a certain extent they can be useful ways of presenting Foresight results, too.

As in in econometrics, quantitative methods implicitly or explicitly use simple models of some sort. For example, time series extrapolations of trends imply a model in that use time as the "independent variable" – really, as a proxy for unmeasured processes that take place in time. More complex models relate variables together so their mutual influences can be tracked. In so-called dynamic models this is tracked over time, whereas the sort of equilibrium models often employed by economists assume a move from a present state towards a (presumably more balanced) future state. Some quantitative approaches involve experts putting numerical values to developments, or creating such values on the basis of the numbers of people agreeing with particular statements or forecasts (as in Delphi).

Data may be generated in various ways. Secondary data is data that was generated for other purposes, but which we can re-use in our own work – often we can use secondary data from official statistics or academic sources. Sometimes we need to generate our own primary data. The most common sources of data are sample surveys (in which a proportion of a population is systematically sampled: a fairly small proportion can give results that are good estimates for the whole population), or censuses of the population. Many statistics are generated by means of questionnaires and other surveys, where the people concerned are requested to provide information for data collection purposes. Otherwise, data may be "captured" from various sources – as a by-product of people's contact with tax, health or other authorities, and the records that these produce; or from other sources which in some way "capture" their behaviour. (For example, a new source of data is websites, and it is, for example, possible to track the growth of activity in a particular field in various regions by counting up and examining the websites addressing the topic.)

Once we have data in a numerical form, there are a great many quantitative techniques that can be employed in the course of the Foresight exercise. Many statistical tools are employed to determine the relationships that can be found between variables, and most good basic textbooks of statistics and data analysis will discuss these techniques and more fundamental procedures such as how to represent averages, trends, etc.

There are considerable advantages to using quantitative methods, which account for the great interest in them. Being able to put information in numerical form means that:

They also have notable disadvantages:

Qualitative methods

Qualitative methods are, of course, often employed where the key trends or developments are hard to capture using simplified indicators, or where such data are not available. In addition, various forms of creative thinking are encouraged by creativity methods. Methods for working systematically with qualitative data are becoming more widely available with the development of Information Technology – tools for "mind mapping" and "conversation analysis", etc. – which can also be helpful devices for facilitating meetings and workshops. For many years the development of qualitative methodologies in social science, as well as in forecasting and Foresight, has lagged behind that of quantitative approaches, and there has often been an explicit or implicit reliance on experts to pull together the strands of qualitative analyses and come up with a synthesis by more or less intuitive means. In the last decade or so this situation has improved considerably, and a great many tools – often computer-based – for capturing and analysing qualitative data, and processing and representing results of such analyses, have become available.

What is the right mix?

In social sciences such as Foresight, the use of one or other type of method has become a matter of controversy and even ideology. Advocates of quantitative methods argue that only by using such methods can social sciences become truly scientific; advocates of qualitative methods have argued that quantitative methods tend to obscure the reality of the social phenomena under study because they underestimate or neglect the non-measurable factors, which may be the most important factors.

The exact mix of methods is highly dependent on access to relevant expertise, and on the nature of the problems being studied. They represent different approaches to handling information, and can contribute powerful insights in their own ways. There is a deeply-rooted tendency to place more weight on statistical information (or quantitative data that may not really merit the term "statistical"), and this is particularly true for the forecasting school of thought. This is misguided: such data can be invaluable in giving a broad overview, in demonstrating the incidence of phenomena, the "representativeness" of example cases or opinions, and the like. But they can rarely probe the dynamics of a phenomenon in any depth, and are restricted to concepts and indicators that are usually quite limited and liable to give only a partial hold on the issues at stake.

The current trend in Foresight is to apply eclectic approaches. In practice Foresight activities can never be completely dominated by quantitative methods and their results. The task is to establish an appropriate role for such methods. Thus, quantitative methods might be used with a globally qualitative framework. And qualitative methods might be used to understand the meaning of the numbers produced by quantitative methods. They may well also involve some quantitative elements. For example, in a scenario workshop participants may be asked to vote on, or rate, topics to be considered in more detail. Statements about factors that need to be taken into account may be captured and grouped according to the frequency with which they have been voiced (for example in conversational analyses, or via the use of computer-conferencing techniques). Equally, qualitative judgements will necessarily inform quantitative activities – the definition of a parameter, the interpretation of a questionnaire item involves qualitative judgements. But perhaps the major issue differentiating the approaches is that qualitative methods still remain less well-documented than quantitative ones, and it can be harder to establish what good practice in applying them to Foresight is. This is particularly true of some of the newer computer-based methods for group-working, and it is likely that most Foresight designers will want to use these in an experimental way only, for the immediate future.