Manufacturing Visions (ManVis)


Online Foresight Guide

1 Why Foresight?

2 Designing the exercise

3 Running an exercise

 

Why Foresight?

Reasons for undertaking Foresight

The ManVis project was launched in response to the following factors:

The results from previous Foresight activities and an empirical survey indicated that manufacturing in Europe needs to strengthen its innovative capacity to get into a more proactive position in the face of the increasing pace of product innovation. The need was therefore felt for policy action at both European and national level to support such a transformation.

The increasing debate on relocation of manufacturing to locations outside of Europe and the resulting fear of loss of European competitiveness, employment and quality of life.

The activities of the European Commission (DG RTD Industrial technologies) in support of manufacturing (Manufacturing Action Plan – MATAP) needed to be underpinned by visions of future states of manufacturing and possible paths towards these visions.

As the 7th framework programme for research funding was being developed, priorities for research funding had to be revised and – if necessary redefined – within the whole range of topics that are subject to research funding. ManVis was to support this process in the area of industrial technologies and specifically production processes. The objective was to make sure that the priorities were supporting European manufacturing to meet not only current but also future challenges.

Many Foresight studies already existed but mainly on national level, no vision for Europe as a whole.

 

Designing the exercise

Context

The ManVis project took place in the context of various initiatives from the European Commission aiming to address the challenges posed by the future of manufacturing on a European level. Thus, it was explicitly targeting the problem from the perspective of EU policy making. The NMP unit of DG research of the European Commission which is responsible for research funding in the realm of manufacturing had long been aiming at basing their funding decisions on a coherent vision of sustainable and competitive manufacturing in Europe. Therefore, it had financed the 'FutMan' project which was a fully fledged Foresight exercise within the 5th framework programme. The FutMan project, which was finished in 2003, was exploring the 'Future of Manufacturing in Europe 2015-2020' with a focus on 'The challenge for Sustainability'. Within the FutMan project scenarios on future manufacturing patterns had been developed together with a number of detailed case studies on selected sectors and crosscutting themes. At the end of the FutMan project DG Research initiated the Manufuture initiative which aimed to bring together various actors concerned with the future of Manufacturing in Europe, especially industry representatives, to support their strategic decisions and to build up a technology platform on manufacturing. A high level expert group was established and elaborated a vision document as well as a strategic research agenda.

Another actor aiming in the same direction was the EUREKA Factory initiative that is financing concerted national research projects on manufacturing. To generate anticipatory intelligence on manufacturing related issues, EUREKA factory financed a project that was collecting, structuring and analysing currently available studies on the Future of manufacturing (Informan).

The ManVis project consortium was partly formed by research institutes that had been taking part in at least one of these activities. Therefore, there the team was aware of this part of the political landscape and was able to build on the results and experience. The NMP unit, as the client, ensured a close cooperation with the Manufuture initiative, so e.g. they asked the ManVis coordinator to participate in various meetings of the Manufuture group. Some of the ManVis results were taken up in the work of the Manufuture high level group and in turn the ManVis team reacted on the Manufuture documents in their various reports. The ManVis team supported one of the Manufuture conferences with substantial input from the Delphi survey results. Nevertheless, due to time constraints and also different perspectives of the two groups a real exchange in terms of content and results between ManVis and Manufuture was not achieved.

While the ManVis project was being implemented a new emphasis on manufacturing emerged within the European Commissions industrial policy (DG Enterprise and Industry). A Communication was launched outlining a 'Policy framework to strengthen EU manufacturing' (COM(2005) 474 final) comprising several measures to support manufacturing competitiveness in Europe. This highly relevant EU level policy initiative was in no way connected with the ManVis project. Although this did not have direct negative impact on the project it can be concluded that an opportunity to support an ongoing policy making was missed. This could partly be attributed to an incomplete analysis of the political landscape. On the other hand it is due to typical features of the policy making process such as lack of coordination between different actors and fast changing priorities as most Foresight projects have experienced.

Political support

Political support for the ManVis project was secured on various levels. From the beginning there was strong support from the client (Directorate Research, NMP) and this positive attitude remained the same even as the head of unit, who had initiated the project, left. However, the principal support did not lead to a high level of engagement in the process. Although provided several opportunities for the client to give input to the project (e.g. to suggest topics for the Delphi survey) this was not done. Support from other DGs was not provided but had not been actively sought. On the national level several of the partners in the various countries approached government agencies for support of the project. In some countries high level participation of policy makers to the workshops could be achieved in others, ministries provided co-funding to enable additional analysis of the ManVis results highlighting national aspects. The final conference was strongly supported by the Slovenian Ministry of Economy and the Slovenian Ministry of Higher Education, Science and Technology.

Defining the focus

The ManVis project focussed on the issue of how to underpin the competitiveness of the European manufacturing industry. Accordingly, it was primarily an issue driven Foresight. The geographical territory that was considered was Europe but global aspects were taken into consideration to determine the pathways for Europe. The issue to be addressed by the project as well as the European level of the analysis were defined by the need of the client (see objectives).

Lessons learned - focus

It might have been worthwhile to define the focus of the project in closer interaction with the client to improve the impact and usefulness of the outcomes. (see also lessons learned scope).

Setting the objectives

Providing support to European policy makers on all levels for informed decision making on policy actions and research funding towards sustainable and competitive manufacturing in Europe by describing visions of future manufacturing as well as various elements forming pathways towards these visions.

Support European manufacturing industries in facing the long-term challenges of changing socio-economic framework.

Initiating a discourse on manufacturing issues across boarders of nations and disciplines to enhance communication and knowledge flow between actors of manufacturing innovation in Europe thus enhancing the capability of European manufacturing to proactively meet future challenges by self sustaining innovation initiatives.

Defining the users

The prime user of the ManVis results was supposed to be the client and sponsor (DG RTD of the European Commission NMP unit). Other DGs were not actively targeted but it was expected that they would be informed of the outcomes through the Manufuture initiative. However, national governments in the various countries involved were seen as potential users of the results. Also it was reckoned that other stakeholders such as industry and associations would make use of the results. There was no special interface for this target group (originally it had been planned to develop an 'SME manual' on how to use results from manufacturing Foresight for companies strategic planning but this was skipped in the course of the negotiation in the early phase).

Defining the outcomes

Tangible outcomes

Statements on the future of manufacturing

The core project team developed 100 draft Statements on the future of manufacturing to be used within the Delphi survey. These statements were discussed within workshops that were held in all the 22 countries participating in the project. On the base of the outcome of these workshops the statements were revised and finalised by the core team. The final list of statements can in itself be considered as a tangible outcome of the ManVis process as it was the result of a widespread debate among stakeholders all-over Europe.

The statements were relating to the following topics:

Overall manufacturing sector (55 statements):


Sector specific statements on the following sectors:

 

Each county was allowed to add 5 'national specific' statements that would only be answered by experts in their country.

Database from Delphi survey

The online Delphi survey was answered by 2993 experts from all-over Europe. However, not every expert answered all statements. The median number of answers per statement was 1983.

The answers were stored in a database which allowed for an in depth analysis of the results such as comparison between countries, groups of experts (age, type of organisation etc.). The core team members had access to this database to compile the various reports. The national correspondents received the datasets for their respective countries to be able to analyse them in more depth.

Reports

When the project was launched a number of formal deliverables in the form of reports had been agreed upon with the client and sponsor of the project (DG-RTD Industrial Technologies, European Commission). These deliverables were all duly elaborated (although some of them with a time delay):

The emerging changes were supposed to have effects on manufacturing working environment, management concepts, modes of knowledge generation, interaction of manufacturing industry with customers and users as well as on product features. An interactive workshop was held with participants from various stakeholder organisations concerned with these aspects such as trade unions, environmental NGOs, consumer organisations, and students’ representatives. The participants collected emerging issues in these fields and discussed their perspectives on future developments and its impact on manufacturing. The results of this workshop were documented in depth. The documentation was integrated in the ' Delphi report' (see below).

Intangible Outcomes

Naturally, an assessment of intangible outcomes is not as straightforward as the listing of tangible products. Moreover, it needs some time to judge the durability of intangible outcomes such as new networks or changed attitudes among a community of actors. Nevertheless it is possible to give some indications starting from the targeted intangible outcomes as mentioned in the ManVis proposal.

Discourse across established lines of thinking

The project consortium was formed by 30 partners from all-over Europe with widely differing disciplinary and professional background. These diverse partners had to work closely together to implement the project. This initiated a mutual learning process followed by a new type of discourse between perspectives that had not been linked before. This was especially valid for the core team consisting of seven European organisations concerned with widely differing aspects of the future of manufacturing employing different methodology and tools to investigate it. As most of the deliverables were elaborated jointly among the core team and many tasks especially the development of the Delphi questionnaire were fulfilled together within various meetings an intensive exchange of perspectives took place. Although this process of mutual learning also involved a number of conflicts the understanding of different perspectives on the future of manufacturing was certainly enhanced.

A further extension of this dialogue was triggered through the various workshops which again brought together experts from different realms of manufacturing that had to interact to contribute to the projects deliverable (e.g. propose the Delphi statements or advance the manufacturing scenarios). It is most likely, that all these people will spread the cross-disciplinary discourse on the future of manufacturing will into their various contexts such as companies, associations and policy agencies all-over Europe.

Finally, the Delphi exercise served to carry the debate about the future of manufacturing to an even larger audience of manufacturing experts. Even though experts were free to answer only a subset of the statements (which many chose to do), they were at least confronted with the whole set of problems addressed by the survey. Thereby it can be assumed that the survey served to make people with various professional backgrounds aware of perspectives from other fields.

Within the Manufuture conference where a number of workgroups were fed with ManVis results as well as in the ManVis final conference this discourse was further disseminated.

Networking

Through the many workshops and gatherings that took place through the ManVis project new linkages between various stakeholders in the realm of manufacturing were established. The networking initiated by ManVis ranged from the local scale (i.e. meeting of managers from two companies within a national workshop) up to the European and even global level (e.g. representative from overseas companies discussing ManVis results within manufuture conference). The network embraced actors from industry, research and policy. The networking benefited from a mutual reinforcement with the Manufuture initiative.

Defining the scope

Selection of the topics/ perspective adopted

The perspective to be adopted was holistic as this was perceived to be the only way of deriving meaningful knowledge related to the issue. This was reflected in the selection of topics which was in no way constrained to specific policy fields or fields of expertise.

To determine the topics to be covered within the project, the ManVis project team tried to find out which topics were needed to give a satisfying input to the issue at stake that is: how to secure lasting competitiveness for the European manufacturing sector. The following considerations were made:

The same kind of decision had to be taken when defining the aspects that should be asked in the questionnaire. The team reasoned that to allow for the generation of a vision of sustainable and competitive manufacturing in Europe and also to asses the existing visions within European manufacturing experts it was not enough to ask only about the likelihood or time horizon of the statements. In addition a need was perceived to find out which were the effects and framework conditions attached to these statements to be able to give reasonable recommendations. For this reason the questionnaire covered a range of aspects from quality of life to economic viability. On the other hand there was an urgent need to limit the number of statements and aspects to be asked to limit the time required to answer the survey and thereby raise the reply rate. The choice of topics was therefore guided by a compromise between depth of analysis and practicality.

Another scoping decision was dealing with the sectors of industry to be cobered by the project. The following sectors were selected to be dealt with in more detail:

The sectors were decided by the core team in agreement with the client.

Lessons learned regarding scoping

Although the holistic approach is certainly justified when looking at manufacturing from a research point of view, it turned out that it was not easy to draw conclusions from the results that were meaningful for the technology funding programme of the client. Especially the 'soft issues' (such as organisational aspects and working conditions) that were addressed by the survey remained largely untapped for the recommendations to the client. It turned out to be very difficult to integrate the various outcomes from different perspectives and translate them into useful messages for policy making. Probably the holistic perspective should rather have been targeted through integration of aspects than through separation and add on. One could argue that as policy is not acting in a holistic way (that is different departments addressing different policy realms) it is difficult for Foresight to convey messages from a holistic perspective. On the other hand this way the project was able to pass a strong message to policy about the need to align measures from various realms to enable a transition in European manufacturing. However, this message was partly lost as only one of the actors (i.e. the client received it).

Probably it would have been useful to think more carefully before launching the project how, by whom and for which purpose the results will or could be used and to select the topics based on this assessment instead of mainly relying on the research perspective. It should be kept in mind that a broad range of topics does not automatically add up to a holistic perspective and that it is quite difficult to integrate these topics afterwards into a holistic view. From the ManVis project it can be concluded that the integration should come earlier and be more emphasised than the diversification. It can also be concluded, that a holistic approach concering various aspects might lead to an overloading of a project. In this case the holistic approach considered

To reduce complexity and to ensure an integrated holistic result for at least some of these aspects it seems to be necessary to accept a non-holistic approach for some dimensions.

Time horizon

ManVis did not adopt a particular time horizon but most of the developments discussed were ranging from 2010 to 2020

Time frame

The ManVis project was roughly carried out within two years (2004 and 2005).

Cost/ Funding of the Manufacturing Visions project

The project was funded with 1.57 million Euros by the European Commission (DG RTD G2). Additional funding was provided to some partner through national sources.

The bulk of costs were providing for the work of the project consortium (172 person month). The costs of the workshops were covered through other resources (e.g. partners used their own facilities or were supported by universities etc.). Travel expenses for the experts were not paid except for the demand side stakeholder workshops.

The project was proposed by the coordinator within a competitive call for projects within the 6 th framework programme for research funding of the European Commission. It was evaluated positively and selected for funding. During negotiations between coordinator and client the budget originally foreseen was reduced and some activities cut accordingly.

Skills and competencies

The project demanded a wide range of skills and competencies. Not least of all, the coordination of the large and extremely diverse consortium (30 partners from 22 countries) required considerable coordination efforts which took almost a full person most of the time.

In-depth knowledge of manufacturing issues in the various fields addressed by the project was necessary to set up the various activities ( Delphi survey generation, scenario workshop etc.) in an adequate way. The facilitation of the workshops demanded skilled facilitators able to moderate the exchange of people from very different professional and institutional backgrounds. Beside that, knowledge on Foresight methodology was crucial to carry out the project.

The online Delphi survey was demanding highly skilled programmers but also design skills to ensure user friendliness.

Team/Coordinator

The coordinator of the ManVis project was Fraunhofer-ISI, a German contract research institute with a strong background in innovation studies and Foresight. The group that was carrying out the project is specialised in analysis of manufacturing innovation and had been involved in both preceding projects (FutMan and Informan).

The demand strand was lead by the Foresight group of JRC-IPTS who specialises in Foresight methodology and also has experience with future oriented analysis of manufacturing through the FutMan project which it was coordinating.

The overseas strand was lead by the Cambridge Manufacturing Institute having in depth knowledge of manufacturing as well as well established connections to manufacturing related research and futures analysis in Asia and United States.

Implementation plan

As the ManVis project was carried out as a regular project funded within FP6 it was following the respective administrational requirements. Therefore the implementation plan was elaborated before the start of the project in form of a technical annex of the contract between coordinator, consortium and the client. Accordingly, it contained all the elements required by the contract such as detailed time table, list of deliverables, description of work packages, allocation of resources to tasks and partners.

Methodology

The main method used in the ManVis project was the Delphi method. In the 'demand strand' a scenario type analysis was carried out. In the following paragraphs the main features of the Delphi survey will be outlined.

ManVis Delphi Survey statement generation

The core element of a Delphi survey is a set of statements describing possible future states of the area under investigation. In ManVis the survey comprised a list of 100 statements on a wide range of manufacturing topics. These statements were generated in the following way:

A draft list of statements was prepared by the ManVis core team within a special meeting. It was developed mainly on the base of results from previous studies on the future of manufacturing. The draft list was sent to all 22 national partners. Each partner translated the statements (if thought necessary) and organised a workshop with around 20 national manufacturing experts. These experts were selected using the same criteria as for the respondents of the survey. The coordinator elaborated and distributed clearly defined tasks for the workshops.

The draft statements were to be assessed with respect to clarity. Furthermore each workshop was supposed to rank the statements as to their relevance for European manufacturing and suitability to be included into the survey. For the latter some criteria such as timeframe of realisation had been proposed by the coordinator.

On top of this, experts were invited to propose additional statements on issues that they thought relevant but the statements did not address.

The core team collected all the input of the workshops and decided on the final list of statements on the base of the workshop results.

 

Lessons learned – Statement generation

It was crucial to have a workshop in each country, primarily because of spreading the debate was one of the aims of the projects that was very much fostered through the workshops. However the work done in the workshops was a necessary precondition to be able to derive meaningful results from the survey later on. In almost every workshop there was a long discussion on the meaning of the statement and how this could best be 'translated' into the respective understanding of the country. This was not only a matter of language but even more a matter of different understanding of various topics within different national and cultural contexts. The different perspectives within the countries alone were a result that gave interesting insights into the fragmented nature of the European manufacturing community (so e.g. one statement was ruled out in one country as being too 'unrealistic/futuristic' while in another workshop it was dismissed with the argument that this was 'long realised' and therefore not worth to be dealt with in a future oriented study).

Nevertheless there were some doubts afterwards whether the setting of the workshop was ideal. Due to the strict instructions given by the coordinator the discussion in the workshops focused almost exclusively on the assessment of the list of statements prepared by the core team. On the one hand this enabled a focussed discussion and ensured comparable results from the workshops. On the other hand it turned out that almost all workshops were completely occupied with understanding and interpreting the statements as well as suggesting clarifications or correcting the translation. While this was useful for the carrying out of the survey it somewhat restricted the openness and creativity. Almost no workshop came up with new suggestions for topics to be dealt with in the statements. Probably it would have been better to start with a more open brainstorming session to make better use of the wealth of knowledge available in the workshops.

Almost all workshops were conducted only by the national partners, which on one hand facilitated the discussion of the experts by e.g. eliminating language problems, but on the other hand interesting commends, remarks and the common understanding of the statements could only partly be transmitted to the core team by a written report of the national partner.

Another problem that was encountered during the generation of the statements was the lack of interaction with the client. Although the team tried to engage the client into the discussion on the statements this was not taken up. Certainly it would have been much easier to draw relevant conclusions for the client later on if there had been a clear common understanding of the statements and even more if at least some of the statements would have directly related to topics the NMP unit had an immediate concern. Ideally a discussion on the statements alone would have been a support to the decision making process of the client. Probably, it should have been stressed by the team early on that the cooperation of the client was part of the process. Maybe a workshop, similar to the national ones, should have been organised on European level integrating the EU actors.

ManVis Delphi survey – the design of the questionnaire

For each statement the following aspects were asked in the survey (cf. figure below):

Importance to European manufacturing Industry (4 grades from low to high)

Highest Level R&D (EU, Japan, US, Other, Do not know)

Current position of your country in comparison to Europe (4 grades from top to lagging)

This design is based on previous large scale Delphi studies with some modifications. It was the result of a long discussion of the core team and an attempt to balance the conflicting aims of reducing the effort required to answer the survey while getting as much relevant information out of the results as possible. (See scope)

Carrying out the online survey

The ManVis project carried out the Delphy survey using an online questionnaire. The questionnaire for each Delphi statement was programmed by a company specialised in web design (see picture below). The aspects to be asked in the questionnaire were decided upon by the core team based on previous large scale Delphi exercises (German Delphi, Japanese Delphi, Spanish Delphi, Turkish Delphi) but some modifications were made (see scope). The aspects asked were supposed to be self explanatory but a further explanation was available via the Help (?) Function.

 

The questionnaire was first developed in English language. It was then translated into 15 languages (some partners carried out the survey in English as they saw no language barrier within their expert community). The translation was done by each national partner online directly within the questionnaire via a tool provided by the web design company.

 

Selection of experts to answer the survey

The core team established criteria for the selection of experts to participate in the survey. Each national partner was asked to identify a minimum number of experts according to these criteria. The number was corresponding to the number of people employed in the manufacturing sector in the respective country. The criteria were the following:

Type of Organisation

 

For companies: minimum share of 20% of SME (<250 employees)

 

Type of expertise (to be specified during data selection only if available)

 

Additional Recommendations:

 

Approaching the experts

The experts to answer the survey were selected by each national partner (see selection of participants). A list of experts was forwarded to the programmers who created a personal link for each nominated expert to his/her questionnaire and compiled a standardised database containing the data of all experts to be approached in the survey. Each partner had access to their section of the database to allow them to monitor the communication with the experts, and to send e-mails to them containing the personal link to the questionnaire.

 

In addition a self-registration function was established, where experts could register themselves for the survey via a website. This was done for two purposes:

At a fixed date every expert who was registered in the database received an e-mail by the national ManVis partner in her/his country containing his/her personal link and asking them to answer the survey within a certain period of time (six weeks). This e-mail had been previously prepared by each partner in the respective language. Some partners were using a special tool developed by the programming company to address the experts personally (title and name) others were using a general address. After a certain time the experts received one reminder and after another period a second one. After an agreed period of time the survey and the results database were closed (with some allowance for latecomers). Each national partner was responsible to reach a certain minimum number of replies using whatever means they felt appropriate. Most of the partners choose to call some experts personally to ask them to respond to the survey other partners sent reminding letters by post.

As a motivation to participate, a lottery was announced among the participants where experts who had answered the survey could win a weekend in Slovenia (in connection with the ManVis final conference). In addition the participants of the survey were given access to a restricted area in the ManVis website, where the reports could be downloaded and other information related to the topic was posted.

Answering the survey

Before starting the actual Delphi-survey, the respondents were asked to give some information as to their expertise as well as personal details (age, gender, position, type of organisation). In the next stage they were asked to give their opinion about four visionary statements corresponding to four different basic scenarios on the future of manufacturing. Then they were asked to select two manufacturing sectors they would like to answer in addition to the general part. The questionnaire was automatically assembled according to that choice so each expert had around 100 statements to answer. Then every expert, independent of his/her expertise or choice of sectors, was asked to answer the general section of the questionnaire (that is all four thematic fields). The rationale behind this was, that in order to assess the visions within European expert communities it would be important to know how experts were judging developments in fields outside their immediate concern (e.g. expectations of technical experts on organisational changes and the other way round). However, to be able to access the assessment of specialists in each field for each statement the expert was first asked to estimate his/her degree of expertise (see picture of questionnaire).

The programming of the questionnaire allowed the respondent to view a list of all statements and to have an overview of the answers. The questions could be answered one by one, but it was also possible to jump to a particular section. While answering the survey a bar-lever showed the progress within the whole questionnaire. It was also possible to save the questionnaire to continue answering later on. Before finishing an overview was given as to what answers had been done and which ones were missing. When an expert finished the survey the results were transferred to an SPSS database. Here the data was stored anonymously so it was not possible to link answers to names later on.

For experts preferring to answer the survey on paper it was possible to request a pdf version that could be printed out and answered manually. However, as this meant sending a more than 100 pages document by post and then re-enter the results manually into the database, the team tried to restrict this as much as possible. However, some experts actually asked for a paper version.

2 nd round of Delphi survey

As foreseen within the Delphi methodology the ManVis survey was carried out in two rounds. However, the second round was done differently than in previous applications of the Delphi method. First of all a limited number of statements (20 out of 100) was selected for the second round to avoid frightening off the experts with another long questionnaire. For the second round the core team selected the statements that had produced the most controversial results among various expert groups (e.g. different countries) or else were contradictory to other statements’ results. The second round questionnaire contained small texts discussing these contradictions. This proceeding was seen as a major methodological innovation. In fact the results between the two rounds differed much more than usual within Delphi surveys. As usual in a Delphi survey the reply rate was considerably lower in the second round but still sufficient to draw relevant conclusions.

Lessons learned – Delphi survey

In many respects the experience in ManVis verified that an electronic Delphi survey has many advantages over a paper questionnaire. It is cheaper and quicker to send the questionnaire, and the answers are available as soon as they are released by the expert. As the results are already in an electronic format, they can quickly be processed and analysed. Furthermore, additional features can be provided to the expert while filling in the survey such as statistics on the answers given etc. Finally, it is easier to elaborate the questionnaire with a team of researchers from various countries (e.g. the translation could be done via a joint web workspace). However, there are also drawbacks. In some countries there is a high reluctance to answer e-mails from unknown sources. This problem was partly avoided in ManVis, as the e-mails were sent by the national partners who were in most cases known to the experts. Nevertheless, in some countries the e-mails containing the link were erased by spam filters, this was especially a problem in the Netherlands where there seems to be a particular feeling of 'information overload'. Some countries reported that in some companies it is difficult for experts to work online, which kept them from answering the survey.

A number of experts complained about the time required to answer the questionnaire (ca. 2 hours). Many of them abandoned answering the survey after only one statement most likely because they were annoyed by the complexity and length of the survey. It became clear that the e-mail addressing the experts for the first time should have contained a clear indication on how long the answering would roughly require. In any case it was obvious that the questionnaire was too long. Probably the number of statements and even more the number of aspects asked per statement should have been severely reduced. This is even more valid as it turned out later, when analysing the results, that many aspects and differentiations could not be analysed in a meaningful way for the client, due to lack of time but also because of too much detail. The main lesson learned was that one should start with a clear idea on what kind of conclusion one would like to draw from the results and starting from that, work out the questionnaire. Although this was attempted within ManVis, it was probably not carried out rigorously enough. Another conclusion was that there should have been more extensive pre-testing of the questionnaire which – due to time constraint and high pressure to start the survey - had been done mainly internally and in a sporadic manner. A feedback of external experts on the questionnaire would certainly have improved the questionnaire substantially and also facilitated a better assessment of the results later on.

The same is true for the content of the statements. Some of the outcomes could not be interpreted in a way relevant for future oriented strategic decisions for R&D and other policies as it was not clear what to make of it. When analysing the results the team was often unsure whether different assessments among experts resulted from different opinions or different understanding of the statement especially when comparing expert groups from different countries. This type of problem could have been avoided, if more time and resources would have been spent on the preparation phase. Also, the problem is also part of the intercultural approach. Different understanding of a statement can only be avoided to a certain extent, since the reduced form a short statement instead of a longer text leads to diverse understandings even among one cultural group.

Also when analysing the statements the team concluded that it would have been useful to allow for a normative dimension in the questionnaire. As it was, the experts had no possibility to say whether they would like or dislike a statement to come true. Many experts (especially in Sweden) remarked that they were missing this element of personal assessment. But also for the analysis this caused problems as it was often suspected that normative aspects were hidden in the answers so e.g. when experts were stating that some developments will not take place when indeed they meant to say that they hoped that these would never happen.

Data analysis

The survey was answered by almost 3000 experts. With a few exceptions all countries reached the minimum number of answers that was thought necessary to form a valid subset. However, as it was to be expected, not every expert answered all the statements. The median number of answers was 1289 (see below). To be able to make assessments on 'European manufacturing experts' opinion it was decided to apply a weighting on the results depending on the number of people employed in the manufacturing sector. Otherwise it would have meant that countries that were able to motivate more people to answer the survey would have influenced the survey’s result disproportionably.

Lessons learned – data analysis

The survey produced an enormously huge amount of data. To be able to analyse this data required a lot of work and support from specialists in statistical analysis. Accordingly, it took some time to perform all steps of data preparation before the team could for the first time discuss so much as a diagram showing a ranking of the statements in terms of importance for manufacturing. Especially in the face of high pressure from the client to get results quickly, the task was extremely demanding. The effort needed in this phase had been severely underestimated when planning the project. This is especially problematic as the quality of all future analysis of the survey depends on the quality of the work done in this phase. A very important lesson to be learned from ManVis is therefore, to foresee in advance sufficient capacities for preparing the data from the survey and to warn the client well in advance that closure of the survey does not mean immediate access to impressive and meaningful diagrams.

 

 

Information gathering

Information gathering to provide an input for the Foresight process was crucial for the project in various stages and for various purposes:

Delphi survey statement generation:

The initial information needed to compile the draft set of statements was taken from previous studies on the future of manufacturing (FutMan and InforMan see Landscape) as well as relevant future oriented studies containing information on manufacturing issues (e.g. Japanese Delphi study). In addition, expertise from the core team partners was used to refine the statements especially the sector specific ones. There was no attempt to derive fresh ideas e.g. through interviews with experts in the field, mainly because the previous studies had been recently carried out using extensive interviews throughout Europe but also due to time pressure in the preparation phase.

Information to underpin decisions on the Set up of project

Some decisions such as number of answers to be achieved per country, weighting factors to be applied in the analysis) needed to be based on quantitative information on the manufacturing sector within the countries involved. To achieve this, an in depth search in the relevant databases (CIS, OECD etc.) was carried out.

Global perspective strand

The analysis within the global perspective strand was based on personal interviews with actors in various countries outside Europe (especially Asia) as well as an analysis of Foresight studies in these countries.

Demand perspective strand

The influence of developments within the demand side of manufacturing was one of the aspects that had been neglected by previous studies. This was why ManVis made a particular effort to integrate this aspect in its analysis. However, this meant that the analysis could only partly start from results of the pre-ceding studies. Information gathering for the demand perspective workshop and the scenario workshop involved an analysis of relevant results from socio-economic research (literature review).

 

The main challenges of the design phase – Sum Up

 

Running an exercise

Continuous Adaptation of the process

In the course of the manufacturing visions project the process had to be adapted many times. Partly this was due to a delay that was accumulated from the beginning when the contract preparation took longer than expected. Other changes occurred due to new demands from the clients that had again been caused by political events. Finally some of the ideas outlined in the implementation plan proved to be difficult to be put into practice so the proceeding had to be changed. Some examples that might be useful:

 

Participants

Enrolling participants

Delphi survey:

see methods: Delphi survey

 

Demand strand

The participants of the demand strand workshops were selected following a careful screening of relevant stakeholder organisations dealing with the main areas that had been identified as demand aspects (demands on products, demands on working and living conditions). While the first workshop was involving activists from the various organisations, the participants in the second one were mainly researchers dealing with the topics that the first workshop had put on the agenda.

Involving participants - Lesson learned (Demand strand)

It was far from easy to establish a group of people from the targeted stakeholder organisations to participate in the project. The reasons were manifold. First of all, to many organisations it was not immediately obvious why they should engage in this kind of process. Organisations like consumer associations as well as environmental and social NGOs are focussing on influencing policy decision making in their respective areas. Accordingly, they are used to participate within negotiations on new legislation or standardisation. However, they are not very much familiar with other policy fields such as R&D policy (with the exception of the trade unions that have strongly developed links to this policy area) and often do not see an immediate reason for engaging into this policy field. Therefore, in many cases it was necessary for the team to convince the participants that this exercise was relevant for them. Another problem was the work overload that many of the activists in such organisations are facing. Many of them have an extremely tight agenda so it is not easy to find a date that suits a relevant number of people. Finally few of the organisations are able to pay for the participation of their members in such exercises. For this reason, the stakeholder workshops were the only ones within the ManVis project were the participants travelling expenses were covered by the projects funding.

 

The main challenges of running the exercise – Summing up

Initially, this initiative comprised other Commission services such as education and environment but this was not very much further developed.