Research & Innovation
Research and Innovation Observatory – Horizon 2020 Policy Support Facility

Service tools

Navigation path

Statistical documentation

EC-JRC - European Commission Joint Research Centre

This report presents the methodology followed to compute the 2014 edition of the Innovation Output Indicator (IOI). The IOI was developed by the European Commission at the request of the European Council in order to benchmark national innovation policies and to monitor the EU’s performance against its main trading partners. The IOI was first presented as a Communication and Staff Working Document1 in 2013, followed by an update in 2014 as part of the 2014 Commission Report on Innovation Union progress at country level (country profiles). The IOI measures the extent to which ideas stemming from innovative sectors are capable of reaching the market, providing better jobs and making Europe more competitive. It covers technological innovation, skills in knowledge-intensive activities, the competitiveness of knowledge-intensive goods and services, and the innovativeness of fast-growing enterprises. It complements the R&D intensity indicator by focusing on innovation output. It aims to support policymakers in establishing new or reinforced actions to remove bottlenecks preventing innovators from translating ideas into successful goods and services. The IOI is a composite of four indicators, chosen for their policy relevance, data quality, International availability and cross-country comparability and robustness. Its four components are:

- technological innovation as measured by patents (PCT);
- employment in knowledge-intensive activities as a percentage of total employment (KIA);
- the average of the share of medium and high-tech goods and services in a countries export (COMP)
- employment dynamism of fast-growing enterprises in innovative sectors (DYN).

By documenting the methodological considerations underlying the IOI, this technical report aims to serve as a reference for future updates.

EC-JRC - European Commission Joint Research Centre

This report proposes a novel way to conceptualise and measure research excellence at the country level using a composite indicator approach. So far, few studies measure scientific and technological research excellence at the country level whilst taking into account the multidimensional nature of research excellence. Following the OECD Oslo Manual, we define research as creative work undertaken on a systematic basis in order to increase the stock of knowledge, including knowledge of man, culture and society, and the use of this stock of knowledge to devise new applications. Akin to the idea of national innovation systems, a national research system is made up of the actors within a country that jointly produce research outcomes. In our conceptual framework, national research systems contain four core elements: components (the operating parts of the system), relationships interactions), attributes (motivations and goals), and outcomes (the creation of excellent knowledge). Scientific and technological research excellence is defined as the top-end quality outcome of systematically performed creative work undertaken to increase the stock of knowledge and new applications. Having evaluated the quality profile of a large set of potential variables, we focus on four variables to measure the top-quality output of scientific and technological research activities at the national level:

1) a field-normalised number of highly cited publications of a country as measured by the top 10 % most cited publications (in all disciplines) per total number of publications (HICIT);
2) the number of high quality patent applications of a country as measured by the number of patents filed under the Patent Cooperation Treaty (PCT) per million inhabitants (PCTPAT);
3) the number of world class universities and research institutes in a country as measured by the number of organisations of a country in the top 250 universities and 50 research institutes divided by gross expenditures in R & D of a country per (TOPINST); and
4) the number of high prestige research grants received by a country as measured by the total value of European Research Council grants received divided by public R & D expenditures of a country (ERC).

The field-normalised number of highly cited publications of a country and the number of high quality patents of a country represent new knowledge attributable to a country that is inscribed in texts and artifacts, the number of world class universities and public research institutes in a country and the number of high prestige research grants received by a country are proxies for monitoring new knowledge that is embodied in the human capital of that country.

EUROSTAT - Statistical Office of the European Union

The present NACE Rev. 2, which is the new revised version of the NACE Rev. 1 and of its minor update NACE Rev. 1.1, is the outcome of a major revision work of the international integrated system of economic classifications which took place between 2000 and 2007. NACE Rev. 2 reflects the technological developments and structural changes of the economy, enabling the modernisation of the Community statistics and contributing, through more comparable and relevant data, to better economic governance at both Community and national level. Development of NACE Rev. 2 has benefited from the work preparing the fourth revision of the United Nations’ International Standard Industrial Classification of All Economic Activities (ISIC Rev. 4). Representatives from Eurostat and Member States of the EU played an important role in this work. NACE Rev. 2 has been created based on ISIC Rev. 4 and adapted to the European circumstances by a working group of experts on statistical classifications from the Member States, candidate Countries as well as EFTA Countries, with the support and guidance of the classification section at Eurostat. Beside the classification, this publication contains also the introductory guidelines containing the main concepts, an historical background and the methodological guidelines for understanding and applying NACE Rev. 2 as well as a detailed description of the different items of the classification.

EUROSTAT - Statistical Office of the European Union

This document is the final description of the work with revising the Nomenclature for the Analysis and Comparison of Scientific Programmes and Budgets (NABS) of 1992 version into the 2007 version and provides a comparison between the two versions. It further contains, as annexes, the final version of the NABS 2007 chapter and sub-chapter headings, the final version of NABS 2007 including the detailed annotations as well as a detailed bridge table between NABS 2007 and NABS 1992. The classification is linked to the Frascati Manual (OECD 2002). The area of socio-economic objectives (SEO) of research and development (R&D) activities is considered in two ways in the Frascati Manual. The first is as a functional classification of the performance of R&D and the second relates to the analysis of government appropriations to R&D (GBOARD). The body responsible for the NABS classification is Eurostat. The NABS nomenclature was originally established in 1969 and earlier revisions were carried out in 1975, 1983, 1992 and recently in 2007.

OECD - The Organisation for Economic Co-operation and Development

It has been long understood that the generation, exploitation and diffusion of knowledge are fundamental to economic growth, development and the well being of nations. Central to this is the need for better measures of innovation. Over time the nature and landscape of innovation have changed, and so has the need for indicators to capture those changes and provide policy makers with appropriate tools of analysis. A considerable body of work was undertaken during the 1980s and 1990s to develop models and analytical frameworks for the study of innovation. Experimentation with early surveys and their results, along with the need for a coherent set of concepts and tools led to the first edition of the Oslo Manual in 1992, which focused on technological product and process (TPP) innovation in manufacturing. This became the reference for various large scale surveys examining the nature and impacts of innovation in the business sector, such as the European Community Innovation Survey (CIS), currently in its fourth round. Results from such surveys have driven further refinements in the Oslo Manual framework in terms of concepts, definitions and methodology leading to second edition published in 1997 which, among other things, expanded coverage to service sectors. Since then, the analysis of results from surveys and changing policy needs led to the launching of another revision of the manual, the result of which can be found in this third edition. As there has been a growing sense that much of innovation in service sectors is not adequately captured by the TPP concept, it was decided to address the question of non technological innovation in this revision. As a result, the scope of what is considered an innovation has now been expanded to include two new types: marketing and organisational innovation. These are certainly new concepts, but they have already been tested in several OECD countries, with promising results. New to this edition is also an effort to address the systemic dimension of innovation, through a chapter focusing on innovation linkages. Lessons drawn from results of previous surveys have also been incorporated in order to refine existing concepts and methodological issues, such as the measurement of innovation inputs and outcomes, as well as the improvement of data collection methods. Innovation also occurs outside the OECD region: a growing number of countries in Latin America, Eastern Europe, Asia and Africa have begun undertaking surveys based on the Oslo Manual. Although the design of those surveys was usually intended to comply with such standards, many of them have adapted the Oslo methodology to take into account specific user needs and the characteristics of statistical systems in these countries with different economic and social backgrounds. National adaptations were developed by each country and followed different approaches. For example, it is widely accepted that diffusion and incremental changes to innovation account for much of the innovation occurring in non OECD countries. Using these rich and diverse experiences, an annex has been added to this edition of the Oslo Manual that draws on some of the lessons learned, and provides further guidance for future innovation surveys in non OECD countries. The Oslo Manual, developed jointly by Eurostat and the OECD, is part of a continuously evolving family of manuals devoted to the measurement and interpretation of data relating to science, technology and innovation. This includes manuals, guidelines and handbooks covering R&D (Frascati Manual), globalisation indicators, patents, the information society, human resources in S&T (Canberra Manual), and biotechnology statistics. Prepared under the joint aegis of the OECD and the European Commission (Eurostat), this third edition of the Oslo Manual is the result of a three year collaborative process that has involved the OECD Working Party of National Experts on Science and Technology Indicators (NESTI) and the Eurostat Working Party on Science, Technology and Innovation Statistics (WPSTI) as well as a number of outside experts. This manual provides guidelines for collecting and interpreting innovation data in an internationally comparable manner. Finding consensus has sometimes meant reaching compromises and agreeing to conventions. As with other such guidelines, there are known limitations, but each edition of the Oslo Manual constitutes a step forward in our understanding of the innovation process. While this ongoing, incremental learning incorporates the lessons of earlier studies, the Manual is also an ambitious tool in which experimentation and testing are used to challenge the boundaries of what is understood by innovation. Many should be thanked for their valuable contributions. A special acknowledgement goes to experts from Canada, France, Germany, Italy, Japan, the Netherlands, Norway and the United Kingdom who led the work of six focus groups which examined a variety of topics and expressed valuable recommendations for the revision. The drafting of the revised Oslo Manual was undertaken by Dr. Peter Mortensen and Dr. Carter Bloch from the Danish Centre for Studies in Research and Research Policy, under the guidance of the OECD and Eurostat. The annex on innovation surveys in developing countries was drafted by the UNESCO Institute for Statistics, based on a proposal and draft paper by the Red Iberoamericana de Indicadores de Ciencia y Tecnología (RICYT) and following a broad process of consultation with many national experts.

OECD - The Organisation for Economic Co-operation and Development

In June 1963, the OECD met with national experts on research and development (R&D) statistics at the Villa Falcioneri in Frascati, Italy. The result was the first official version of the Proposed Standard Practice for Surveys of Research and Development, better known as the Frascati Manual. This publication is the sixth edition. Since the fifth edition was issued in 1994, attention has increasingly been paid to R&D and innovation as key elements in the knowledge-based economy. Reliable and comparable statistics and indicators to monitor this area are of crucial importance. This edition therefore makes an effort to strengthen various methodological recommendations and guidelines, in particular for improving R&D statistics in the services sector and collecting more detailed data on human resources for R&D. Because globalisation is a challenge for R&D surveys, the Manual recommends some changes in classifications in an attempt to address this issue. Today’s R&D statistics are the result of the systematic development of surveys based on the Frascati Manual and are now part of the statistical system of the OECD member countries. Although the Manual is basically a technical document, it is a cornerstone of OECD efforts to increase the understanding of the role played by science and technology by analysing national systems of innovation. Furthermore, by providing internationally accepted definitions of R&D and classifications of its component activities, the Manual contributes to intergovernmental discussions on “best practices” for science and technology policies. The Frascati Manual is not only a standard for R&D surveys in OECD member countries. As a result of initiatives by the OECD, UNESCO, the European Union and various regional organisations, it has become a standard for R&D surveys worldwide. The Frascati Manual is based on experience gained from collecting R&D statistics in OECD member countries. It is a result of the collective work of national experts in NESTI (the Working Party of National Experts on Science and Technology Indicators). The Group, supported by an effective Secretariat, first led by the late Yvan Fabian and subsequently by Alison Young, John Dryden, Daniel Malkin and Andrew Wyckoff, has elaborated over the last 40 years on the concept of science and technology indicators and developed a series of methodological manuals known as the “Frascati Family”, which includes manuals on: R&D (Frascati Manual), innovation (Oslo Manual), human resources (Canberra Manual), technological balance of payments and patents as science and technology indicators. The Frascati Manual is also published in electronic format on the OECD Website. The idea is to update the electronic version more frequently, as newer material becomes available. The electronic version is complemented by further material related to R&D surveys. The sixth edition of the Manual was prepared by teams of experts drawn from the NESTI Group. The OECD Secretariat (especially Dominique Guellec, Laudeline Auriol, Mosahid Khan, Geneviève Muzart and Sharon Standish) played an active role in co-ordinating the process and drafting certain sections. Bill Pattinson (a former Australian NESTI delegate) was responsible for preliminary revisions while working in the OECD. Mikael Åkerblom (Statistics Finland and a Finnish NESTI delegate) worked in the final stage for one year at the OECD to draft the Manual, incorporating various comments and suggestions from NESTI members. Thanks to a generous voluntary contribution to the OECD from the Japanese government, this revision benefited from substantive contributions by experts and proceeded in a timely fashion. Japan’s contribution is gratefully acknowledged. The Manual is published on the responsibility of the Secretary-General of the OECD.

OECD - The Organisation for Economic Co-operation and Development

This Manual is intended to provide guidelines for the measurement of Human Resources devoted to Science and Technology (HRST) and the analysis of such data. It has been prepared in close co-operation between the OECD and the DGXII and Eurostat of the European Commission, other OECD directorates (notably DEELSA), UNESCO and the International Labour Office (ILO) and with the support of national experts. The text was discussed at specialist workshops at the OECD in 1992 and 1993 and then submitted to the Group of National Experts on Science and Technology Indicators (NESTI) at its meeting in Canberra in April 1994. In recognition of the hospitality of the Australian authorities in organising the meeting, a suggestion was made and approved by NESTI that the guidelines should henceforth be known as the "Canberra Manual". The Group recommended that the text be submitted to the Committee for Scientific and Technological Policy for derestriction after incorporation of the proposals of the meeting and subsequent written comments and after professional editing. After these amendments had been made, the document was approved by the Committee and is now made available to the public under the responsibility of the Secretary-General of the OECD.

Last update: 27/02/2017 | Top | Legal notice | Contact | Search