T14P01. S&T Policy and Evaluation

Topic : Science, Internet and Technology Policy

Chair : Li TANG

Second Chair : Jue Wang

Share to Facebook Share to Twitter Share to Linkedin Share by mail

General Objectives, Research Questions and Scientific Relevance

In the emerging knowledge economy, science and technology (S&T) has been applauded as effective tools to connect the nature world to the human welfare and promote sustainable economic development. Broadly defined, S&T policy includes scholarship investigating the creation and supporting S&T resources and the coordination of S&T activities. Spanning across a wide spectrum S&T policy can be studied from the economic, social, and political perspectives. The demand for S&T policy and evaluation has been escalating over the last decades. Different levels of government, legislatures, public organizations, and other types of funding agencies are increasingly demanding systematic policy and program evaluation. For example, In the US, the 1993 GPRA requires federal agencies to develop and update strategic plans, to establish annual performance targets, and to report annually on program performance. In Japan, the Science and Technology Agency set up a program in Science of Science, Technology and Innovation Policy to assess the economic and social impact of research investment. Similarly, national research funding agencies around the globe have been sponsoring internal projects to evaluate how funded research projects performed. Looking ahead to the next 50 years, we believe that S&T policy and evaluation is in is critical with the information and insights that brings for better decision- making, good governance, and sustainable development for the well-being of all.

This panel aims to advance our understanding on S&T policy evaluation from interdisciplinary perspective. We are interested in both theoretical and methodological (qualitative, quantitative, or mixed-method)studies that further our understanding on evaluating S&T policy and programs,including but not limited to the impacts of government funding, talent program, and R&D assessment through bibliometric, experimental, or comparative approaches. Any systematic assessment of the operation and/or the outcomes of a program or policy, compared to a set of explicit or implicit standards, as a means of contributing to the improvement of the science program or technology policy are particularly welcome.

 

 

Call for papers

The panel of S&T Policy and Evaluation aims to increase recognition and collaboration among the researchers in S&T policy domain, while exchanging advanced research on policy evaluation in selective domains. This panel features multidisciplinary and interdisciplinary scholarship on a range of science and research policy issues. We invite both promising scholars and established researchers to propose a research paper to present their ideas, reflections, and cutting-edge research related to policy evaluation. The panel encourages research relevant to the economics of S&T, research and policy evaluation, as well as S&T management practice. We are interested in both theoretical and methodological studies that advance our understanding on the roles of government in S&T development, such as the impacts of government funding, talent program, and R&D assessment through bibliometric, experimental, or comparative approaches. The following research questions are particularly interesting to be integrated into this proposed panel: What is the current practice of S&T policy? What role should government play in developing S&T? What are the commonalities and uniqueness of evaluating S&T policy compared to other public policy evaluations? What are the common pitfalls or caveats when evaluate S&T policy? Other proposals that are relevant to the panel theme “S&T policy and evaluation” or a broader S&T policy research agenda will also be considered.

ROOM
Block B 5 - 2
Thu 29th
10:30
Session 1

Discussants : Najmoddin Yazdi (Sharif Policy Research Institute (SPRI); Sharif University of Technology)

Discussants : Feng Li (Hohai University)

Evaluation of Science and Technology in China: Historical Evolution and Future Development

LI qiang (Institute of Policy and Management, Chinese Academy of Sciences)

Evaluation of science and technology was initiated in the United States at the beginning of the 20th century. In recent decades, it is very active and plays an important role in policy development, strategic planning, programme implementation, budget adjustment, as well as many other science and technology activities in many countries.

As in China, although the assessment of sci-tech achievements and the appraisal for science awards had already carried out in 1950s, the Institutionalized evaluation and related study did not rise until early 1990s. Based on co-word analysis, co-citation analysis and cross-citation analysis of 3000 more “Sci-tech Evaluation” related papers, we described the evolution of research themes, analyzed the hot issues, mapped the frontier direction and their transform with the transformation of society and reform of science and technology system in China, as well as the academic building, personnel training, research platforms and important research teams. We also analyzed the strategic demand and new trend for future development in the field of evaluation research.

Responsive image
PDF
Back on track: Factors influencing returnee scholars’ regaining transnational capital?

Feng Li (Hohai University)

wangbing Shen (Hohai University)

Returnee talents have become rare academic resources in Chinese universities. Having transnational capital, accumulated through years of overseas experiences, makes returnee scholars extraordinary, compared to local scholars who are educated and trained domestically. Generally, universities expect more from returnee scholars in terms of doing quality research and connecting to overseas academic networks via frequent international collaborations. However, the reality is that, overseas scholars face several challenges (e.g., cultural difference, week local networks, working stress and so on) right after their return, and these challenges may hinder the full exploitation of transnational capital.

 

Based on our observations, most returnees have to deal with two “sets” of opposing strategies in research activities. Firstly, returnee scholars face broken foreign ties and week local ties at the same time. Both keeping connected to prior foreign ties and rebuilding local ties cost time and energy. Due to the importance of local networks in academic recognition and career advancement, some returnees may carry out more domestic collaboration while have great decreases in the tendencies towards international collaboration. Secondly, returnees face the choice of two conflicting research goals like most Chinese scholars do: the quantity goal of getting more research published in a short time and the quality goal of pursuing quality research regardless of research productivity. Because of the unique research culture in China where quantitative evaluation tools are overused, as well as the stress to pass tenure evaluation in a period of three to five years, some returnees may choose a quantity strategy over a quality strategy. Unfortunately, both the local networking strategy and the quantity strategy may devalue returnees’ transnational capital.

 

Through a preliminary analysis, we found that most returnees’ research quality measured by journal impact had a “V” shape trend — decreased during the first few years of returning, and began to improve afterwards. The changing dynamics of research quality indicates that returnees’ strategy choices may change from the quantity strategy to the quality strategy. Therefore, returnees’ transnational capital may be devalued at the beginning, but later on, some returnees are capable of getting back on track by regaining their transnational capital.

 

Extant literature mostly focused on returnees’ academic contribution, while the changing dynamics of returnees’ performance has not been studied sufficiently. This paper aims to fill this gap by analyzing the research performance of 249 returnee professors recruited by Chang Jiang Scholars Program. These professors spread across four research fields: Mathematics, Physics, Life Sciences and Environmental Sciences. Our study will focus on three research questions:

  1. How do the research quantity and quality interact before and after professors’ return?
  2. Do returnees have higher research quality in the after-return period than in the staying-abroad period? If yes to this question, then we have,
  3. How long does it take to surpass the staying-abroad research quality?

Beyond these questions, we will investigate the disciplinary difference, the personal and institutional factors in returnees’ changing research performance. The results of this study will have some implications for the policy-making in returnee talents’ recruitment, retaining and evaluation.

Responsive image
PDF
Analysis and evaluation of Chinese overseas high-level talents support programs

Jianzhong Zhou (Institutes of Science and Development, Chinese Academy of Sciences)

Fan Chunliang (Institutes of Science and Development, Chinese Academy of Sciences)

Abstract:In recent years, with the steady increase of economy and development of S&T in China, especially since the 2008 global financial crisis, on the one hand Chinese government has increased investment in S&T and infrastructure. On the other hand various talents programs have been launched to attract global high-level talents which results in the brain drain gradually transformed into brain gain in China. Generally, overseas high-level talents have played important roles in promoting the overall level and strength of Chinese science and technology, expanding international cooperation and exchanges, and enhancing Chinese scientific community's international status and influence. However, it is still worth analyzing and evaluating Chinese overseas high-level talents support programs for more efficient policy-making.

In this work, we attempted to carry out a comparison study among three Chinese typical and significant overseas talent programs: "Hundreds of People Plan" launched by CAS, “Yangtze Scholars Program” launched by the Ministry of Education, and "Thousands of People Plan " launched by Organization Department of the CPC Central Committee. The overall analysis is carried out from two aspects: one is to explore the status and problems of winners of three projects (a total of 5098 samples) through a CV method based quantitative analysis, such as studying background, returning locations, research filed as well as their international organization positions. Further, a questionnaire based survey has been applied to researchers in universities, research institutes and companies. We are interested to find out their attitudes on the fairness and reasonableness of those talents programs, as well as the influence factors which may affect researchers’ returning back to China. Based on this research, we proposed that the introduction of high-level overseas talents programs need top-level design and demand analysis, and a more relaxed policy environment is required to attract more foreign top scientists to work in China.

Sino-German Research Collaboration: Evidence from Highly Cited Papers

Guangyuan Hu

LIU Weishu (Zhejiang University of Finance and Economics)

In the emerging knowledge economy, scientific pursuit in the form of international collaboration has been escalating. Studies consistently report that collaboration cross national borders is not only common among advanced economies but also emerging scientific nations. This has aroused interest from social scientists and captured the attention of policymakers. Indeed, recognizing its importance as a means of monitoring and exploiting other countries’ R&D investment, more and more countries champion and participate in international joint research. International collaboration between Germany and China is very interesting but insufficiently explored yet. This study seeks to address some of research gap by profiling the Germany-China collaboration and its impact on the most influential research highly cited papers. We a special group of highly cited paper, the driver of Germany-China collaboration as an instrument to examine the knowledge diffusion and associated impact on China’s most influential research development. The study found evidence in support of the positive impact of international collaboration on the quality of China’ research. It also reveals Germany-China collaboration has a diminishing effect over time on China’s research quality at individual paper level, but not at journal level. Policy implications are discussed in the end.

Responsive image
PDF
Categorization of Science, Technology and Innovation (STI) Indicators’ Frameworks: Purpose and Functions

Najmoddin Yazdi (Sharif Policy Research Institute (SPRI); Sharif University of Technology)

Ali Maleki (Sharif Policy Research Institute (SPRI); Sharif University of Technology)

In today’s world, where knowledge-based economy is playing a major role in promotion of welfare and sustainable economic development of nations, the demand for evaluation of science, technology and innovation (STI) has been escalating. Monitoring and evaluation (M&E) of science and technology (S&T) at country-level usually comes with STI indicators and the framework accommodating them.

The problem is that when designing a national STI framework as a role of government, policy-makers would encounter a lot of diverse frameworks in connection with STI area being developed by international organizations and countries. Dealing with this plethora of frameworks requires skills to compare, categorize, benchmark and utilize them in order to best leverage previous fragmented efforts. This paper extends the recently published research, in which four criteria were suggested for categorization and comparison of STI indicator frameworks, including Coverage of STI (comprehensiveness), Implementation, logical Simplicity and Comparability of a framework, its components and indicators (CISC model). The present extension suggests purpose and functions of frameworks as two complementary criteria for categorization of 22 well-known STI indicator frameworks.

By STI indicator frameworks, it is meant to cover models as diverse as Global Innovation Index (GII), Knowledge Economy Framework of World Bank, EU Innovation Union Scoreboard, STI Framework of UNCTAD for developing countries (2010), US National Science Board’s (NSB) model (2012), Competitive Industrial Performance of UNIDO (2001), Global Competitiveness Report of World Economic Forum (WEF), ArCo Model of Technological Capability (2004), framework of Canberra Manual of OECD (1995), National Innovative Capacity of Furman et al. (2002), Schematic Diagram of National System of Innovation (NSI) of UNCTAD (2011) and Technology Life Cycle Framework of Tassey (2011). While title, goal and positioning of the 22 studied frameworks may seem diverge and sometimes their comparison peculiar, it is the logical framework behind and their connection with STI areas in various ways which has made them nominated for the analysis. These categorization criteria are the results of literature review and continuous discussions with academics and practitioners involved with designing an indicators framework for monitoring and evaluation of Iran’s STI since 2013.

The results indicate that criteria of purpose and functions covered by a STI indicators framework could provide valuable insights for categorization and comparison of them. By purpose criterion, STI frameworks could be classified as oriented toward development of 1) science, 2) R&D, 3) technology, 4) innovation, 5) economic growth, 6) human resources or 7) sustainability. By functions criterion, one may seek if a framework covers 1) human resources, labor and education areas; 2) firms and private sector; 3) government and institutions; 4) ICT and other physical infrastructures; 5) educational, trade, legal, intellectual property, business and labor environments; 6) market, export and trade outputs; 7) R&D activities and resources; and 8) STI expenditures and finance. The resulting categorization is presented in a user-friendly visualization for the 22 frameworks.

The categorization would benefit governments and policy-makers in setting up STI indicator frameworks, while comparing and leveraging diverse international and national frameworks available based on their purpose and functions covered.

ROOM
Block B 5 - 2
Thu 29th
13:30
Session 2

Discussants : Najmoddin Yazdi (Sharif Policy Research Institute (SPRI); Sharif University of Technology)

Discussants : Feng Li (Hohai University)

Responsive image
PDF
Innovation Systems for Agricultural Countries: Comparing Malaysia’s Palm Oil and Vietnam’s Rice Industries

Lim Guanie (Nanyang Centre for Public Administration (Nanyang Technological University))

Chan Yuan Wong

This paper analyses the innovation systems of two agriculture-oriented economies, Malaysia and Vietnam. The Malaysian palm oil and Vietnamese rice industries represent two of the most renowned crops powering Malaysia’s and Vietnam’s transformation from backward economies to two of the developing world’s best performers in recent times. The paper compares the manners in which different developing countries create and capture value from the cultivation and marketization of a non-food crop (palm oil) and a food crop (rice) respectively. It analyses this issue by engaging with the literature on innovation systems from the national level down to the sector-specific level, relying on a mixed-methods approach.


This paper asks: ‘How do developing countries take advantage of seemingly labour-intensive and low value-added agro-based industries to promote economic growth?’ It hypothesizes that a well-designed sectoral system of innovation (SSI) can help achieve such objectives, generating sustainable gains and profits for the relevant stakeholders. It illustrates this argument by examining the critical private and public sector elements of the SSI of both the Malaysian palm oil and Vietnamese rice industries. In a bid to further our understanding on science and technology policy, particularly with reference to that of the developing countries, this paper is relevant as it explores in detail the nature, structure, and dynamics of such elements in both industries.

Responsive image
PDF
Science and Technology Policy Paradox in Iran: Analyzing the role of power plays and Institutions

Ali Maleki (Sharif Policy Research Institute (SPRI); Sharif University of Technology)

Ali Babaee

Rouholah Hamidimotlagh (Sharif University of Technology)

This paper aims to analyze and explain a Science and Technology Policy paradox in Iran. Science and Technology policy in Iran has been put on top of policy agenda over last two decades. However, it is an example of a strange policy paradox. Indicators of scientific outputs such as the number of international journal articles have experienced a considerable growth, following a popular concentration of several top-level policy documents on development of science and technology for future prosperity of the country. In contrast, technology and innovation performance of the country economy has been relatively poor. In addition, indicators of the knowledge economy does not show the expected outcome which predicted as the outcome of scientific growth. We recommend a two-level analysis, using document analysis combined with more than 20 semi-structured interviews to find-out why this policy paradox emerged. Employing a theoretical framework drawn from the literature of ‘catch-up’ and ‘innovation studies’ in the context of developing countries, we explore the root cause of this paradox. At first level, we found two main factors. (1) Iran’ industrial policy has largely failed to face the Iranian firms into global competition, with lack of sufficient push for export performance. As a result, there is no enough demand for innovation and more advanced science and technology in many Iranian industries. (2) Iranian universities and the research system have largely failed to depart from their traditional educational orientation and their “Ivory tower” approach, while many successful catching-up countries have transformed their university and research system into a “solution provider”, contributing to their economic and industrial competitiveness. Instead, Iranian universities have become a “paper production machine” which are not much concerned of the society challenges. At the second level of analysis, we provide a deeper explanation. Failure of the industrial policy could be attributed to the historical excessive power of industry incumbents which prevents the challengers to emerge and block constructive institutional changes. The particular orientation of the university system also goes back to type of institutions rules of the game which dominate the system, emerged from mistaken linear approach to science and technology policy. The main contribution of this research is to illustrate how “the theory of the fields” explain the role of incumbents and unsuitable institutions could prevent the industrial and technology catch-up to happen.

Responsive image
PDF
Nurturing Young Researchers at Change - “Young Researchers Support Program”

Simon Byung Jin Lee (National Research Foundation of Korea / Tallinn University of Technology)

The basic research is regarded as one of the major determinant in the level of science and technology development of the nation. The level of basic research in the long run is mostly affected by the research activities of young researchers, which are activated by the support at the right timing.

 

The young researchers in general are supported from universities and government. Due to lack of support from the universities, the young researchers depend on the support from the national R&D program.

 

“Young Researchers Support Program” supports young researchers in science and engineering with a view to strengthening the research competence and expanding Korea’s overall research infrastructure. The program, which is based on the concept so called “Grass Roots Research Support,” is classified into 3 types - Type Ⅰ(research funding support), Type Ⅱ(research funding and facility support), and Type Ⅲ(research support for outstanding young researcher).

 

However, several drawbacks have been observed which may lessen the performance of the program. The study is designed to find remedies against drawbacks. In order to find remedies, the study will look at restructuring measures to improve the program. The research will introduce a new budget system so as to increase the size of the research support by the Start-up Package.

Responsive image
PDF
Organization Design and Resources Allocation: Analysis Based on Chinese Scientific Research Funds Management System

du baogui (School of Humanities&Law Northeastern University)

men lixiang

it elaborates the basic constitution of Chinese science and technology management organizational architecture, analyzes the problems of science and technology management organization design and resources allocation of Chinese government, including repeated setting of institutions, and obscure definition of functions; low coordination efficiency and high difficulty due to the similar management tier authority of the sector; segmentation of science and technology funds management sectors leading to low utilization efficiency of resources; convergence of local and central management architecture leading to insufficient vertical communication and coordination, and then proposes the path to optimize Chinese science and technology organization design and science and technology resource allocation, i.e. enhance top level design to lawfully clarify the function and power of the central science and technology management departments, straighten out the science and technology funds management system to establish central finance science and technology budget planning and coordination mechanism; define the division of labor and positioning of the central and local government to build regular communication and coordination mechanism.

Where are you from? The omission of author address information in Web of Science

LIU Weishu (Zhejiang University of Finance and Economics)

Guangyuan Hu

Li TANG

The Web of Science database is a widely used data source in S&T policy research. Author address derived indicators (e.g., performance and collaboration of institutions, regions, and countries/territories) are commonly used in these studies. However, the author address missing problem in Web of Science has not attracted enough attention from scholars and practitioners. The characteristics of address missing records are also under-explored. This study probes the author address missing problem over time, among document types and publishing languages in SCIE, SSCI, and A&HCI databases respectively. We find that address missing rate varies greatly during different periods, among different document types and publishing languages. The address missing problem was serious for records published before 1973 in SCIE and SSCI databases and even for publications indexed recently in A&HCI. Although the address missing rate has dropped during the past few decades, the address missing problem is still not negligible and need to be emphasized in practice. We argue that despite its great value, Web of Science database should be used with caution.

The Problems and Suggestions of China 's S&T Evaluation: Evidence from Blog

Tao Dai (Institutions of Science and Development, Chinese Academy of Sciences)

The problems of S&T evaluation have become an important issue restricting the development of science and technology in China, which has aroused widespread concern. The development of information technology has changed the way people communicate, such as the blog provides a platform for free expression of personal views. Compared to academic articles, blog articles are not very strong academic and normative, but better reflect people's thinking. This study provides a new perspective to analyze the problems and suggestions of China‘s S & T evaluation. In this paper, content analysis and bibliometric methods are used to analyze the blog articles about S&T evaluation in the past 10 years. The changes of China’s S & T evaluation focus and the evolution of the problems are described. Finally, suggestions of reforming China’s S&T evaluation are offered.