LIST OF WORKSHOPS

Evidence Based Policy (EBP) as a manifestation of knowledge utilisation informed by data and scientific evidence is a relevant subject of inquiry for public policy and is generally considered quintessential for good governance. Evidence is commonly seen as holding much value for policy makers in solving societal problems and in legitimating their decisions, especially under conditions of uncertainty and complexity. As it currently stands, evidence-based policy is widespread as a leitmotif and normative principle in highly technical policy sectors, such as health or environment policies, but can find itself in good currency also in sectors where the importance of values, beliefs and interests are typically seen to be strongly present. Despite a growing consensus about ‘evidence informed policy’ (Head, 2016) as a more realistic and appropriate label to approach the role of evidence in policy making, a basic epistemological dilemma persists among scholars applying a rational view about the phenomenon, and more constructivists who view the role of evidence as the result of an interplay among actors and resources in an often chaotic and unpredictable environment. Scholars adopting the latter perspective have highlighted how the use of evidence in policies cannot be taken for granted, and is far from a linear process (Sanderson, 2006; Strassheim and Kettunen, 2014). Literature discusses a wide range of drivers and impediments for evidence utilisation (Oliver, 2014a; 2014b; Johnson et al. 2009; Leviton and Hughes, 1981), relating to the characteristics of the evidence supplied, the actor configurations prevailing in a particular policy setting, the attributes of the evidence producer and consumer, or the way how evidence is built and constructed along the policy process -to name but a few of the plethora of conditions influencing evidence use. While our knowledge base on determinants for evidence use is gradually expanding, important theoretical and empirical gaps remain when it comes to understanding the conditions and mechanisms for evidence use, and the specific role which evidence takes in different stages of the policy process.

More in particular, despite the multitude of factors that have been named to be influencing evidence use, we still know little about the technical, organisational, and systemic factors that play a role in this process, and how these interact.

As for the technical side of evidence-based policy making, we lack key insights about the type of capacities (analytical, strategic, or communicative) that facilitate (or hamper) the use of evidence in the policy process. Policy capacity, as a phenomenon, is addressed by a growing scholarly community (Wu et al. 2015; Wu et al. 2017). Yet, insights in this field have only been limitedly connected to the EBP. As such, it remains a puzzle whether there are any (combination of) capacities that are necessary and/or sufficient to make evidence ‘work’ within and across contexts. An in-depth scrutiny of different types of capacities can add value to both literatures, i.e. revolving around policy capacity, or EBP.

In the same line of thinking, at a meso-level, it remains unclear which actor constellations, institutional or organisational arrangements foster or hamper access and use of evidence in the policy process. Within the evaluation field, for instance, researchers have expressed concerns about the limited returns of a strong institutionalisation of policy evaluations (Leeuw and Furubo, 2008; Raimondo and Leeuw, 2021). The question is yet whether and how organisations can promote the utilisation of evidence in an effective way, through ‘smart’ organisational procedures, and when or which institutional arrangements do not matter at all in fostering evidence use.

Similar knowledge deficits exist when approaching evidence based policy from a macro-oriented lens. While literature has shown how different knowledge regimes (Campbell and Pedersen, 2014), advisory systems (Craft and Howlett, 2012), or civic epistemologies (Jasanoff, 2011) may impact how evidence is conceived in a particular setting, the implications for the actual use of evidence in particular settings have hardly been scrutinized. More systematic research is needed on how exactly political, cultural or economic conditions can promote or hamper evidence use, and how mechanisms may unfold in different settings. Having such evidence can help advancing middle-range theories, in line with more realistic understandings (Pawson and Tilley, 1997) of evidence use.

Other than research on influencing factors for evidence use, the field would strongly benefit from a more fine-grained unravelling of the policy process when studying EBP (Cairney 2016). Despite the label, evidence based ‘policy’, studies are often not specific as to the stage of policy making (agenda-setting, formulation, decision, implementation and evaluation) on which different types of evidence have an impact (Oliver et al. 2014a). Similarly, literature is often very concise in terms of the actor configurations using particular types of evidence in these different policy phases. Do politicians resort to different evidence manifestations than bureaucrats, for instance, and why is this so? And what about the role of media, external policy advisors, think tanks, or other experts in this evidence process? Similarly, and importantly, research remains fragmented about the very type of output that evidence generates in different technical, organisational and political settings. On which conditions and through which key mechanisms can evidence impact problem definition, instrument selection, coalition building, instrument calibration, or learning and legitimation? Evidence based policy making is currently dealt with by different literature strands (Oliver and Boaz, 2019), making it challenging to keep the overview of available knowledge in this regard, and rendering it almost impossible to draw any systematic conclusions on the linkage between different types of evidence, the antecedents for evidence use, and the actual impact on the policy process.

This workshop aims to enhance our understanding of how evidence is generated and used in the different phases of the policy process, in an effort to advance the field conceptually, theoretically and practically. Conceptually, it challenges scholars to articulate more explicitly what type of knowledge is more likely to be used in different contexts, and urges scholars to operationalise the influence of knowledge, expertise, or advice in a more robust and rigid way (Christensen 2021). Theoretically, it deliberately opens the black box of how technical, organisational and system level attributes foster or jeopardize the uptake of evidence in different stages of the policy process, and through which key mechanisms this operates. Insights generated also hold much practical potential, and can contribute to more coherent, efficient and effective policy designing, which takes account of the particularities of specific policy settings. 

References

Cairney, P. (2016), The politics of evidence-based policy making. Springer, Palgrave Macmillan.

Campbell, J.L. & Pedersen, O.K. (2014). The National Origins of Policy Ideas: Knowledge Regimes in the United States, France, Germany, and Denmark, Princeton University Press, New Jersey

Christensen, J. (2021), Expert knowledge and policymaking: a multi-disciplinary research agenda, Policy & Politics, 49(3), 455-471

Craft, J., & Howlett, M. (2012). Policy formulation, governance shifts and policy influence: Location and content in policy advisory systems. Journal of Public Policy, 32(2), 79-98.

Head, B. W. (2016). Toward more “evidence‐informed” policy making? Public Administration Review, 76(3), 472-484.

Jasanoff, S. (2011). Designs on nature. Princeton University press.

Johnson, K., Greenseid, L. O., Toal, S. A., King, J. A., Lawrenz, F., & Volkov, B. (2009). Research on evaluation use: A review of the empirical literature from 1986 to 2005. American Journal of Evaluation, 30(3), 377-410.

Leeuw, F. L., & Furubo, J.‑E. (2008). Evaluation Systems: What Are They and Why Study Them? Evaluation, 14(2), 157–169.

Leviton, L. C., & Hughes, E. F. (1981). Research on the utilization of evaluations: A review and synthesis. Evaluation review, 5(4), 525-548.

Oliver, K., Boaz, A. (2019) Transforming evidence for policy and practice: creating space for new conversations.  Palgrave Communications, 5(60): 1-10.

Oliver, K., Innvar, S., Lorenc, T., Woodman, J., & Thomas, J. (2014a). A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC health services research, 14(1), 1-12.

Oliver, K., Lorenc, T., & Innvær, S. (2014b). New directions in evidence-based policy research: a critical analysis of the literature. Health research policy and systems, 12(1), 1-11.

Pawson, R., Tilley, N., & Tilley, N. (1997). Realistic evaluation. Sage.

Raimondo, E., & Leeuw, F. L. (2021). Evaluation systems and bureaucratic capture. Locked in the system and potential avenues for change. In B. Perrin & T. Tyrrell (Eds.), Comparative policy evaluation: Vol 29. Changing bureaucracies: Adapting to uncertainty, and how evaluation can help. Routledge.

Sanderson I (2006) Complexity, ‘practical rationality’ and evidence-based policy making. Policy Politics 34(1):115–132

Strassheim H., Kettunen P. (2014) When does evidence-based policy turn into policy-based evidence? Configurations, contexts and mechanisms. Evidence & Policy 10(2):259–277

Wu, X., Ramesh, M., & Howlett, M. (2015). Policy capacity: A conceptual framework for understanding policy competences and capabilities. Policy and Society, 34(3-4), 165-171.

Wu, X., Howlett, M., & Ramesh, M. (Eds.). (2017). Policy capacity and governance: Assessing governmental competences and capabilities in theory and practice. Springer

This workshop gathers theoretical, conceptual, and empirical contributions that enhance our understanding of how evidence is generated and used in public policy, and through which conditions and mechanisms this process unfolds. We conceive the panel as a forum to exchange research experience about this issue, and welcome contributions relating to different policy domains, institutional settings and countries. Any methodological approach (qualitative and/or quantitative) is possible. We particularly welcome comparative papers, but are also open to single case studies. Papers can both be theoretical or empirical. We are equally eager to receive review papers that take stock of the known knowns and unknown knowns on evidence use in public policy, and which can help us developing a sound research agenda on the phenomenon. 

Substantively, we are seeking papers that deal with different types of evidence, the determinants for evidence use, and the actual use and impact of evidence in different phases of the policy process, also considering policy advice and data driven policies. Which capacities (analytical, strategic, communicative) foster or hamper the use of evidence in the policy process? What other capacities make evidence work within and across contexts? Which actor constellations, institutional or organisational arrangements can promote the utilisation of evidence in an effective way? What is the role of political, cultural or economic or other systemic factors in triggering or jeopardizing evidence use? And what about the output generated by evidence in particular/technical, organisational and political settings. On which conditions and through which key mechanisms can evidence impact problem definition, instrument selection, coalition building, instrument calibration, or learning, evaluation, and legitimation?  Papers givers can use this list of research questions as a source of inspiration, but they may as well tackle alternative questions relating to the focus of the workshop.

We approach evidence in a deliberately broad way: it can include scientific studies produced by academics, but also policy evaluations coming from consultants, policy advice from think tanks, or input by experts. Abstracts should include a clear puzzle, research question, outline of the approach, research design, and main contribution of the proposed paper.

The conveners of the workshop aim to propose a special issue to a policy oriented journal and/or a practitioner oriented journal (such as Policy Design and Practice) with a selection of papers.

Evidence Based Policy (EBP) as a manifestation of knowledge utilisation informed by data and scientific evidence is a relevant subject of inquiry for public policy and is generally considered quintessential for good governance. Evidence is commonly seen as holding much value for policy makers in solving societal problems and in legitimating their decisions, especially under conditions of uncertainty and complexity. While our knowledge base on determinants for evidence use is gradually expanding, important theoretical and empirical gaps remain when it comes to understanding the conditions and mechanisms for evidence use, and the specific role which evidence takes in different stages of the policy process. In particular, we still know little about: a) the technical, organisational, and systemic factors that play a role in this process b) the role of EBP in the different stages of policy making (agenda-setting, formulation, decision, implementation and evaluation) also considering policy advice and data driven policies. This workshop aims to enhance our understanding of how evidence is generated and used in the different phases of the policy process, in an effort to advance the field conceptually, theoretically and practically. Conceptually, it challenges scholars to articulate more explicitly what type of knowledge is more likely to be used in different contexts, and urges scholars to operationalise the influence of knowledge, expertise, or advice in a more robust and rigid way. Theoretically, it deliberately opens the black box of how technical, organisational and system level attributes foster or jeopardize the uptake of evidence in different stages of the policy process, and through which key mechanisms this operates. Insights generated also hold much practical potential, and can contribute to more coherent, efficient and effective policy designing, which takes account of the particularities of specific policy settings.

Maria Tullia Galanti is Associate Professor in Political Science at the Department of Social and Political Sciences at the University of Milan, Italy, where she teaches policy analysis and evaluation. Her current research interest is focused on policy advice, and more precisely on policy advisory systems and analytical capacities in Southern Europe, and on the role of the advisory systems in educational reforms. She is also interested in policy entrepreneurship and leadership in the policy process in comparative perspective. Her work appeared in journals such as Policy & Politics, Policy & Society, Social Policy and Administration.

Valérie Pattyn is Assistant Professor in Public Policy at the Institute of Public Administration of Leiden University, The Netherlands, and is partially affiliated to the KU Leuven Public Governance Institute. Her research program examines the demands for knowledge and advice of policy makers under conditions of uncertainty; the conditions fostering and impeding evidence-informed policy making; and the politics of policy evaluation. In addition, she is involved in research projects about policy advice production and use within and outside the civil service (such as parliaments, think tanks, consultants). She has published in leading policy and evaluation journals on these topics, for instance in Journal of European Public Policy; Policy & Society; American Journal of Evaluation; Evaluation; Journal of Comparative Policy Analysis; Public Management Review.