P11. Advancing the Understanding of Evidence Use in Public Policymaking – Insights from Multiple Conceptual Approaches
POLICY DESIGN, POLICY ANALYSIS, EXPERTISE AND EVALUATIONCATEGORISATION
KEYWORDS
GENERAL OBJECTIVES, RESEARCH QUESTIONS AND SCIENTIFIC RELEVANCE
The panel will gather papers that attempt to further our understanding of the processes, mechanisms, or normative concerns involved in the use of evidence to inform policymaking. It specifically will call on authors using novel conceptual or methodological approaches to the study of evidence use in policy making, rather than seeking additional case studies using existing methods.
The ideas of ‘evidence based’ or ‘evidence informed’ policymaking are now well established in most public policy sectors. Indeed, there has been a tremendous proliferation of work related to evidence use, including multiple international conferences focussed on evidence use now regularly held. Yet only a small percentage of work under this heading explicitly tries to apply concepts, theories, or methods from the social and political sciences to understand the nature of evidence use processes.
Those works which have studied evidence use from a more conceptually developed position have also taken a number of different approaches. Examples include application of: ideational theories to consider how evidence constructions link to policy ideas (Smith 2013); critical discourse analysis to see how evidence is constructed and framed in relation to political ideologies (Parkhurst 2012); cognitive psychological insights to consider bias in individuals’ understanding of evidence (Cairney 2016, Parkhurst 2017); bounded rationality concepts to consider policymaker choice realities (Cairney 2016); social network analysis to look at who is involved in evidence use and their relationsihps (Oliver and Faul 2018); institutional analysis to study the structures and rules that shape evidence use (Parkhurst, Ettelt, and Hawkins 2018); or Science and Technology Studies approaches considering the performative nature of science advice in shaping boundaries of science-policy interaction (Hoppe 2010).
We posit that there will likely be other ways to provide useful insights into the study of evidence use for policy which draw on these and other novel conceptual, theoretical or methodological approaches. This panel will thus aim to invite scholars who are trying to push the boundaries of knowledge around evidence use in one or more ways, informed by broader concepts and theories, or by applying new methods to the questions.
The panel will ultimately enable a discussion between scholars interested in this area to think about which approaches, theories, or methods are helpful for which questions in the realm of understating evidence use in public policymaking.
References:
Cairney, Paul. 2016. The politics of evidence-based policymaking. London: Palgrave Pivot.
Hoppe, Robert. 2010. "From “knowledge use” towards “boundary work”: sketch of an emerging new agenda for inquiry into science-policy interaction." In Knowledge democracy. Consequences for science, politics and media, edited by Roel in t Veld, 169-186. Heidelberg: Springer.
Oliver, Kathryn, and Moira V Faul. 2018. "Networks and network analysis in evidence, policy and practice." Evidence & Policy: A Journal of Research, Debate and Practice no. 14 (3):369-379.
Parkhurst, Justin. 2017. The politics of evidence: from evidence based policy to the good governance of evidence. Abingdon: Routledge.
Parkhurst, Justin, Stefanie Ettelt, and Benjamin Hawkins, eds. 2018. Evidence use in health policy making: an international public policy perspective. Cham: Palgrave Macmillan.
Parkhurst, Justin O. 2012. "Framing, ideology and evidence: Uganda’s HIV success and the development of PEPFAR’s ‘ABC’ policy for HIV prevention." Evidence and Policy no. 8 (1):19-38.
Smith, Katherine. 2013. Beyond evidence based policy in public health: The interplay of ideas. Basingstoke: Palgrave Macmillan.
CALL FOR PAPERS
We invite papers that attempt to further our understanding of the processes, mechanisms, or normative concerns involved in the use of evidence to inform policymaking. We are specifically interested to hear from authors using novel conceptual or methodological approaches to the study of evidence use in policy making, rather than seeking additional descriptive case studies using well established methods. Our goal is to provide a forum to consider the important questions that remain unanswered in studies of evidence use for public policymaking, and to discuss the approaches, theories, or methods that are helpful for such questions.
The ideas of ‘evidence based’ or ‘evidence informed’ policymaking are now well established in most public policy sectors. Indeed, there has been a tremendous proliferation of work related to evidence use, including multiple international conferences focussed on evidence use now regularly held. Yet only a small percentage of work under this heading explicitly tries to apply concepts, theories, or methods from the social and political sciences to understand key questions and aspects of evidence use processes within policy settings.
Those works which have studied evidence use from a more conceptually developed position have also taken a number of different approaches. Examples include application of: ideational theories to consider how evidence constructions link to policy ideas (Smith 2013); critical discourse analysis to see how evidence is constructed and framed in relation to political ideologies (Parkhurst 2012); cognitive psychological insights to consider bias in individuals’ understanding of evidence (Cairney 2016, Parkhurst 2017); bounded rationality concepts to consider policymaker choice realities (Cairney 2016); social network analysis to look at who is involved in evidence use and their relationsihps (Oliver and Faul 2018); institutional analysis to study the structures and rules that shape evidence use (Parkhurst, Ettelt, and Hawkins 2018); or Science and Technology Studies approaches considering the performative nature of science advice in shaping boundaries of science-policy interaction (Hoppe 2010).
We posit that there will likely be other ways to provide useful insights into the study of evidence use for policy which draw on these and other novel conceptual, theoretical or methodological approaches. This panel thus invites scholars who are trying to push the boundaries of knowledge around evidence use in one or more ways, informed by broader concepts and theories, or by applying new methods to these issues.
References:
Cairney, Paul. 2016. The politics of evidence-based policymaking. London: Palgrave Pivot.
Hoppe, Robert. 2010. "From “knowledge use” towards “boundary work”: sketch of an emerging new agenda for inquiry into science-policy interaction." In Knowledge democracy. Consequences for science, politics and media, edited by Roel in t Veld, 169-186. Heidelberg: Springer.
Oliver, Kathryn, and Moira V Faul. 2018. "Networks and network analysis in evidence, policy and practice." Evidence & Policy: A Journal of Research, Debate and Practice no. 14 (3):369-379.
Parkhurst, Justin. 2017. The politics of evidence: from evidence based policy to the good governance of evidence. Abingdon: Routledge.
Parkhurst, Justin, Stefanie Ettelt, and Benjamin Hawkins, eds. 2018. Evidence use in health policy making: an international public policy perspective. Cham: Palgrave Macmillan.
Parkhurst, Justin O. 2012. "Framing, ideology and evidence: Uganda’s HIV success and the development of PEPFAR’s ‘ABC’ policy for HIV prevention." Evidence and Policy no. 8 (1):19-38.
Smith, Katherine. 2013. Beyond evidence based policy in public health: The interplay of ideas. Basingstoke: Palgrave Macmillan.
CO-AUTHORS
ABSTRACT
In the field of environmental policy, there has been increasing necessity in evidence-based policy making (EBPM). As Cairney (2016, ch.4) acknowledged, research on evidence and environmental policy has made less progress than other fields. It seems to be difficult and inadequate to apply the idea of EBPM, as it is being conducted in other fields, directly to the environmental area for several reasons: diversity of evidences, uncertainties in environmental science, and difficulties in gathering evidence for ethical and practical reasons. The purpose of this paper is to clarify the concept of evidence and to develop a framework for evaluation and utilization of evidence in environmental policy making. What does ‘evidence-based’ practice mean in environmental policies and what merits would be expected? As previous studies on EBPM in other areas have pointed out, the utilization of evidence in policy making mainly allows estimating expected policy outcomes. However, the role of evidence is not limited to econometric forecast; evidence is used more diversely in many aspects of policy making, including results of animal experiments in environmental toxicology and numerical analysis of models in climate science, which are related to the assumptions of policy makers and the appropriateness of policy implementation. We clarify the various effects of evidence in environmental policies by introducing the distinction between ‘effect-validated evidence’ and ‘legislative fact evidence’. We examine then how various pieces of evidence are evaluated in environmental policy making by reviewing the policy formulations to environmental problems in a revision of environmental quality standards of particulate matter in the USA (US EPA 2009) and regulation of a neonicotinoid pesticide in the EU (EFSA 2018). Political decisions of these regulations indicated by case studies are related to the trade-off between scientific rationality and social benefit. The way of dealing with it is a central issue of ‘regulatory science’. Our study does not intend just to articulate the difficulties in environmental policies, but to provide the perspectives for balancing the different types of evidence in the policy process by linking regulatory science with the EBPM. In this regard, this paper would improve the evaluation methods of evidence in policy making at a more general level.
Reference
Cairney, P. 2016. The Politics of Evidence-Based Policy Making, Palgrave.
US EPA. 2009. Integrated Science Assessment for Particulate Matter (Final Report, Dec 2009), National Center for Environmental Assessment-RTP Division Office of Research and Development.
EFSA. 2018. Peer review of the pesticide risk assessment for bees for the active substance imidacloprid considering the uses as seed treatments and granules, EFSA Journal 2018, 16(2): 5178.
CO-AUTHORS
ABSTRACT
Many interventions designed to promote evidence use focus on individuals. Parkhurst, in The Politics of Evidence: From Evidence Based Policy to the Good Governance of Evidence (2017, Routledge), notes that significant effort has gone into developing roles, training, and forums to bridge gaps between the ‘two communities’ of officials and researchers. And yet, as Oliver, Lorrenc & Innvaer noted in 2014 (‘New Directions in Evidence-Based Policy Research: A Critical Analysis of the Literature’), research into policy workers’ experiences has largely concentrated on perceived barriers to and facilitators of evidence use with relatively little work exploring utilisation or engagement: how and why evidence is used (and what forms are used when). Responding to this gap, we present interpretive repertoires as an approach to understanding policy workers’ engagement with evidence in their professional practice.
One reason for the above neglect may be that whereas many fields treat ‘evidence-based’ as a characteristic of practice, in the policy field it is common to find outputs – advice or initiatives –referred to as being evidence-based (or not) but rare to find processes judged in the same way. Moreover, greater evidence use has often been framed rhetorically as an intervention to improve policy quality by reducing the influence of other factors. This situates ‘what works’ knowledge – treated as scientifically objective – as inherently superior to the professional knowledge of policy workers. Both these features minimise the importance of understanding practice in its own right, suggesting that scholarship should focus on simply identifying what affects practitioners’ ability to access and apply the ‘best’ evidence to produce correct policy conclusions.
The first section of this paper establishes a case for focusing on ‘practitioners’ as a key site for understanding evidence use. Drawing on interpretive perspectives, it does so by considering policy work as an inherently humanistic and socially-situated process. Such approaches position policy workers as active and interested participants in policy creation rather than neutral analysts or arbiters, and frame policy processes as occasions for debate and interaction between interests, values, and the like rather than problem-solving exercises. From such viewpoints, evidence use is not an impersonal technical activity but rather involves the expression and influence of deeper social structures on policy workers.
Building on this, the paper’s main section introduces interpretive repertoires as a tool for analysing policy workers’ engagement with evidence. In Discourse and Social Psychology: Beyond Attitudes and Behaviour (Sage, 1987) Potter & Wetherell define these as “recurrently used systems of terms used for characterizing and evaluating actions, events and other phenomena” (149). These socially constructed networks of descriptions, explanations, metaphors, and arguments are used by people to make sense of and interact with a given concept (such as evidence). The final section draws on data from current research, presenting example repertoires used by New Zealand skills policy workers to construct their positions as practitioners, their concepts of evidence, and the meanings assigned to evidence-based policy work. These examples illustrate how using repertoires can contribute to a deeper understanding of evidence utilisation as a social practice.
CO-AUTHORS
ABSTRACT
While there have been important contributions from policy and politics studies on the role of evidence in policy-making (Cairney, 2016; Oliver and Faul, 2018; Parkhurst, 2017; Smith, 2013), there is a need for more research that acknowledges and investigates the politics of Evidence-Informed Policy-Making (EIPM).
Although Foucault dedicated an important part of his research to analysing the role of science and knowledge in policy and society, there remains a lack of studies looking at EIPM via the prism of discourse and the articulation of ‘knowledge’ and ‘evidence’ in policy(Dryzek, 1990; Newman, 2011). Discourse presents three advantages as an approach to analysing evidence use in policy (Howarth, 2013). First, it allows to unpick the origins of a given phenomenon, especially one that is as taken-for-granted as EIPM is in Britain and elsewhere. Second, discourse is also helpful for examining the articulation of meaning – be they ideas, norms, beliefs – in analysing how evidence or knowledge are assessed and built into the policy-making process, or how alliances are built – for instance between researchers/academics/disciplines/centres and policy-makers. Finally, discourse allows the researcher to interpret a given phenomenon, mobilising rich qualitative data alongside theoretical concepts and tools such as rhetorical political analysis (Finlayson, 2007). This paper combines this discursive framework with recent depoliticisation research (Wood and Flinders, 2014)to examine how political choice and contingency are renegotiated via the discourse of EIPM and the emergence of tools such as evidence syntheses and knowledge broker organisations (KBOs) or evidence intermediaries.
What matters to this discursive framework is to uncover the workings of power, often hidden by the mobilisation of notions of evidence and knowledge, in order to highlight alternatives and argue for more negotiated, plural, open and transparent decision-making (Fischer, 2009; Mouffe, 2007). It will also allow to critically reflect in situon the roles, relations and negotiations at play in EIPM.
This framework will be retroductively applied and refined using the cases of knowledge brokering bodies such as UK What Works Centres and affiliates in Wales and Scotland, university spin-outs, and similar bodies worldwide, which sit in-between research/evidence and policy and produce evidence syntheses as well as evidence-based policy recommendations. Different from think tanks, these relatively new bodies have multiplied in number and it is important to examine how and why they have emerged and spread, whether and how they influence policy-making, how they are mobilised and referred to by policy-makers, and how evidence and knowledge are understood and articulated. Building on an ongoing comparative case study of knowledge broker organisations in the United Kingdom and Canada, this paper will present some early findings, notably regarding the importance of hegemonic discourses or narratives surrounding evidence and its use in different countries and political cultures, and the importance of power in understanding the fluctuating role and uses of evidence in policy.
Word count without references: 465
References:
Cairney, P. (2016), The Politics of Evidence-Based Policy Making, Palgrave Macmillan UK, London.
Dryzek, J.S. (1990), Discursive Democracy, Cambridge University Press.
Finlayson, A. (2007), ‘From Beliefs to arguments: Interpretive methodology and rhetorical political analysis’, British Journal of Politics and International Relations, Vol. 9, pp. 545–563.
Fischer, F. (2009), Democracy and Expertise: Reorienting Policy Inquiry, Oxford University Press, Oxford.
Howarth, D. (2013), Poststructuralism and After, Palgrave, London.
Mouffe, C. (2007), ‘Artistic activism and agonistic spaces’, Art and Research, Vol. 1 No. 2, pp. 1–5.
Newman, J. (2011), ‘Boundary troubles: Working the academic–policy interface’, Policy & Politics, Vol. 39 No. 4, pp. 473–484.
Oliver, K. and Faul, M. V. (2018), ‘Networks and network analysis in evidence, policy and practice’, Evidence & Policy: A Journal of Research, Debate and Practice, Vol. 14 No. 3, pp. 369–379.
Parkhurst, J.O. (2017), The Politics of Evidence : From Evidence-Based Policy to the Good Governance of Evidence, Routledge, London.
Smith, K. (2013), Beyond Evidence-Based Policy-Making in Public Health, Palgrave Macmillan, Basingstoke.
Wood, M. and Flinders, M. (2014), ‘Rethinking depoliticisation: Beyond the governmental’, Policy and Politics, Vol. 42 No. 2, pp. 151–170.
CO-AUTHORS
ABSTRACT
Ending teen pregnancy and child marriage have become part of the global agenda especially targeted at low and middle-income countries (LMICs), teen pregnancy, which is closely linked to child marriage, has attracted high attention influenced by political actors in Zambia over the past 5-6 years.
This article, examines how teen pregnancy and child marriage have been framed to be a problem that needs to be prevented and/or eliminated. In this case, two perspectives are discussed, the Social constructionists, that view teen pregnancy as a socially constructed problem and what we have termed as the ‘one-sided view’, which distinguishes it as a ‘serious problem’ associated with negative outcomes in one’s life course. However, there appears to be very scarce research on the social construction of teen pregnancy in Zambia and other LMICs compared to the vast literature from high-income countries (HICs). Most research on teen pregnancy tend to be skewed towards negative effects, impact and consequences relating to low educational attainment, poverty and high disease burden among other things. Through the examination of some selected studies and theoretical analysis on the subject, a few things remain questionable.
We argue that teen pregnancy has been politicized in the following ways; (i) framing of teen pregnancy as a problem to be prevented limits other avenues of problem solving (ii) there is no clear definition of what sort of problem it or whose problem it is, thus creating loopholes in policy intervention (iii) there appears to a clear lack of solid evidence to back up claims of the pursued problem as much of the evidence relies on statistics. Based on these arguments, researchers should begin to conduct more empirical research and apply analysis that is more critical to the social construction of teenage reproductive health matters (especially in the context of LMICs) and how they are framed in order to device appropriate policy action through evidence based policy.
Keywords: Bounded rationality, framing, Social construction, Teen pregnancy/early marriage, Zambia
CO-AUTHORS
ABSTRACT
Most, if not all empirical research on evidence-based policy has three features: it typically focuses on the application of science and scientific expertise on policy; and it is executed by ‘outsider’ researchers who are not part of the public administration or policy-making process but observers of it (e.g. Stevens 2010). Thirdly, the major topical focus is in social policy areas such as health, education and crime (Oliver et al. 2014) This study advances the perspectives on evidence-based policy making by first exploring the role of engineering expertise in policy-making and secondly by deploying an ‘insider’ research perspective (Brannick and Coghlan 2007), and thirdly by topically examining a ‘technical’ domain in energy policy. We first make the case that, although related, science and engineering represent different epistemic communities in relation to policy practice. This difference we argue can give rise to particular styles of interaction that can make the governance of expertise in policy making different to that for science or scientists. We then report on the findings of a study of the relationship between a new engineering team in a UK ministry with a technical portfolio and the policy colleagues they worked with across a range of programme areas. Through 19 interviews with policy officials, we identify a range of interactions that imply a need to consider not just epistemic challenges facing technical experts but also styles of management and approaches to internal deployment of experts within policy organisations. Finally, we argue that ‘insider’ research is an important way of understanding the nuances and lived experience of expert advice, and is critical for underpinning training programmes that help to identify and develop new skills and competencies in this domain.
CO-AUTHORS
ABSTRACT
The energy policy arena does not give the warranted thrust and the rightful place to 'evidence' in terms of- 1) nuanced understanding of energy poverty phenomenon, 2) locale-specific energy demand profiling in rural parts of the Indian sub-continent that entails an in-depth understanding of spaces and cultures of energy consumption, and 3) micro-level, renewable energy resource profiling to service local, small, micro-demands. Evidence in terms of qualitative and quantitative, empirical database can be instrumental in rightly and adequately informing the policy making process. The phenomenon of energy poverty still lies unrecognized in the Indian policy estate that governs the functional framework of the energy planning process. The policy design in respect of providing energy access is skewed and flawed for want of clinical evidence to recognize the signs and symptoms of rural energy poverty. the local electrical load demand is primarily a function of- a) variations in ownership status of different electricity driven appliances, b) temporal span of usage of an appliance, c) flexibility in usage that is totally dependent on consumer behaviour, d)seasonality, local weather conditions, and the levels of community engagement, e) functional typology of the energy service provider, f) grid-based extended supply or an off-grid, stand alone system, g)the design of the energy production system, and h) local political engagements in terms of access to the nodes of influence. There are numerous, scattered pieces of evidence gathered from an empirical and contextual assessment of electrical loads and related patterns of consumption of cooking fuels. The paper presents a strong case in favour of evidence-based policy that could lay the foundation for a robust policy dose to be administered on the policy recipients, the ultimate end-users.
Keywords: energy poverty, evidence-based approach, energy service provider.
CO-AUTHORS
ABSTRACT
The recent literature on evidence-based policy has sought to locate evidence use more clearly in relation to established bodies of policy process theory; more sophisticated understandings of decision-making; normative and institutional constraints on utilisation; and the role socio-economic context plays in shaping and interpreting evidence of effectiveness and its potential for translation. This paper seeks to extend the scope of the discussion by reflecting further on two contemporary aspects of the policy terrain that have not so far featured very prominently in the evidence-based policy debate. The two issues at the heart of the paper are systems thinking and anticipation/foresight activities. In a number of policy areas – for example, public health, homelessness, education – there is a growing interest in thinking holistically and systemically. This can be framed as a route to addressing so-called ‘wicked’ problems more effectively. At the same time, in the face of existential threats such as climate change, there is growing interest in various forms of anticipatory activity to inform policy making for the medium- to long-term. This debate is accompanied by reflections on the temporalities of policy making and how to “lock-in” favoured policy for the long-term. Both these turns in the practice of policy raise profound questions about the nature of knowledge and evidence; the relationship between knowledge and action; the role of politics and depoliticization strategies; and the challenges of integrating the outputs of analysis into policy processes. This paper will draw on relevant debates in the systems and anticipations literatures to argue that grappling with these developments, and the tensions between then, sharpens our understanding of the subtleties of the ways in which evidence is deployed in the policy process. Two themes that emerge as particularly important are pragmatism and performativity.
CO-AUTHORS
ABSTRACT
This paper presents findings from a series of projects to apply integrated (linked administrative and other open source) data and research derived insight to inform local policymaking (2013-2018). The research was co-produced with three senior policy officials from two statutory authorities within the South West of England and responded to their call for evidence of this kind to support policymaking processes and practices. It provided a unique window to study the interaction of policy and evidence, and critically reflect on how, to what extent and under what conditions, evidence can inform decision making (Oliver et al, 2014). We expose and conceptualise the political dimensions of the policy context that influenced policymaking and determined how evidence was used. From this we develop conceptual approach to theorising the conditions for evidence use in local policymaking that takes seriously the politics of policymaking, the local institutional structures and actors that shape evidence use (Parkhurst et al, 2018) and the bounded rationality of policymakers (Cairney, 2016).
In common with public policy scholars, we start from the premise that policymaking is a messy, multifaceted and contingent process that involves contestation, negotiation and co-option of ideas about what is to be governed, by whom, how and to what end (Shore et al, 2011; Lasswell, 1990 [1936]). Evidence arrives/is created in, interacts with and has implications for this context. More recently, sets of ideas about the potential of data and data analytics are emerging in this context; fuelled by powerful rhetoric about the significance for policymaking (Shakespeare, 2013). Despite claims to the contrary (Anderson, 2008) these data practices do not speak for themselves and have to be understood in relation to public policy.
Against this backdrop, the aim of our projects with policymakers was to realise and reflect on the benefits of integrated administrative and other data to policymaking at the local level. This aim was applied to four current policy priorities for the authorities involved; financial hardship, community health, wellbeing services and educational needs. The approach involved a synthesis of existing evidence on the subject of policy concern. This was followed by collaborative and iterative cycles of data curation, analysis, interpretation, action and reflection.
Our findings show that various aspects of the policy context defined and delimited the use of this type of evidence. We found that contextual policymaking modes - for instance, whether policy was urgent in reaction to an external shock or routine as part of a periodic service review – influenced policymakers mentalities towards insights from novel forms of evidence and their perception of how open evidential policy solutions were to consideration. Additionally, the degree of authority over policy aims affected the boundaries of policymakers’ choice, and tempered potential appetite for curious exploration of new policy options. Significantly, the extent to which ownership over the framing of policy problems and possible solutions was discrete or dispersed, and the degree to which these were contested within and between policy stakeholder communities, mattered for both data sharing, deliberation over data insight and decision making.
References:
Anderson, C. (2008, June 23). The end of theory: Will the data deluge make the scientific method obsolete? Wired. Retrieved from https://www.wired.com/2008/06/pb-theory
Cairney, P. (2016). The politics of evidence-based policymaking. London: Palgrave Pivot
Lasswell, H. D. (1990 [1936]). Politics; who gets what, when, how. Gloucester: Peter Smith Publisher.
Oliver, K. Lorenc, T. and Innvær, S. (2014). New directions in evidence-based policy research: a critical analysis of the literature. Health Research Policy and System, 12
Parkhurst, J., Ettelt, S., and Hawkins, B. eds. (2018). Evidence use in health policy making: an international public policy perspective. Cham: Palgrave Macmillan.
Shakespeare, S. (2013). Shakespeare review: An independent review of public sector information. London: Department for Business, Innovation and Skills. Retrieved from https://www.gov.uk/government/publications/shakespeare-review-of-public-sector-information
Shore. C., Wright. S., and Però D. (2011), Policy Worlds: Anthropology and Analysis of Contemporary Power. Oxford: Berghahn.
CO-AUTHORS
ABSTRACT
Introduction
In recent years the research programme on evidence-based policymaking has resulted in more attention paid to how experts are engaged in policy formulation, implementation and evaluation. The literature has become quite broad, using a wide range of concepts and narratives, i.e. epistemic communities, types and sources of expertise (scientific, technocratic, inside or outside from bureaucratic bodies), experts credibility and legitimacy, ethical issues of expertise, relations between technocratic knowledge and value-laden advice, discursive and communicative approaches to analysis of expert engagement in policymaking. This dialogue on experts and expertise recently invigorated with the so-called ‘war on expertise’. I claim that this broad literature could be still enriched with the substantial body of empirical findings and valuable theoretical concepts from cognitive sciences.
Aim of the paper
Therefore, the aim of this paper is to propose and integrative conceptual framework of experts and expertise in policy based on standard policy approaches supplemented with concepts and evidence from cognitive sciences, especially from the fields of judgment and decision making (naturalistic decision making in particular), as well as cognitive and social psychology.
Methods
Two stage mixed methods research design was used for the study. During first stage a systematic literature review was conducted to map relevant theories and concepts from both policy and cognitive sciences. As a result integrative conceptual framework was developed. To verify its descriptive properties it was then applied to study the role of experts in R&D project funding policy in Poland. This empirical stage utilised quantitative data analysis (data derived from expert judgments) and qualitative data analysis (interviews with experts, project applicants and administrartive staff involved in implementation of funding scheeme).
Contribution
I hope that the paper will be informative for both academics and practitioners. Paper shows how integration of the cognitive science concepts into policy science discourse enriches answers for such policy questions as: what does expert knowledge mean (relations between tacit and explicit knowledge)? How experts behave in real-world settings (information search strategies, mental models and pattern recognition, experts heuristics and adaptive strategies)? What should be expected from experts in terms of their performance? What kind of expert fallacies could we expect in policy advice (expert overconfidence, domain specificity)? Practical issues relate to: expert selection, development of cognitive task structure, procedures of expert knowledge elicitation and feedback provision to experts.
CO-AUTHORS
ABSTRACT
The complementary approach in Public Policy: Behavioral and Standard perspective of Law and Economics.
Abstract
In public policy studies, behavioral and standard perspective of law and economics are complementary approaches to explain public problems and to theorize design of public policy. To support this thesis, we discuss the work of Kahneman, Knetsch and Thaler on “endowment effect” in Coase theorem (1990) and the first analysis of the behavioral approach to law and economics (Jolls, Sunstein and Thaler 1998) that show both the scientific explanation and the theoretical assumptions of two prospects are shared. We use the methodology of the “Scientific Research Programmes” (Lakatos 1969) for analyzing the complementarity of these perspectives because this methodology helps us to understand their limits and the common insights of the approaches.
The principal results are that behavioral law and economics is a complementary prospect of the standard perspective: It is not a substitute paradigm, and the use of behavioral law and economics imperatively involves the study of standard law and economics. These theoretical contributions come from an original formulation of behavioral and standard perspective of law and economics as “Scientific Research Programme” (Lakatos 1969).
As far as public policy is concerned, the first conclusion is that it is not necessary to choose one theory over another “because it appears to produce the best results or the most scientific approach” (Cairney 2013, 1). Instead of rejecting one theory, the contrast of insights with scientific standard demonstrate the complementary aspects of the approaches. The second conclusion is both behavioral and standard perspectives of law and economics are useful to design regulation, because each approach addresses the focus of the regulator in different ways, but these perspectives are complementary.
Keywords: Law and Economics; Economic analysis of Law; Public Policy; Behavioral Economics; Epistemology; Scientific Research programmes; Positivism.
CO-AUTHORS
ABSTRACT
In the field of environmental policy, there has been increasing necessity in evidence-based policy making (EBPM). As Cairney (2016, ch.4) acknowledged, research on evidence and environmental policy has made less progress than other fields. It seems to be difficult and inadequate to apply the idea of EBPM, as it is being conducted in other fields, directly to the environmental area for several reasons: diversity of evidences, uncertainties in environmental science, and difficulties in gathering evidence for ethical and practical reasons. The purpose of this paper is to clarify the concept of evidence and to develop a framework for evaluation and utilization of evidence in environmental policy making. What does ‘evidence-based’ practice mean in environmental policies and what merits would be expected? As previous studies on EBPM in other areas have pointed out, the utilization of evidence in policy making mainly allows estimating expected policy outcomes. However, the role of evidence is not limited to econometric forecast; evidence is used more diversely in many aspects of policy making, including results of animal experiments in environmental toxicology and numerical analysis of models in climate science, which are related to the assumptions of policy makers and the appropriateness of policy implementation. We clarify the various effects of evidence in environmental policies by introducing the distinction between ‘effect-validated evidence’ and ‘legislative fact evidence’. We examine then how various pieces of evidence are evaluated in environmental policy making by reviewing the policy formulations to environmental problems in a revision of environmental quality standards of particulate matter in the USA (US EPA 2009) and regulation of a neonicotinoid pesticide in the EU (EFSA 2018). Political decisions of these regulations indicated by case studies are related to the trade-off between scientific rationality and social benefit. The way of dealing with it is a central issue of ‘regulatory science’. Our study does not intend just to articulate the difficulties in environmental policies, but to provide the perspectives for balancing the different types of evidence in the policy process by linking regulatory science with the EBPM. In this regard, this paper would improve the evaluation methods of evidence in policy making at a more general level.
Reference
Cairney, P. 2016. The Politics of Evidence-Based Policy Making, Palgrave.
US EPA. 2009. Integrated Science Assessment for Particulate Matter (Final Report, Dec 2009), National Center for Environmental Assessment-RTP Division Office of Research and Development.
EFSA. 2018. Peer review of the pesticide risk assessment for bees for the active substance imidacloprid considering the uses as seed treatments and granules, EFSA Journal 2018, 16(2): 5178.
CO-AUTHORS
ABSTRACT
Many interventions designed to promote evidence use focus on individuals. Parkhurst, in The Politics of Evidence: From Evidence Based Policy to the Good Governance of Evidence (2017, Routledge), notes that significant effort has gone into developing roles, training, and forums to bridge gaps between the ‘two communities’ of officials and researchers. And yet, as Oliver, Lorrenc & Innvaer noted in 2014 (‘New Directions in Evidence-Based Policy Research: A Critical Analysis of the Literature’), research into policy workers’ experiences has largely concentrated on perceived barriers to and facilitators of evidence use with relatively little work exploring utilisation or engagement: how and why evidence is used (and what forms are used when). Responding to this gap, we present interpretive repertoires as an approach to understanding policy workers’ engagement with evidence in their professional practice.
One reason for the above neglect may be that whereas many fields treat ‘evidence-based’ as a characteristic of practice, in the policy field it is common to find outputs – advice or initiatives –referred to as being evidence-based (or not) but rare to find processes judged in the same way. Moreover, greater evidence use has often been framed rhetorically as an intervention to improve policy quality by reducing the influence of other factors. This situates ‘what works’ knowledge – treated as scientifically objective – as inherently superior to the professional knowledge of policy workers. Both these features minimise the importance of understanding practice in its own right, suggesting that scholarship should focus on simply identifying what affects practitioners’ ability to access and apply the ‘best’ evidence to produce correct policy conclusions.
The first section of this paper establishes a case for focusing on ‘practitioners’ as a key site for understanding evidence use. Drawing on interpretive perspectives, it does so by considering policy work as an inherently humanistic and socially-situated process. Such approaches position policy workers as active and interested participants in policy creation rather than neutral analysts or arbiters, and frame policy processes as occasions for debate and interaction between interests, values, and the like rather than problem-solving exercises. From such viewpoints, evidence use is not an impersonal technical activity but rather involves the expression and influence of deeper social structures on policy workers.
Building on this, the paper’s main section introduces interpretive repertoires as a tool for analysing policy workers’ engagement with evidence. In Discourse and Social Psychology: Beyond Attitudes and Behaviour (Sage, 1987) Potter & Wetherell define these as “recurrently used systems of terms used for characterizing and evaluating actions, events and other phenomena” (149). These socially constructed networks of descriptions, explanations, metaphors, and arguments are used by people to make sense of and interact with a given concept (such as evidence). The final section draws on data from current research, presenting example repertoires used by New Zealand skills policy workers to construct their positions as practitioners, their concepts of evidence, and the meanings assigned to evidence-based policy work. These examples illustrate how using repertoires can contribute to a deeper understanding of evidence utilisation as a social practice.
CO-AUTHORS
ABSTRACT
While there have been important contributions from policy and politics studies on the role of evidence in policy-making (Cairney, 2016; Oliver and Faul, 2018; Parkhurst, 2017; Smith, 2013), there is a need for more research that acknowledges and investigates the politics of Evidence-Informed Policy-Making (EIPM).
Although Foucault dedicated an important part of his research to analysing the role of science and knowledge in policy and society, there remains a lack of studies looking at EIPM via the prism of discourse and the articulation of ‘knowledge’ and ‘evidence’ in policy(Dryzek, 1990; Newman, 2011). Discourse presents three advantages as an approach to analysing evidence use in policy (Howarth, 2013). First, it allows to unpick the origins of a given phenomenon, especially one that is as taken-for-granted as EIPM is in Britain and elsewhere. Second, discourse is also helpful for examining the articulation of meaning – be they ideas, norms, beliefs – in analysing how evidence or knowledge are assessed and built into the policy-making process, or how alliances are built – for instance between researchers/academics/disciplines/centres and policy-makers. Finally, discourse allows the researcher to interpret a given phenomenon, mobilising rich qualitative data alongside theoretical concepts and tools such as rhetorical political analysis (Finlayson, 2007). This paper combines this discursive framework with recent depoliticisation research (Wood and Flinders, 2014)to examine how political choice and contingency are renegotiated via the discourse of EIPM and the emergence of tools such as evidence syntheses and knowledge broker organisations (KBOs) or evidence intermediaries.
What matters to this discursive framework is to uncover the workings of power, often hidden by the mobilisation of notions of evidence and knowledge, in order to highlight alternatives and argue for more negotiated, plural, open and transparent decision-making (Fischer, 2009; Mouffe, 2007). It will also allow to critically reflect in situon the roles, relations and negotiations at play in EIPM.
This framework will be retroductively applied and refined using the cases of knowledge brokering bodies such as UK What Works Centres and affiliates in Wales and Scotland, university spin-outs, and similar bodies worldwide, which sit in-between research/evidence and policy and produce evidence syntheses as well as evidence-based policy recommendations. Different from think tanks, these relatively new bodies have multiplied in number and it is important to examine how and why they have emerged and spread, whether and how they influence policy-making, how they are mobilised and referred to by policy-makers, and how evidence and knowledge are understood and articulated. Building on an ongoing comparative case study of knowledge broker organisations in the United Kingdom and Canada, this paper will present some early findings, notably regarding the importance of hegemonic discourses or narratives surrounding evidence and its use in different countries and political cultures, and the importance of power in understanding the fluctuating role and uses of evidence in policy.
Word count without references: 465
References:
Cairney, P. (2016), The Politics of Evidence-Based Policy Making, Palgrave Macmillan UK, London.
Dryzek, J.S. (1990), Discursive Democracy, Cambridge University Press.
Finlayson, A. (2007), ‘From Beliefs to arguments: Interpretive methodology and rhetorical political analysis’, British Journal of Politics and International Relations, Vol. 9, pp. 545–563.
Fischer, F. (2009), Democracy and Expertise: Reorienting Policy Inquiry, Oxford University Press, Oxford.
Howarth, D. (2013), Poststructuralism and After, Palgrave, London.
Mouffe, C. (2007), ‘Artistic activism and agonistic spaces’, Art and Research, Vol. 1 No. 2, pp. 1–5.
Newman, J. (2011), ‘Boundary troubles: Working the academic–policy interface’, Policy & Politics, Vol. 39 No. 4, pp. 473–484.
Oliver, K. and Faul, M. V. (2018), ‘Networks and network analysis in evidence, policy and practice’, Evidence & Policy: A Journal of Research, Debate and Practice, Vol. 14 No. 3, pp. 369–379.
Parkhurst, J.O. (2017), The Politics of Evidence : From Evidence-Based Policy to the Good Governance of Evidence, Routledge, London.
Smith, K. (2013), Beyond Evidence-Based Policy-Making in Public Health, Palgrave Macmillan, Basingstoke.
Wood, M. and Flinders, M. (2014), ‘Rethinking depoliticisation: Beyond the governmental’, Policy and Politics, Vol. 42 No. 2, pp. 151–170.
CO-AUTHORS
ABSTRACT
Ending teen pregnancy and child marriage have become part of the global agenda especially targeted at low and middle-income countries (LMICs), teen pregnancy, which is closely linked to child marriage, has attracted high attention influenced by political actors in Zambia over the past 5-6 years.
This article, examines how teen pregnancy and child marriage have been framed to be a problem that needs to be prevented and/or eliminated. In this case, two perspectives are discussed, the Social constructionists, that view teen pregnancy as a socially constructed problem and what we have termed as the ‘one-sided view’, which distinguishes it as a ‘serious problem’ associated with negative outcomes in one’s life course. However, there appears to be very scarce research on the social construction of teen pregnancy in Zambia and other LMICs compared to the vast literature from high-income countries (HICs). Most research on teen pregnancy tend to be skewed towards negative effects, impact and consequences relating to low educational attainment, poverty and high disease burden among other things. Through the examination of some selected studies and theoretical analysis on the subject, a few things remain questionable.
We argue that teen pregnancy has been politicized in the following ways; (i) framing of teen pregnancy as a problem to be prevented limits other avenues of problem solving (ii) there is no clear definition of what sort of problem it or whose problem it is, thus creating loopholes in policy intervention (iii) there appears to a clear lack of solid evidence to back up claims of the pursued problem as much of the evidence relies on statistics. Based on these arguments, researchers should begin to conduct more empirical research and apply analysis that is more critical to the social construction of teenage reproductive health matters (especially in the context of LMICs) and how they are framed in order to device appropriate policy action through evidence based policy.
Keywords: Bounded rationality, framing, Social construction, Teen pregnancy/early marriage, Zambia
CO-AUTHORS
ABSTRACT
Most, if not all empirical research on evidence-based policy has three features: it typically focuses on the application of science and scientific expertise on policy; and it is executed by ‘outsider’ researchers who are not part of the public administration or policy-making process but observers of it (e.g. Stevens 2010). Thirdly, the major topical focus is in social policy areas such as health, education and crime (Oliver et al. 2014) This study advances the perspectives on evidence-based policy making by first exploring the role of engineering expertise in policy-making and secondly by deploying an ‘insider’ research perspective (Brannick and Coghlan 2007), and thirdly by topically examining a ‘technical’ domain in energy policy. We first make the case that, although related, science and engineering represent different epistemic communities in relation to policy practice. This difference we argue can give rise to particular styles of interaction that can make the governance of expertise in policy making different to that for science or scientists. We then report on the findings of a study of the relationship between a new engineering team in a UK ministry with a technical portfolio and the policy colleagues they worked with across a range of programme areas. Through 19 interviews with policy officials, we identify a range of interactions that imply a need to consider not just epistemic challenges facing technical experts but also styles of management and approaches to internal deployment of experts within policy organisations. Finally, we argue that ‘insider’ research is an important way of understanding the nuances and lived experience of expert advice, and is critical for underpinning training programmes that help to identify and develop new skills and competencies in this domain.
CO-AUTHORS
ABSTRACT
The energy policy arena does not give the warranted thrust and the rightful place to 'evidence' in terms of- 1) nuanced understanding of energy poverty phenomenon, 2) locale-specific energy demand profiling in rural parts of the Indian sub-continent that entails an in-depth understanding of spaces and cultures of energy consumption, and 3) micro-level, renewable energy resource profiling to service local, small, micro-demands. Evidence in terms of qualitative and quantitative, empirical database can be instrumental in rightly and adequately informing the policy making process. The phenomenon of energy poverty still lies unrecognized in the Indian policy estate that governs the functional framework of the energy planning process. The policy design in respect of providing energy access is skewed and flawed for want of clinical evidence to recognize the signs and symptoms of rural energy poverty. the local electrical load demand is primarily a function of- a) variations in ownership status of different electricity driven appliances, b) temporal span of usage of an appliance, c) flexibility in usage that is totally dependent on consumer behaviour, d)seasonality, local weather conditions, and the levels of community engagement, e) functional typology of the energy service provider, f) grid-based extended supply or an off-grid, stand alone system, g)the design of the energy production system, and h) local political engagements in terms of access to the nodes of influence. There are numerous, scattered pieces of evidence gathered from an empirical and contextual assessment of electrical loads and related patterns of consumption of cooking fuels. The paper presents a strong case in favour of evidence-based policy that could lay the foundation for a robust policy dose to be administered on the policy recipients, the ultimate end-users.
Keywords: energy poverty, evidence-based approach, energy service provider.
CO-AUTHORS
ABSTRACT
The recent literature on evidence-based policy has sought to locate evidence use more clearly in relation to established bodies of policy process theory; more sophisticated understandings of decision-making; normative and institutional constraints on utilisation; and the role socio-economic context plays in shaping and interpreting evidence of effectiveness and its potential for translation. This paper seeks to extend the scope of the discussion by reflecting further on two contemporary aspects of the policy terrain that have not so far featured very prominently in the evidence-based policy debate. The two issues at the heart of the paper are systems thinking and anticipation/foresight activities. In a number of policy areas – for example, public health, homelessness, education – there is a growing interest in thinking holistically and systemically. This can be framed as a route to addressing so-called ‘wicked’ problems more effectively. At the same time, in the face of existential threats such as climate change, there is growing interest in various forms of anticipatory activity to inform policy making for the medium- to long-term. This debate is accompanied by reflections on the temporalities of policy making and how to “lock-in” favoured policy for the long-term. Both these turns in the practice of policy raise profound questions about the nature of knowledge and evidence; the relationship between knowledge and action; the role of politics and depoliticization strategies; and the challenges of integrating the outputs of analysis into policy processes. This paper will draw on relevant debates in the systems and anticipations literatures to argue that grappling with these developments, and the tensions between then, sharpens our understanding of the subtleties of the ways in which evidence is deployed in the policy process. Two themes that emerge as particularly important are pragmatism and performativity.
CO-AUTHORS
ABSTRACT
This paper presents findings from a series of projects to apply integrated (linked administrative and other open source) data and research derived insight to inform local policymaking (2013-2018). The research was co-produced with three senior policy officials from two statutory authorities within the South West of England and responded to their call for evidence of this kind to support policymaking processes and practices. It provided a unique window to study the interaction of policy and evidence, and critically reflect on how, to what extent and under what conditions, evidence can inform decision making (Oliver et al, 2014). We expose and conceptualise the political dimensions of the policy context that influenced policymaking and determined how evidence was used. From this we develop conceptual approach to theorising the conditions for evidence use in local policymaking that takes seriously the politics of policymaking, the local institutional structures and actors that shape evidence use (Parkhurst et al, 2018) and the bounded rationality of policymakers (Cairney, 2016).
In common with public policy scholars, we start from the premise that policymaking is a messy, multifaceted and contingent process that involves contestation, negotiation and co-option of ideas about what is to be governed, by whom, how and to what end (Shore et al, 2011; Lasswell, 1990 [1936]). Evidence arrives/is created in, interacts with and has implications for this context. More recently, sets of ideas about the potential of data and data analytics are emerging in this context; fuelled by powerful rhetoric about the significance for policymaking (Shakespeare, 2013). Despite claims to the contrary (Anderson, 2008) these data practices do not speak for themselves and have to be understood in relation to public policy.
Against this backdrop, the aim of our projects with policymakers was to realise and reflect on the benefits of integrated administrative and other data to policymaking at the local level. This aim was applied to four current policy priorities for the authorities involved; financial hardship, community health, wellbeing services and educational needs. The approach involved a synthesis of existing evidence on the subject of policy concern. This was followed by collaborative and iterative cycles of data curation, analysis, interpretation, action and reflection.
Our findings show that various aspects of the policy context defined and delimited the use of this type of evidence. We found that contextual policymaking modes - for instance, whether policy was urgent in reaction to an external shock or routine as part of a periodic service review – influenced policymakers mentalities towards insights from novel forms of evidence and their perception of how open evidential policy solutions were to consideration. Additionally, the degree of authority over policy aims affected the boundaries of policymakers’ choice, and tempered potential appetite for curious exploration of new policy options. Significantly, the extent to which ownership over the framing of policy problems and possible solutions was discrete or dispersed, and the degree to which these were contested within and between policy stakeholder communities, mattered for both data sharing, deliberation over data insight and decision making.
References:
Anderson, C. (2008, June 23). The end of theory: Will the data deluge make the scientific method obsolete? Wired. Retrieved from https://www.wired.com/2008/06/pb-theory
Cairney, P. (2016). The politics of evidence-based policymaking. London: Palgrave Pivot
Lasswell, H. D. (1990 [1936]). Politics; who gets what, when, how. Gloucester: Peter Smith Publisher.
Oliver, K. Lorenc, T. and Innvær, S. (2014). New directions in evidence-based policy research: a critical analysis of the literature. Health Research Policy and System, 12
Parkhurst, J., Ettelt, S., and Hawkins, B. eds. (2018). Evidence use in health policy making: an international public policy perspective. Cham: Palgrave Macmillan.
Shakespeare, S. (2013). Shakespeare review: An independent review of public sector information. London: Department for Business, Innovation and Skills. Retrieved from https://www.gov.uk/government/publications/shakespeare-review-of-public-sector-information
Shore. C., Wright. S., and Però D. (2011), Policy Worlds: Anthropology and Analysis of Contemporary Power. Oxford: Berghahn.
CO-AUTHORS
ABSTRACT
Introduction
In recent years the research programme on evidence-based policymaking has resulted in more attention paid to how experts are engaged in policy formulation, implementation and evaluation. The literature has become quite broad, using a wide range of concepts and narratives, i.e. epistemic communities, types and sources of expertise (scientific, technocratic, inside or outside from bureaucratic bodies), experts credibility and legitimacy, ethical issues of expertise, relations between technocratic knowledge and value-laden advice, discursive and communicative approaches to analysis of expert engagement in policymaking. This dialogue on experts and expertise recently invigorated with the so-called ‘war on expertise’. I claim that this broad literature could be still enriched with the substantial body of empirical findings and valuable theoretical concepts from cognitive sciences.
Aim of the paper
Therefore, the aim of this paper is to propose and integrative conceptual framework of experts and expertise in policy based on standard policy approaches supplemented with concepts and evidence from cognitive sciences, especially from the fields of judgment and decision making (naturalistic decision making in particular), as well as cognitive and social psychology.
Methods
Two stage mixed methods research design was used for the study. During first stage a systematic literature review was conducted to map relevant theories and concepts from both policy and cognitive sciences. As a result integrative conceptual framework was developed. To verify its descriptive properties it was then applied to study the role of experts in R&D project funding policy in Poland. This empirical stage utilised quantitative data analysis (data derived from expert judgments) and qualitative data analysis (interviews with experts, project applicants and administrartive staff involved in implementation of funding scheeme).
Contribution
I hope that the paper will be informative for both academics and practitioners. Paper shows how integration of the cognitive science concepts into policy science discourse enriches answers for such policy questions as: what does expert knowledge mean (relations between tacit and explicit knowledge)? How experts behave in real-world settings (information search strategies, mental models and pattern recognition, experts heuristics and adaptive strategies)? What should be expected from experts in terms of their performance? What kind of expert fallacies could we expect in policy advice (expert overconfidence, domain specificity)? Practical issues relate to: expert selection, development of cognitive task structure, procedures of expert knowledge elicitation and feedback provision to experts.
CO-AUTHORS
ABSTRACT
The complementary approach in Public Policy: Behavioral and Standard perspective of Law and Economics.
Abstract
In public policy studies, behavioral and standard perspective of law and economics are complementary approaches to explain public problems and to theorize design of public policy. To support this thesis, we discuss the work of Kahneman, Knetsch and Thaler on “endowment effect” in Coase theorem (1990) and the first analysis of the behavioral approach to law and economics (Jolls, Sunstein and Thaler 1998) that show both the scientific explanation and the theoretical assumptions of two prospects are shared. We use the methodology of the “Scientific Research Programmes” (Lakatos 1969) for analyzing the complementarity of these perspectives because this methodology helps us to understand their limits and the common insights of the approaches.
The principal results are that behavioral law and economics is a complementary prospect of the standard perspective: It is not a substitute paradigm, and the use of behavioral law and economics imperatively involves the study of standard law and economics. These theoretical contributions come from an original formulation of behavioral and standard perspective of law and economics as “Scientific Research Programme” (Lakatos 1969).
As far as public policy is concerned, the first conclusion is that it is not necessary to choose one theory over another “because it appears to produce the best results or the most scientific approach” (Cairney 2013, 1). Instead of rejecting one theory, the contrast of insights with scientific standard demonstrate the complementary aspects of the approaches. The second conclusion is both behavioral and standard perspectives of law and economics are useful to design regulation, because each approach addresses the focus of the regulator in different ways, but these perspectives are complementary.
Keywords: Law and Economics; Economic analysis of Law; Public Policy; Behavioral Economics; Epistemology; Scientific Research programmes; Positivism.
EXPORT PDF (FULL)