Format

P19. Causal Claims and Causal inferences in Comparative Policy Analysis

COMPARATIVE PUBLIC POLICY
PANEL CHAIR(S)
Iris Geva-May
The Wagner School, NYU and SPPA, Carleton Univ. Canada
Main chair
Guillaume Fontaine
FLACSO Ecuador
Second chair
Philipp Trein
University of Lausanne
Third chair
CATEGORISATION
POLICY TOPIC
COMPARATIVE PUBLIC POLICY
SECTOR
—
KEYWORDS
—
GENERAL OBJECTIVES, RESEARCH QUESTIONS AND SCIENTIFIC RELEVANCE

Causal claims and inferences have become important criteria for research design and methods in comparative policy analysis (CPA). “Why”-questions are often present in CPA, for instance to address why policies on related policy problems are most similar or most different. Degrees of similar or different causes may be the answer. Yet the concept of causality often remains implicit and scholars conceive it in a different way — either as patterns of regularity, counterfactuals or causal forces— according to their philosophical and scientific ontology. Some argue that explicit causal inferences should be at the core of CPA while others contend that causation is not to be explained but, at best, interpreted.
These similarities and discrepancies affect the very theory and practice of CPA, when it comes to address “Why” questions about agenda-setting, policy formulation, adoption, implementation or evaluation. Why does a government put a problem on the agenda? Why is a policy mix preferred to another? Why is a policy adopted or not? Why is there a gap in a policy implementation? Why does a policy produce unexpected outputs or outcomes? 
Such questions are given a partial treatment by conventional analytical frameworks. For instance, the advocacy coalition framework and the multiple stream approach focus on the agenda-setting, while policy design focuses on the instrumentation; comparative historical analysis and political economy have produced strong theories of policy change, while econometrics has been more thorough to assess
policies impacts. Yet while the theoretical statement made by these frameworks are conspicuous, their methodological foundations are not always explicit and their procedures often suffer from a lack of transparency.
Two logics of causation currently coexist in CPA —probabilistic and deterministic— hence raising a problem of alignment between ontology and methodology at the time of selecting a comparative method for confirming or disconfirming purposes. On one hand, the probabilistic logic has produced theories of causation based on mean causal effects, general trends and symmetric causation, which commands variable-centered research designs for large-n comparisons. On the other hand, the deterministic logic has produced theories of causation based on differences in kind, necessity claims and asymmetric causation, which commands case-based research designs for small-n comparisons and within-case studies. The dilemma extends to the conceptualization of causal conditions (necessity and sufficiency), the status of set membership and even the techniques to measure variations in kind or degree. Whatever their choice, scholars´ research agenda derives from different designs and practices, and ultimately ends up making symmetric or asymmetric causal claims, which produce formal models or causal mechanisms, and so on and so forth. Some may contend that only co-variance in large-n comparisons allows to establish a causal relation in a policy process, or that small-n comparisons are a better way to identify the causal forces at stake in that process. Some may advocate for experiments or quasi-experiments while others would object these as inaccurate for practical or even ethical reasons. Eventually a growing number of comparative policy studies advocate for a multi-method research design.

 

 

JUNE 25,

4-6 pm - Preconference JCPA Workshop Keynotes on comparative methods: Beryl Radin and Guy Peters (MB 9EG)

6-7 pm - Reception of the JCPA and ICPA - Forum Scholarly Society (MB 9 lobby)

 

JUNE 27,

12:30-2pm - Lunch meeting, Editorial board and Friends of the JCPA and ICPA-Forum (MB 9A)

 

More info 

CALL FOR PAPERS

The objective of this workshop is to discuss the theoretical implications of different concepts of causal inferences and causal claims in CPA, and in particular how scholars and practitioners identify and deal with  causation in theory and practice. Participants are expected to contribute to the discussion with an 8,000 word paper based on a small-n or a large-n comparison of substantive policies within or in-between social units. 

 

The following questions may be addressed as guidelines for the discussion:

1. What is the nature of causality in your comparative framework?
2. What theory (or theories) of causation has it been supported or produced by?
3. How was this theory operationalized?
4. Which comparative methods have been favored to assess the causal inferences or causal claims?
5. How have these methods combined with others in the research design?
6. What contribution does the research make to CPA methods?


The workshop will be sponsored by the ICPA-Forum scholarly society and the Journal of Comparative Policy Analysis, Routledge on its 20th anniversary, to advance this important CPA issue. Submitted papers may be shortlisted for publication in the JCPA following the journal's standard blind fold reviewing process.

ACCEPTED
Format
01 - An ACF Explanation of the Canadian Firearms Act: Regularity Versus Mechanistic Causation
Tim Heinmiller (Department of Political Science, Brock University)
CO-AUTHORS
—
ABSTRACT

The purpose of this paper is to compare two different causal explanations of the same major policy change, the passage of the Canadian Firearms Act in 1995. The Firearms Act was a major policy change because it introduced universal firearms registration to Canada for the first time. The two causal explanations compared in this paper utilize the same theory – the Advocacy Coalition Framework (ACF) – but are based in different ontological conceptions of causality. The current ACF theory of policy change is based on a deterministic-regularity conception of causality. Based on the empirical evidence, this approach seems to provide a good explanation of the Firearms Act’s adoption: the causal factors posited in the two ACF policy change hypotheses are empirically supported in this case. Nevertheless, the ACF theory of policy change is also compatible with a deterministic-mechanistic conception of causality and this approach provides several causal insights that a deterministic-regularity approach does not. First, it provides a clearer picture of how the two causal hypotheses interact and intersect to produce major policy change. Second, it investigates whether the two necessary conditions posited in the current causal hypotheses are actually necessary for major policy change. Third, it investigates (and potentially resolves) the over-determination problem that occurs in ACF explanations when multiple policy pathways - each of which are considered sufficient conditions within the first ACF policy change hypothesis - are present. By comparing a deterministic-regularity explanation with a deterministic-mechanistic explanation, this paper explores a potentially important avenue for further improving and refining ACF policy change theory.

ACCEPTED
Format
02 - Better understanding of the policy process through randomised designs in the field
Peter John (KCL)
CO-AUTHORS
—
ABSTRACT

In spite of a long pedigree, the use of randomised controlled trials (RCT) to answer general questions in social science has been a recent phenomenon. In the past, the attraction of the RCT was as a robust form of evaluation, testing government programmes or new procedures, where the attraction is the real-world context, the mimicking of an intervention, and then assessing the counterfactual of no intervention. The method continues as a major success story in disciplines interested in testing effectiveness, such as education or social policy, and where other forms of evaluation suffer from familiar problems of faulty comparisons or inconclusive time series/panel designs. In the late 1990s, more interest emerged about how to use RCTs to answer general questions in social science, such as in political science about vote choice or turnout. The big challenge for political science and public policy is that very importance aspects of decision-making involve a small number of actors at elite levels where the possibility of using trials is more challenging. Trials depend on large samples for power and require non-interference to respect the SUTVA assumption, as well as needing access and consent of policy-makers. It would seem that policy process studies in particular is shut off from this important development reinforcing its lag behind methodological developments in other parts of social science. To address this issue, the paper argues that there is potential for the use of RCTs in studies of the policy process from: a) elite responsiveness studies; b) behavioural games with decision-makers; and c) public management simulations with policy-makers, which are important first steps. The paper discusses these contributions and sums up what they mean for classic questions in public policy, such as rationality of decision-making, errors, bias, correction of errors/learning, and innovation. Ideas for future experiments are discussed.

ACCEPTED
Format
03 - Using comparative methods in search of causality: Advancing the state of energy justice research by using a case-oriented approach
Brent Burns (Michigan Technological University)
CO-AUTHORS
—
ABSTRACT

In order to provide more meaningful and impactful suggestions to policymakers, policy scholars should perform causal analysis and seek subsequent causal mechanisms, explaining why the observations and patterns which they have discovered happen.  This deeper level of analysis will lead to more specific conclusions and recommendations that will help decision makers solve problems closer to their root cause, thus preventing problems from re-occurring, rather than focusing on fixing problems when they arise.  The lack of causal analysis in policy studies is evident. This paper focuses on analyzing the literature from one emerging energy policy field, energy justice.  A spin-off field of study from environmental justice theory, energy justice’s driving principle is that all individuals, across all areas, have the right to safe, affordable and sustainable energy, as energy has become a modern primary good required to achieve a basic quality of life.  To date, most studies in this field, whether qualitative or quantitative, are descriptive and fail to seek causality through their research design.  One essential item in search of causality is the in-depth case-oriented approach.  This paper describes how an in-depth case oriented approach can assist researchers in seeking causal inference and subsequent causal mechanisms, a key conclusion missing in the energy justice literature.  Next, this paper summarizes the current state of methods used in energy justice literature, briefly describing specific studies and their methodological approaches, then further illustrating which components of their research design should change in order to utilize the comparative methods, thus allowing better causal analysis.  Lastly, this paper describes some existing studies using comparative methods that demonstrate the proper way shift their focus to causal analysis.  This research will provide insights and evidence to other scholars that shifting research to case-oriented approach can lead to greater causal analysis and more actionable recommendations for policy and decision makers to act upon.

ACCEPTED
Format
04 - From a causal to a consequentialist perspective: a pragmatic approach to compare policy processes differently
Philippe Zittoun (ENTPE - LAET - University of Lyon)
CO-AUTHORS
—
ABSTRACT

Most policy process approaches propose to analyze the policy process as a phenomenon that can be explained causally. Unfortunately, as Mahoney suggests, there are many definitions of causal mechanism in the social sciences literature (Mahoney 2001) which make discussions around causality difficult. Moreover, these multiple definitions make it difficult to clearly identify the key objective of policy research. Given impetus by vibrant debate about process tracing methodologies (Benett, Checkel, 2014; Hall, 2013), discussions around causality continue to be as essential as they are difficult for public policy.

 

The determinist perspective is generally the first one selected to discuss the causality perspective (Bennett, Checkel, 2014; Mahoney, 2001). In this perspective, researchers propose to establish a link between the process itself and various phenomena or variables which precede it. In this regard, the “causal link” is, first and foremost, a temporal link between a succession of two or more phenomena - the first is referred to as “causes” and the second as “effects”. From an epistemological perspective, the main problem this approach encounters is the difficulty to determine a method capable of helping transform this temporal correlation between different phenomena observed empirically into a causal link, i.e., the difficulty associated with ensuring that each time the first phenomenon is observed, the second is observed soon after (Hume, 1748; Searle, 1985). This type of proof requires one to observe an event regularly and repeatedly, for instance an experience, and to reproduce it as many times as necessary. In this way, the proof of causality provides a first link to the temporal question and to the process and consists in getting rid of the contextuality and singularity of each phenomenon, even if repetition alone cannot fix causality (Popper, 1973). Social scientists face a major dilemma because they have no possibility to undertake experiences with infinite possibilities of reproduction such as is common in physical sciences. While they may use comparison to test these links through repetition, they must decontextualize each phenomenon to transform it into a “case”. They also come up against Popper’s classical criticism which underlines that the multiplication of cases cannot offer proof of causality in the absence of falsifiability rules. The Popperian framework thus emerges as a problem for social scientists.

 

Drawing on different perspectives, the probabilistic perspective of causality (Benett, Checkel) considers that, in cases where there is causality between different phenomena, the relationship is consistently affected by the context and the specificity of each phenomenon. In this regard, causality can always be identified by multiplying cases and the adoption of a probabilistic perspective. This suggests that the relationship between two events is simply a probability, i.e., it may be observed frequently but not systematically (Muleik, 1986). In this case, causality is defined by the probability that an event occurs. Inspired by Bayes, identifying the probable cause requires researchers to compare a large number of situations which must be transformed into “comparable” cases.

 

In this paper, we will propose a third approach to these epistemologically challenging questions by shifting from a causal to a consequentialist perspective that draws on a pragmatist approach.

 

First, we will develop the three main ontological and epistemological characteristics of this perspective. Inspired by the pragmatist philosophy, this perspective considers, above all, that if each social event generates consequences (meaning that there is a link between different phenomena), these consequences are always unexpected, uncertain and varied. These various potential consequences are linked to the singularity of the context within which each phenomenon takes place and the fact that each human being has a margin of freedom in the social relationships he/she is engaged in. The second characteristic, developed by John Dewey, suggests that irrespective of the situation, a phenomenon must be observed via a knowledge device, but this device deforms the phenomenon observed. Quantum mechanics critically assesses the fundamental indetermination of reality (Bohm, 1952; Popper, 1974) and considers that all observation is influenced by the observer and the methods used to observe a phenomenon. The third characteristic is largely inspired by Weber’s comprehensive theory and by pragmatist sociology (Boltanski, 2014; Latour, 2015). These perspectives consider that, contrary to what Durkheim suggests, a social phenomenon is not a fact which can be determined by a “cause”. Rather, it is a complex phenomenon defined and affected by a multitude of causes including actors’ intentionality, the meaning of their discourse, their interactions, and their constraints which the researcher never succeeds in determining as a “cause” but may only grasp some of the “conditions” which have contributed to the existence and the shaping of a given phenomenon. Drawing on Foucault and his genealogy, or Latour and his associations, this approach focuses specifically on how different phenomena and meanings “chain together” in a specific sequence.

 

In the second section of our paper, we will analyze the relationship between the consequentialist perspective and empirical analysis. In this inductive perspective, we will not focus on empirical observation as a “case study” in which a theory must be tested and proved. Drawing on Merton (1965) who argues that serendipity and the discovery of the unexpected are more important that the “case” to analyze, this perspective considers, above all, that each empirical phenomenon is a phenomenon that offers the possibility to discover considerable knowledge about the complex chaining and the relationship between actors, discourses and contexts. In this perspective, the main empirical research question always begins with an enigma (Boltanski, 2016) which supposes a distinction between a normative understanding of the world and a phenomenon challenging this normativity. This micro-study involves reconstituting the concrete interactions between actors via the trajectories of different phenomena to better understand the processes involved. In this non-Popperian perspective, focusing on specific phenomena through a qualitative perspective constitute a relevant scientific choice (Becker, 2014; Passeron, 1991).

 

In the last section, we will explain how the consequentialist approach allows us to view the policy process differently. We will focus specifically on the “condition” of success or failure of the “policy coalition”. By proposing a concrete observation of the development of proposals and all the discussions involved, the consequentialist approach proposes to comprehensively assess the genealogy of a policy proposal to understand how policy entrepreneurs, tools, social definitions, publics, predictive fiction and unequal power relations are chained together, and how these chainings evolve during the policy process.

ACCEPTED
Format
05 - Contending with Policy Legacy: Comparative Case Study of Local Health Insurance in Indonesia and the Philippines
Kidjie Saguin (University of Melbourne)
CO-AUTHORS
—
ABSTRACT

In 2014, almost 460 local health insurance schemes, collectively known as Jamkesda, exist in Indonesia. The widespread adoption of locally-delivered and financed health insurance programs that are independent of national programs is puzzling considering the large amount of resources and capacity to sustain such programs. Mainstream decentralization literature argues that political devolution in the early 2000s widened the space for local government participation in social protection while political science studies argue for the centrality of clientelism in local politics as the main variables for this unique policy development. This paper uses comparative historical analysis through process tracing of local and national institutions in Indonesia to examine the factors that account for explosion of Jamkesda. In order to tease out the most critical factor that explains local insurance prevalence, the paper compares the Indonesian experience with the Philippines following a most-similar comparative case study design. While institutionally and developmentally similar, Indonesia and the Philippines diverge in the trajectory of the development local social health insurance where the Philippines only has two documented cases of locally-financed social health insurance despite decentralisation. Following a historical narrative of the two cases, the paper argues that the policy legacy of a nationally-run social health insurance can constrain the ability of local governments to use such type of social policy to generate clientelistic returns. This is evident in the Philippines’ 36-year experience of implementing Medicare while such policy legacy is relatively weak in Indonesia.

ACCEPTED
Format
06 - MODELING THE BLACK BOXES OF CAUSATION
Alessia Damonte (Dept. Social and Political Sciences - Università degli Studi di Milano)
CO-AUTHORS
—
ABSTRACT

The recognized advantage of Qualitative Comparative Analysis rests on its capacity to grasp complex causation as conjunctions of necessary and sufficient conditions that hold at both levels of the population and of the cases under analysis. This makes QCA especially suited to address ex-post evaluation questions about the reasons for local differences in some effect that other techniques may dismiss as stochastic or idiosyncratic policy responses. While its solutions cannot be generalized beyond the space-time boundaries of the actual study, they nevertheless allow for learning about the reasons for policy success and failure. As far as these reasons are manipulable, QCA promises to yield usable knowledge for improving policy effectiveness.

 

To keep its promise, however, a QCA shall find compounds that are theoretically interpretable, logically valid, and empirically sound at once. While the algorithmic conditions of logical validity and the practices of calibration have animated rich methodological conversations, the issue of theoretical interpretability has remained comparatively neglected. With the notable exception of Amenta and Poulsen’s seminal article, little systematic guidance exists on the consequential issue of how to provide an explanatory QCA with a solid starting point. The general tenet that algorithms cannot protect findings from the GIGO syndrome is doubly true for configurational analyses, where solutions directly follow from the reduction of the starting hypothesis.

 

Amenta and Poulsen suggest that only a configurational starting hypothesis warrants configurationally interpretable results. The contribution refines and develops this suggestion in the light of the latent consensus among QCA scholars that the configurational nature of a hypothesis commits the researcher to a realist ontology of inus causation and to the assumption that mechanisms generate the effect. It reviews the concept of mechanisms as envisaged in the tradition of analytical sociology, and finds they respond to two intertwined research questions – one about the shape of the process; the other about the structural conditions that make that process “unavoidable”. It then resorts to the methodological tradition to clarify the analytical trade-off the two questions raise and the different knowledge they compel. Exploring or retrieving the shape of the process opens the black box of causation, but responds to a how-question that corroborates a causal claim about the effect of a cause. Identifying the conjunctions of the “right” conditions under which the process unfolds “unimpeded”, instead, answers a why-question about the causes of an effect, although it does not open the black box of causation – rather, it specifies its many versions. Finding the composition of those local black boxes that ensure a policy effect is the distinctive heuristic task that a QCA can perform. A starting hypothesis is configurational as it includes those special factors that, jointly given, fulfill the key functions of a theoretically effective black box relative to a given mechanism.

 

The contribution then details the general shape of a configurational hypothesis suitable for explanatory QCA and provides practical suggestions on how to single out its components from variable-oriented and case-oriented strands of the empirical literature. It concludes with considerations about the relationships that explanatory QCA can fruitfully entertain with intensive and extensive research strategies.

ACCEPTED
Format
07 - Scope conditions for causal claims in comparative policy analysis
Eva Thomann (University of Exeter)
CO-AUTHORS
—
ABSTRACT

This presentation elaborates on the notion of scope in comparative policy research. Both quantitative and qualitative comparative empirical public policy research typically relies on a specific set of cases or observations for analysis. Researchers then typically engage in at least some modest form of abstraction or generalization of the findings. At the same time, most causal explanations and theories of public policy are bound to specific contexts rather than universal. It follows that there can only be cumulative theoretical progress in policy analysis if we specify the extent to which different empirical findings can “travel”. This requires researchers to specify scope conditions that set empirical and theoretical limits on the extent to which an inference can be generalized. However, arguably the notion of scope is generally underdeveloped in comparative policy analysis. Quantitative approaches view scope as an empirical question of statistical generalizability. This focus neglects that even statistical populations are usually context-specific—for example, by drawing on subjects from specific countries, regions, or cultures or by focusing on specific types of cases of a more general phenomenon.  Moreover, generalizations can not only be drawn “empirically” toward populations of cases, but also in more “analytical” terms toward more general theories and concepts. Conversely, case-oriented and set-theoretic approaches to comparative public policy have developed a more diverse understanding of generalization. Nevertheless, they rarely actually specify the scope of their findings, beyond acknowledging limits on statistical generalizability. The result is an accumulation of fragmented findings; we do not know to what extent the results of different studies speak to each other.

 

To address this problem, I present a general framework and a step-wise recipe for quantitative and qualitative researchers to specify scope conditions for their findings. I introduce the closely interrelated notions of empirical, conceptual, and causal scope. The empirical scope refers to the empirical—spatial, temporal, and social—context within which results are situated. The conceptual scope refers to how cases represent the core concepts that are of analytical interest. Finally, the causal scope refers to contexts within causal mechanisms or relations can be theoretically plausible. Identifying empirical, conceptual, and causal scope conditions encourages researchers to reflect on a) the definition and structure of underlying concepts, b) how cases were defined and selected, c) what the relevant context and its role is, and d) what plausible counterfactuals are. Reflecting on concepts and their measurement helps us to specify the boundaries of homogeneity. Thinking about casing helps understand what our cases are “a case of”. Moreover, theory-guided case selection principles help us distinguish typical from deviant, likely from unlikely cases and identify (ir)relevant negative cases. Furthermore, it is helpful to specify the relevant context and whether it acts as a cause, a barrier, or changes meaning. These insights enable researchers to separate “good” from implausible counterfactuals and identify the extent to which a given causal finding can inform comparative policy analysis empirically, conceptually, and theoretically.

ACCEPTED
Format
08 - Modes of policy learning as causal mechanisms: coming up with a “policy learning measuring instrument” for qualitative research
Jonathan Kamkhaji (European University Institute)
CO-AUTHORS
—
ABSTRACT

Drawing on the recent systematisation of the field of Dunlop and Radaelli (2013), this contribution seeks to achieve a number of conceptual and operational goals related to the use of modes of policy learning as causal mechanisms in the context of qualitative, case-study research, providing hence a practical application of the arguments advanced by Felleti and Lynch (2009) and Heikkila and Gerlak (2013).
At the outset, in line with the seminal work of Hugh Heclo (1974), the paper puts forth a provocative conceptualisation of learning as one of the ontologies of policy making – complementary to, but distinct from, a more classical “powering” ontology. Under the ontological account of learning, modes of policy learning work as epistemological lenses that are to be applied empirically to the policy process in general, and to policy interactions in particular. In fact, the paper argues that the empirical locus whereby policy learning occurs and needs to be typified is the policy-specific set of interactions taking place among actors.
But how to apply these lenses in practice? How to distinguish between one mode of learning and another in the context of a given interactive policy process and how to assess explanatory leverage?
To answer these questions the paper extends the dimensions of variation of policy learning detected by Dunlop and Radaelli (actor certification and problem tractability) by adding the time-frame that characterises policy interactions and the institutional venues where they take place. Equipped with these four dimensions of variation, the paper proceeds to develop a “policy learning measuring instrument”. In more detail, through the review of a broad corpus of literature, the paper identifies mode-specific empirical scope conditions for each dimension of variation. The final step to come up with the measuring instrument involves systematising a series of observable implications related to each mode of learning. Importantly, these mode-specific expected outcomes are not skewed toward a value-laden view of learning as an invariably positive mechanism for the sake of the policy process, but also include instances of dysfunctional learning.
The resulting measuring instrument allows policy learning scholars to tackle case study research with a systematic analytical device that suits extremely well process-tracing informed methodologies based on causal mechanisms. In fact, by pinning down the observed scope conditions for each of the dimensions of variation of policy learning, the researcher is able to precisely characterise a case (i.e. a set of policy-specific interactions) as belonging to one or another mode of learning and then to test whether the observed outcomes fit with the implications theorised for that mode. This operationalisation arguably allows for claiming solid explanatory leverage for modes of policy learning as causal mechanisms.

ACCEPTED
Format
09 - Why not privatize schools? A Bayesian Process Tracing analysis of divergent school choice reform paths in France and England
Charlotte Haberstroh (London School of Economics and Political Science)
CO-AUTHORS
—
ABSTRACT

This paper contributes to our understanding of processes driving the marketisation of public schooling in Western Europe. It applies Bayesian process tracing (Fairfield and Charman 2017) to update our confidence in the existing accounts that have stressed the incentives for governments from both the Left and the Right to introduce market-elements in education (e.g. Gingrich 2011, Hicks 2015, Hicks 2014, Zehavi 2012). Cases that have not introduced such market elements in the 1980s and 1990s (e.g. France, Denmark) are implicitly viewed as “negative cases” where governments missed out on such incentives because reform proposals were stifled by opposing producer groups’ power resources (e.g. Dobbins 2014). 

The paper puts this proposition on its head and asks about the incentives for governments not to marketise. Introducing market elements into public schooling comes with considerable technical costs as it reduces the public sector’s ability to plan and fund school places efficiently. Why did governments choose to marketise in spite of the costs? How did the evaluation of these costs enter the policy-making process of marketisation reform? The paper also innovates methodologically, as it applies Bayesian process tracing in a small-N comparative research design to explain variation in outcomes. Existing approaches to Bayesian process tracing have the aim to explain singular outcomes within cases, and compare cases only to show similar processes are at work across different contexts (e.g. Fairfield and Garay 2017). The paper thus discusses the compatibility of Bayesian process tracing with small-N case selection techniques with their set-theoretic or frequentist probabilistic fundaments. 

Applying Fairfield and Charman’s guidance (2017), the paper specifies and tests rival hypotheses for two cases. France: the decision of the Chirac government of 1986-1988 to break its campaign promise of introducing market elements into public schooling; England: the road to Margaret Thatcher’s 1988 Education Reform Act which expanded parental choice of schools and diversified providers of publicly funded education. Both cases take diverse values on both the dependent and independent variables of the conventional hypothesis for marketisation in education (diverse case selection technique, cf Seawright and Gerring). The evidence from elite interviews, archival and secondary sources supports that costs to marketisation played an important part in both policy processes. Throughout the 1980s, Thatcher’s government delayed reform proposals due to worries about how the market model would work in practice. The project was picked up again only following intense local grievances on public services that became salient at the national level. In contrast, in the French case, similar local grievances did not trigger system-wide reform. The government gradually introduced school choice in several areas as a policy “experiment”, without abolishing planning tool of the “carte scolaire” (catchment areas). This policy option was not available in the decentralized English school system.

The Bayesian approach provides tools to re-examine well-established hypotheses in the light of new theoretical and empirical insights. It is an asset to Comparative Policy Analysis, as it helps avoiding the tendency to assign national cases to “types”, which risk becoming stereotypes (e.g. difficulty to reform French education).

ACCEPTED
Format
10 - The Role and Impact of Policy Learning Approaches in Comparative Policy Analysis
Claire Dunlop (University of Exeter)
CO-AUTHORS
—
ABSTRACT

Policy learning has been an enduring lens on the policy process for more than four
decades. Despite being a classic theme offering various analytical strategies, it is not clear
what insights for comparative research have been generated by the vast learning
literature. This paper begins with a review of the empirical policy learning field examining
its comparative content. The results are disappointing. Despite policy learning having been
effectively launched by Heclo’s (1974) groundbreaking comparative study of social policy
in Britain and Sweden and further reinforced by Hall’s (1993) within case analysis of
economic policy change in Britain 1970-1989, no comparative research agenda has
emerged. This is explained, in part, by the absence of a unifying analytical framework.
Rather, piecemeal conceptualisations have flourished at the expense of a cumulative and
comparative research agenda. Recent theoretical developments offer a way forward
however. Two sets of authors in particular have pushed the agenda – Heikkila & Gerlak on
collective learning (2013) and Dunlop & Radaelli’s varieties of learning (2013). Here, we
focus on the model of varieties of learning. We start in theoretical mode, outlining the
framework, its ontological foundations, causal mechanisms and epistemological
implications. We then draw on a high profile health and safety case within the UK to
empirically appraise the learning framework’s comparative credentials in relation to four
conceptual challenges: defining public policy; dealing with political context;
understanding time and policy change; and, measuring policy outcomes. We conclude
sketching a comparative research agenda for the future.

ACCEPTED
Format
11 - Getting Underneath the Hood : The Implications of Mechanism Approaches for Policy-Process Theorizing
Adam Wellstead (Michigan Technological University)
CO-AUTHORS
Evert Lindquist (School of Public Administration)
ABSTRACT

Weible (2018) asks how policy process scholars can improve the quality of theoretical and
empirical work. He suggests that more attention should be directed to empirical studies
and testing, and less to more theory and propositions. We think that delineating policy
mechanisms that link theoretical assumptions and empirical investigation offers an exciting
opportunity to infuse policy process theorizing just as it has become a bit staid and reified,
and yet it does not purport to challenge what each of the traditions seeks to accomplish.
Focusing on mechanisms may assist those working with different policy process traditions:
a more granular approach can encourage further parsing out and deepening theories
associated with broad frameworks with the goal of identifying more specific empirically
grounded studies. By working in more detail with policy mechanism thinking and process
testing, policy scholars can better work across the frameworks to develop theoretical and
empirical insight into how their components link together, potentially enabling progress in
different functional domains to buttress development in other domains. Indeed, the policy
mechanism approach.

 

Policy process frameworks contain distinctly different research questions, although they
often share many of the same variables and causal links, which most contributors
acknowledge in varying degrees. The precise explanatory focus means that researchers
will identify different causes. However, when all the frameworks compared, the extent of
their overlap is considerable. It is difficult to keep track of all the frameworks, let alone fully
appreciate where one starts and the others leave off, particularly since many of the
frameworks import or point to similar components from other frameworks. This state of
affairs can lead to confusion or the perception that the frameworks agree in the main about
many things, but disagree or diverge in order to address particular dynamics and issues in
policy systems. But bringing a mechanistic
perspective to bear on this challenge raises the intriguing possibility that delineating causal
chains and animating conditions for one policy framework might lead to progress in
another.
We argue considerable progress can be made in the policy process theory field by thinking
the causal “wiring” or vascular network fanning out vertically, horizontally, and diagonally in
the across these policy policy systems. A mechanisms approaches may well sharpen our
appreciation of what each framework does and cannot do.

 

These frameworks have considerable overlap in terms of identifying the variables,
components,and functions at play in policy-making systems. At one level this is reassuring, and at
another level, concerning, because it seems increasingly difficult to distinguish where one
framework ends and another begins. Our paper suggests that, from a policy mechanism
perspective, the frameworks as articulated have different levels of maturity and promise.
The challenge will be to carefully appraise whether they have become theoretically better developed, and whether and how empirical studies could test and deepen theoretical
assumptions and propositions.

 

 

 

ACCEPTED
Format
12 - Understanding the effectiveness of policy interventions. On the methodological implications of different conceptualizations
Valerie Pattyn (Institute of Public Administration - Leiden University)
CO-AUTHORS
—
ABSTRACT

Investigating policy effectiveness is at the core business of many policy evaluations, and at
the basis of much research in comparative policy analysis. How effectiveness is exactly
defined reflects a certain theoretical approach to the nature of policy interventions. This
approach, in turn, will determine which methodology is best used to measure policy
effectiveness. The question is, however, whether policy practitioners are sufficiently
aware of the methodological implications of a certain conceptualization of policy
effectiveness.
In the presentation, we deconstruct the concept of policy effectiveness and link several
possible conceptualizations with particular methodological approaches. On this basis, we
critically reflect on prevailing notions of policy effectiveness as stipulated in evaluation
guidelines of governments worldwide and international organisations. While scholars now
tend to agree that several approaches to causality can provide good effectiveness
evidence, this does not seem to be reflected in many such guidelines, or in the particular
formulation of evaluation questions.
We conclude the presentation by listing several key theoretical choices to consider when
engaging in policy effectiveness research. Applying more fine-grained definitions of policy
effectiveness should help both policy makers and evaluators to decide on the most
appropriate evaluation method in evaluating policy interventions, and can contribute to
more robust comparative policy analysis.

ACCEPTED
Format
13 - The Familial Foundations of the Welfare State: Government as a Nurturing Institution in Cross-National and Historical Perspectives
Alla Baranovsky (Harvard University)
CO-AUTHORS
Eileen McDonagh (Northeastern University)
ABSTRACT

Our research focuses on what explains the wide variation in the degree to which
governments endorse social provision in contemporary Western European democracies. We use a
probabilistic causal model to build upon previous research, which established that the public and
political elites in past centuries viewed monarchies not only as authoritarian institutions, but also
as nurturing ones analogous to a nurturing family. We connect this scholarship to that of George
Lakoff’s, who argues that people with nurturing parental attitudes about the family support the
welfare state as an analogous form of nurturing government. Our research contributes to CPA
research methods by our use of OLS regression, as well as OLS with Fixed Effects, to establish
that retaining the legitimacy of a country’s nurturing, monarchical heritage when democratizing
promotes positive public attitudes about the welfare state as a contemporary form of nurturing
government. The results illuminate from cross-national, historical, institutional, and causal
perspectives the familial foundations of today’s welfare democracies.

ACCEPTED
Format
14 - Causal Claims and Causal Inferences in Comparative Policy Analysis
B. Guy Peters (University of Pittsburgh)
CO-AUTHORS
—
ABSTRACT

Demonstrating causation in the social sciences is inherently difficult, given the complexity of the
relationships about which we do our research. But difficult does not mean we can not try, or that
we should not pursue research designs and methodologies that can give us the strongest inklings
of causation. This introductory speech will address the difficulties for address challenges, and
opportunities, for making causal inferences in comparative public policy analysis.

CO-AUTHORS
—
ABSTRACT

The purpose of this paper is to compare two different causal explanations of the same major policy change, the passage of the Canadian Firearms Act in 1995. The Firearms Act was a major policy change because it introduced universal firearms registration to Canada for the first time. The two causal explanations compared in this paper utilize the same theory – the Advocacy Coalition Framework (ACF) – but are based in different ontological conceptions of causality. The current ACF theory of policy change is based on a deterministic-regularity conception of causality. Based on the empirical evidence, this approach seems to provide a good explanation of the Firearms Act’s adoption: the causal factors posited in the two ACF policy change hypotheses are empirically supported in this case. Nevertheless, the ACF theory of policy change is also compatible with a deterministic-mechanistic conception of causality and this approach provides several causal insights that a deterministic-regularity approach does not. First, it provides a clearer picture of how the two causal hypotheses interact and intersect to produce major policy change. Second, it investigates whether the two necessary conditions posited in the current causal hypotheses are actually necessary for major policy change. Third, it investigates (and potentially resolves) the over-determination problem that occurs in ACF explanations when multiple policy pathways - each of which are considered sufficient conditions within the first ACF policy change hypothesis - are present. By comparing a deterministic-regularity explanation with a deterministic-mechanistic explanation, this paper explores a potentially important avenue for further improving and refining ACF policy change theory.

CO-AUTHORS
—
ABSTRACT

In spite of a long pedigree, the use of randomised controlled trials (RCT) to answer general questions in social science has been a recent phenomenon. In the past, the attraction of the RCT was as a robust form of evaluation, testing government programmes or new procedures, where the attraction is the real-world context, the mimicking of an intervention, and then assessing the counterfactual of no intervention. The method continues as a major success story in disciplines interested in testing effectiveness, such as education or social policy, and where other forms of evaluation suffer from familiar problems of faulty comparisons or inconclusive time series/panel designs. In the late 1990s, more interest emerged about how to use RCTs to answer general questions in social science, such as in political science about vote choice or turnout. The big challenge for political science and public policy is that very importance aspects of decision-making involve a small number of actors at elite levels where the possibility of using trials is more challenging. Trials depend on large samples for power and require non-interference to respect the SUTVA assumption, as well as needing access and consent of policy-makers. It would seem that policy process studies in particular is shut off from this important development reinforcing its lag behind methodological developments in other parts of social science. To address this issue, the paper argues that there is potential for the use of RCTs in studies of the policy process from: a) elite responsiveness studies; b) behavioural games with decision-makers; and c) public management simulations with policy-makers, which are important first steps. The paper discusses these contributions and sums up what they mean for classic questions in public policy, such as rationality of decision-making, errors, bias, correction of errors/learning, and innovation. Ideas for future experiments are discussed.

CO-AUTHORS
—
ABSTRACT

In order to provide more meaningful and impactful suggestions to policymakers, policy scholars should perform causal analysis and seek subsequent causal mechanisms, explaining why the observations and patterns which they have discovered happen.  This deeper level of analysis will lead to more specific conclusions and recommendations that will help decision makers solve problems closer to their root cause, thus preventing problems from re-occurring, rather than focusing on fixing problems when they arise.  The lack of causal analysis in policy studies is evident. This paper focuses on analyzing the literature from one emerging energy policy field, energy justice.  A spin-off field of study from environmental justice theory, energy justice’s driving principle is that all individuals, across all areas, have the right to safe, affordable and sustainable energy, as energy has become a modern primary good required to achieve a basic quality of life.  To date, most studies in this field, whether qualitative or quantitative, are descriptive and fail to seek causality through their research design.  One essential item in search of causality is the in-depth case-oriented approach.  This paper describes how an in-depth case oriented approach can assist researchers in seeking causal inference and subsequent causal mechanisms, a key conclusion missing in the energy justice literature.  Next, this paper summarizes the current state of methods used in energy justice literature, briefly describing specific studies and their methodological approaches, then further illustrating which components of their research design should change in order to utilize the comparative methods, thus allowing better causal analysis.  Lastly, this paper describes some existing studies using comparative methods that demonstrate the proper way shift their focus to causal analysis.  This research will provide insights and evidence to other scholars that shifting research to case-oriented approach can lead to greater causal analysis and more actionable recommendations for policy and decision makers to act upon.

CO-AUTHORS
—
ABSTRACT

Most policy process approaches propose to analyze the policy process as a phenomenon that can be explained causally. Unfortunately, as Mahoney suggests, there are many definitions of causal mechanism in the social sciences literature (Mahoney 2001) which make discussions around causality difficult. Moreover, these multiple definitions make it difficult to clearly identify the key objective of policy research. Given impetus by vibrant debate about process tracing methodologies (Benett, Checkel, 2014; Hall, 2013), discussions around causality continue to be as essential as they are difficult for public policy.

 

The determinist perspective is generally the first one selected to discuss the causality perspective (Bennett, Checkel, 2014; Mahoney, 2001). In this perspective, researchers propose to establish a link between the process itself and various phenomena or variables which precede it. In this regard, the “causal link” is, first and foremost, a temporal link between a succession of two or more phenomena - the first is referred to as “causes” and the second as “effects”. From an epistemological perspective, the main problem this approach encounters is the difficulty to determine a method capable of helping transform this temporal correlation between different phenomena observed empirically into a causal link, i.e., the difficulty associated with ensuring that each time the first phenomenon is observed, the second is observed soon after (Hume, 1748; Searle, 1985). This type of proof requires one to observe an event regularly and repeatedly, for instance an experience, and to reproduce it as many times as necessary. In this way, the proof of causality provides a first link to the temporal question and to the process and consists in getting rid of the contextuality and singularity of each phenomenon, even if repetition alone cannot fix causality (Popper, 1973). Social scientists face a major dilemma because they have no possibility to undertake experiences with infinite possibilities of reproduction such as is common in physical sciences. While they may use comparison to test these links through repetition, they must decontextualize each phenomenon to transform it into a “case”. They also come up against Popper’s classical criticism which underlines that the multiplication of cases cannot offer proof of causality in the absence of falsifiability rules. The Popperian framework thus emerges as a problem for social scientists.

 

Drawing on different perspectives, the probabilistic perspective of causality (Benett, Checkel) considers that, in cases where there is causality between different phenomena, the relationship is consistently affected by the context and the specificity of each phenomenon. In this regard, causality can always be identified by multiplying cases and the adoption of a probabilistic perspective. This suggests that the relationship between two events is simply a probability, i.e., it may be observed frequently but not systematically (Muleik, 1986). In this case, causality is defined by the probability that an event occurs. Inspired by Bayes, identifying the probable cause requires researchers to compare a large number of situations which must be transformed into “comparable” cases.

 

In this paper, we will propose a third approach to these epistemologically challenging questions by shifting from a causal to a consequentialist perspective that draws on a pragmatist approach.

 

First, we will develop the three main ontological and epistemological characteristics of this perspective. Inspired by the pragmatist philosophy, this perspective considers, above all, that if each social event generates consequences (meaning that there is a link between different phenomena), these consequences are always unexpected, uncertain and varied. These various potential consequences are linked to the singularity of the context within which each phenomenon takes place and the fact that each human being has a margin of freedom in the social relationships he/she is engaged in. The second characteristic, developed by John Dewey, suggests that irrespective of the situation, a phenomenon must be observed via a knowledge device, but this device deforms the phenomenon observed. Quantum mechanics critically assesses the fundamental indetermination of reality (Bohm, 1952; Popper, 1974) and considers that all observation is influenced by the observer and the methods used to observe a phenomenon. The third characteristic is largely inspired by Weber’s comprehensive theory and by pragmatist sociology (Boltanski, 2014; Latour, 2015). These perspectives consider that, contrary to what Durkheim suggests, a social phenomenon is not a fact which can be determined by a “cause”. Rather, it is a complex phenomenon defined and affected by a multitude of causes including actors’ intentionality, the meaning of their discourse, their interactions, and their constraints which the researcher never succeeds in determining as a “cause” but may only grasp some of the “conditions” which have contributed to the existence and the shaping of a given phenomenon. Drawing on Foucault and his genealogy, or Latour and his associations, this approach focuses specifically on how different phenomena and meanings “chain together” in a specific sequence.

 

In the second section of our paper, we will analyze the relationship between the consequentialist perspective and empirical analysis. In this inductive perspective, we will not focus on empirical observation as a “case study” in which a theory must be tested and proved. Drawing on Merton (1965) who argues that serendipity and the discovery of the unexpected are more important that the “case” to analyze, this perspective considers, above all, that each empirical phenomenon is a phenomenon that offers the possibility to discover considerable knowledge about the complex chaining and the relationship between actors, discourses and contexts. In this perspective, the main empirical research question always begins with an enigma (Boltanski, 2016) which supposes a distinction between a normative understanding of the world and a phenomenon challenging this normativity. This micro-study involves reconstituting the concrete interactions between actors via the trajectories of different phenomena to better understand the processes involved. In this non-Popperian perspective, focusing on specific phenomena through a qualitative perspective constitute a relevant scientific choice (Becker, 2014; Passeron, 1991).

 

In the last section, we will explain how the consequentialist approach allows us to view the policy process differently. We will focus specifically on the “condition” of success or failure of the “policy coalition”. By proposing a concrete observation of the development of proposals and all the discussions involved, the consequentialist approach proposes to comprehensively assess the genealogy of a policy proposal to understand how policy entrepreneurs, tools, social definitions, publics, predictive fiction and unequal power relations are chained together, and how these chainings evolve during the policy process.

CO-AUTHORS
—
ABSTRACT

In 2014, almost 460 local health insurance schemes, collectively known as Jamkesda, exist in Indonesia. The widespread adoption of locally-delivered and financed health insurance programs that are independent of national programs is puzzling considering the large amount of resources and capacity to sustain such programs. Mainstream decentralization literature argues that political devolution in the early 2000s widened the space for local government participation in social protection while political science studies argue for the centrality of clientelism in local politics as the main variables for this unique policy development. This paper uses comparative historical analysis through process tracing of local and national institutions in Indonesia to examine the factors that account for explosion of Jamkesda. In order to tease out the most critical factor that explains local insurance prevalence, the paper compares the Indonesian experience with the Philippines following a most-similar comparative case study design. While institutionally and developmentally similar, Indonesia and the Philippines diverge in the trajectory of the development local social health insurance where the Philippines only has two documented cases of locally-financed social health insurance despite decentralisation. Following a historical narrative of the two cases, the paper argues that the policy legacy of a nationally-run social health insurance can constrain the ability of local governments to use such type of social policy to generate clientelistic returns. This is evident in the Philippines’ 36-year experience of implementing Medicare while such policy legacy is relatively weak in Indonesia.

CO-AUTHORS
—
ABSTRACT

The recognized advantage of Qualitative Comparative Analysis rests on its capacity to grasp complex causation as conjunctions of necessary and sufficient conditions that hold at both levels of the population and of the cases under analysis. This makes QCA especially suited to address ex-post evaluation questions about the reasons for local differences in some effect that other techniques may dismiss as stochastic or idiosyncratic policy responses. While its solutions cannot be generalized beyond the space-time boundaries of the actual study, they nevertheless allow for learning about the reasons for policy success and failure. As far as these reasons are manipulable, QCA promises to yield usable knowledge for improving policy effectiveness.

 

To keep its promise, however, a QCA shall find compounds that are theoretically interpretable, logically valid, and empirically sound at once. While the algorithmic conditions of logical validity and the practices of calibration have animated rich methodological conversations, the issue of theoretical interpretability has remained comparatively neglected. With the notable exception of Amenta and Poulsen’s seminal article, little systematic guidance exists on the consequential issue of how to provide an explanatory QCA with a solid starting point. The general tenet that algorithms cannot protect findings from the GIGO syndrome is doubly true for configurational analyses, where solutions directly follow from the reduction of the starting hypothesis.

 

Amenta and Poulsen suggest that only a configurational starting hypothesis warrants configurationally interpretable results. The contribution refines and develops this suggestion in the light of the latent consensus among QCA scholars that the configurational nature of a hypothesis commits the researcher to a realist ontology of inus causation and to the assumption that mechanisms generate the effect. It reviews the concept of mechanisms as envisaged in the tradition of analytical sociology, and finds they respond to two intertwined research questions – one about the shape of the process; the other about the structural conditions that make that process “unavoidable”. It then resorts to the methodological tradition to clarify the analytical trade-off the two questions raise and the different knowledge they compel. Exploring or retrieving the shape of the process opens the black box of causation, but responds to a how-question that corroborates a causal claim about the effect of a cause. Identifying the conjunctions of the “right” conditions under which the process unfolds “unimpeded”, instead, answers a why-question about the causes of an effect, although it does not open the black box of causation – rather, it specifies its many versions. Finding the composition of those local black boxes that ensure a policy effect is the distinctive heuristic task that a QCA can perform. A starting hypothesis is configurational as it includes those special factors that, jointly given, fulfill the key functions of a theoretically effective black box relative to a given mechanism.

 

The contribution then details the general shape of a configurational hypothesis suitable for explanatory QCA and provides practical suggestions on how to single out its components from variable-oriented and case-oriented strands of the empirical literature. It concludes with considerations about the relationships that explanatory QCA can fruitfully entertain with intensive and extensive research strategies.

CO-AUTHORS
—
ABSTRACT

This presentation elaborates on the notion of scope in comparative policy research. Both quantitative and qualitative comparative empirical public policy research typically relies on a specific set of cases or observations for analysis. Researchers then typically engage in at least some modest form of abstraction or generalization of the findings. At the same time, most causal explanations and theories of public policy are bound to specific contexts rather than universal. It follows that there can only be cumulative theoretical progress in policy analysis if we specify the extent to which different empirical findings can “travel”. This requires researchers to specify scope conditions that set empirical and theoretical limits on the extent to which an inference can be generalized. However, arguably the notion of scope is generally underdeveloped in comparative policy analysis. Quantitative approaches view scope as an empirical question of statistical generalizability. This focus neglects that even statistical populations are usually context-specific—for example, by drawing on subjects from specific countries, regions, or cultures or by focusing on specific types of cases of a more general phenomenon.  Moreover, generalizations can not only be drawn “empirically” toward populations of cases, but also in more “analytical” terms toward more general theories and concepts. Conversely, case-oriented and set-theoretic approaches to comparative public policy have developed a more diverse understanding of generalization. Nevertheless, they rarely actually specify the scope of their findings, beyond acknowledging limits on statistical generalizability. The result is an accumulation of fragmented findings; we do not know to what extent the results of different studies speak to each other.

 

To address this problem, I present a general framework and a step-wise recipe for quantitative and qualitative researchers to specify scope conditions for their findings. I introduce the closely interrelated notions of empirical, conceptual, and causal scope. The empirical scope refers to the empirical—spatial, temporal, and social—context within which results are situated. The conceptual scope refers to how cases represent the core concepts that are of analytical interest. Finally, the causal scope refers to contexts within causal mechanisms or relations can be theoretically plausible. Identifying empirical, conceptual, and causal scope conditions encourages researchers to reflect on a) the definition and structure of underlying concepts, b) how cases were defined and selected, c) what the relevant context and its role is, and d) what plausible counterfactuals are. Reflecting on concepts and their measurement helps us to specify the boundaries of homogeneity. Thinking about casing helps understand what our cases are “a case of”. Moreover, theory-guided case selection principles help us distinguish typical from deviant, likely from unlikely cases and identify (ir)relevant negative cases. Furthermore, it is helpful to specify the relevant context and whether it acts as a cause, a barrier, or changes meaning. These insights enable researchers to separate “good” from implausible counterfactuals and identify the extent to which a given causal finding can inform comparative policy analysis empirically, conceptually, and theoretically.

CO-AUTHORS
—
ABSTRACT

Drawing on the recent systematisation of the field of Dunlop and Radaelli (2013), this contribution seeks to achieve a number of conceptual and operational goals related to the use of modes of policy learning as causal mechanisms in the context of qualitative, case-study research, providing hence a practical application of the arguments advanced by Felleti and Lynch (2009) and Heikkila and Gerlak (2013).
At the outset, in line with the seminal work of Hugh Heclo (1974), the paper puts forth a provocative conceptualisation of learning as one of the ontologies of policy making – complementary to, but distinct from, a more classical “powering” ontology. Under the ontological account of learning, modes of policy learning work as epistemological lenses that are to be applied empirically to the policy process in general, and to policy interactions in particular. In fact, the paper argues that the empirical locus whereby policy learning occurs and needs to be typified is the policy-specific set of interactions taking place among actors.
But how to apply these lenses in practice? How to distinguish between one mode of learning and another in the context of a given interactive policy process and how to assess explanatory leverage?
To answer these questions the paper extends the dimensions of variation of policy learning detected by Dunlop and Radaelli (actor certification and problem tractability) by adding the time-frame that characterises policy interactions and the institutional venues where they take place. Equipped with these four dimensions of variation, the paper proceeds to develop a “policy learning measuring instrument”. In more detail, through the review of a broad corpus of literature, the paper identifies mode-specific empirical scope conditions for each dimension of variation. The final step to come up with the measuring instrument involves systematising a series of observable implications related to each mode of learning. Importantly, these mode-specific expected outcomes are not skewed toward a value-laden view of learning as an invariably positive mechanism for the sake of the policy process, but also include instances of dysfunctional learning.
The resulting measuring instrument allows policy learning scholars to tackle case study research with a systematic analytical device that suits extremely well process-tracing informed methodologies based on causal mechanisms. In fact, by pinning down the observed scope conditions for each of the dimensions of variation of policy learning, the researcher is able to precisely characterise a case (i.e. a set of policy-specific interactions) as belonging to one or another mode of learning and then to test whether the observed outcomes fit with the implications theorised for that mode. This operationalisation arguably allows for claiming solid explanatory leverage for modes of policy learning as causal mechanisms.

CO-AUTHORS
—
ABSTRACT

This paper contributes to our understanding of processes driving the marketisation of public schooling in Western Europe. It applies Bayesian process tracing (Fairfield and Charman 2017) to update our confidence in the existing accounts that have stressed the incentives for governments from both the Left and the Right to introduce market-elements in education (e.g. Gingrich 2011, Hicks 2015, Hicks 2014, Zehavi 2012). Cases that have not introduced such market elements in the 1980s and 1990s (e.g. France, Denmark) are implicitly viewed as “negative cases” where governments missed out on such incentives because reform proposals were stifled by opposing producer groups’ power resources (e.g. Dobbins 2014). 

The paper puts this proposition on its head and asks about the incentives for governments not to marketise. Introducing market elements into public schooling comes with considerable technical costs as it reduces the public sector’s ability to plan and fund school places efficiently. Why did governments choose to marketise in spite of the costs? How did the evaluation of these costs enter the policy-making process of marketisation reform? The paper also innovates methodologically, as it applies Bayesian process tracing in a small-N comparative research design to explain variation in outcomes. Existing approaches to Bayesian process tracing have the aim to explain singular outcomes within cases, and compare cases only to show similar processes are at work across different contexts (e.g. Fairfield and Garay 2017). The paper thus discusses the compatibility of Bayesian process tracing with small-N case selection techniques with their set-theoretic or frequentist probabilistic fundaments. 

Applying Fairfield and Charman’s guidance (2017), the paper specifies and tests rival hypotheses for two cases. France: the decision of the Chirac government of 1986-1988 to break its campaign promise of introducing market elements into public schooling; England: the road to Margaret Thatcher’s 1988 Education Reform Act which expanded parental choice of schools and diversified providers of publicly funded education. Both cases take diverse values on both the dependent and independent variables of the conventional hypothesis for marketisation in education (diverse case selection technique, cf Seawright and Gerring). The evidence from elite interviews, archival and secondary sources supports that costs to marketisation played an important part in both policy processes. Throughout the 1980s, Thatcher’s government delayed reform proposals due to worries about how the market model would work in practice. The project was picked up again only following intense local grievances on public services that became salient at the national level. In contrast, in the French case, similar local grievances did not trigger system-wide reform. The government gradually introduced school choice in several areas as a policy “experiment”, without abolishing planning tool of the “carte scolaire” (catchment areas). This policy option was not available in the decentralized English school system.

The Bayesian approach provides tools to re-examine well-established hypotheses in the light of new theoretical and empirical insights. It is an asset to Comparative Policy Analysis, as it helps avoiding the tendency to assign national cases to “types”, which risk becoming stereotypes (e.g. difficulty to reform French education).

CO-AUTHORS
—
ABSTRACT

Policy learning has been an enduring lens on the policy process for more than four
decades. Despite being a classic theme offering various analytical strategies, it is not clear
what insights for comparative research have been generated by the vast learning
literature. This paper begins with a review of the empirical policy learning field examining
its comparative content. The results are disappointing. Despite policy learning having been
effectively launched by Heclo’s (1974) groundbreaking comparative study of social policy
in Britain and Sweden and further reinforced by Hall’s (1993) within case analysis of
economic policy change in Britain 1970-1989, no comparative research agenda has
emerged. This is explained, in part, by the absence of a unifying analytical framework.
Rather, piecemeal conceptualisations have flourished at the expense of a cumulative and
comparative research agenda. Recent theoretical developments offer a way forward
however. Two sets of authors in particular have pushed the agenda – Heikkila & Gerlak on
collective learning (2013) and Dunlop & Radaelli’s varieties of learning (2013). Here, we
focus on the model of varieties of learning. We start in theoretical mode, outlining the
framework, its ontological foundations, causal mechanisms and epistemological
implications. We then draw on a high profile health and safety case within the UK to
empirically appraise the learning framework’s comparative credentials in relation to four
conceptual challenges: defining public policy; dealing with political context;
understanding time and policy change; and, measuring policy outcomes. We conclude
sketching a comparative research agenda for the future.

CO-AUTHORS
Evert Lindquist (School of Public Administration)
ABSTRACT

Weible (2018) asks how policy process scholars can improve the quality of theoretical and
empirical work. He suggests that more attention should be directed to empirical studies
and testing, and less to more theory and propositions. We think that delineating policy
mechanisms that link theoretical assumptions and empirical investigation offers an exciting
opportunity to infuse policy process theorizing just as it has become a bit staid and reified,
and yet it does not purport to challenge what each of the traditions seeks to accomplish.
Focusing on mechanisms may assist those working with different policy process traditions:
a more granular approach can encourage further parsing out and deepening theories
associated with broad frameworks with the goal of identifying more specific empirically
grounded studies. By working in more detail with policy mechanism thinking and process
testing, policy scholars can better work across the frameworks to develop theoretical and
empirical insight into how their components link together, potentially enabling progress in
different functional domains to buttress development in other domains. Indeed, the policy
mechanism approach.

 

Policy process frameworks contain distinctly different research questions, although they
often share many of the same variables and causal links, which most contributors
acknowledge in varying degrees. The precise explanatory focus means that researchers
will identify different causes. However, when all the frameworks compared, the extent of
their overlap is considerable. It is difficult to keep track of all the frameworks, let alone fully
appreciate where one starts and the others leave off, particularly since many of the
frameworks import or point to similar components from other frameworks. This state of
affairs can lead to confusion or the perception that the frameworks agree in the main about
many things, but disagree or diverge in order to address particular dynamics and issues in
policy systems. But bringing a mechanistic
perspective to bear on this challenge raises the intriguing possibility that delineating causal
chains and animating conditions for one policy framework might lead to progress in
another.
We argue considerable progress can be made in the policy process theory field by thinking
the causal “wiring” or vascular network fanning out vertically, horizontally, and diagonally in
the across these policy policy systems. A mechanisms approaches may well sharpen our
appreciation of what each framework does and cannot do.

 

These frameworks have considerable overlap in terms of identifying the variables,
components,and functions at play in policy-making systems. At one level this is reassuring, and at
another level, concerning, because it seems increasingly difficult to distinguish where one
framework ends and another begins. Our paper suggests that, from a policy mechanism
perspective, the frameworks as articulated have different levels of maturity and promise.
The challenge will be to carefully appraise whether they have become theoretically better developed, and whether and how empirical studies could test and deepen theoretical
assumptions and propositions.

 

 

 

CO-AUTHORS
—
ABSTRACT

Investigating policy effectiveness is at the core business of many policy evaluations, and at
the basis of much research in comparative policy analysis. How effectiveness is exactly
defined reflects a certain theoretical approach to the nature of policy interventions. This
approach, in turn, will determine which methodology is best used to measure policy
effectiveness. The question is, however, whether policy practitioners are sufficiently
aware of the methodological implications of a certain conceptualization of policy
effectiveness.
In the presentation, we deconstruct the concept of policy effectiveness and link several
possible conceptualizations with particular methodological approaches. On this basis, we
critically reflect on prevailing notions of policy effectiveness as stipulated in evaluation
guidelines of governments worldwide and international organisations. While scholars now
tend to agree that several approaches to causality can provide good effectiveness
evidence, this does not seem to be reflected in many such guidelines, or in the particular
formulation of evaluation questions.
We conclude the presentation by listing several key theoretical choices to consider when
engaging in policy effectiveness research. Applying more fine-grained definitions of policy
effectiveness should help both policy makers and evaluators to decide on the most
appropriate evaluation method in evaluating policy interventions, and can contribute to
more robust comparative policy analysis.

CO-AUTHORS
Eileen McDonagh (Northeastern University)
ABSTRACT

Our research focuses on what explains the wide variation in the degree to which
governments endorse social provision in contemporary Western European democracies. We use a
probabilistic causal model to build upon previous research, which established that the public and
political elites in past centuries viewed monarchies not only as authoritarian institutions, but also
as nurturing ones analogous to a nurturing family. We connect this scholarship to that of George
Lakoff’s, who argues that people with nurturing parental attitudes about the family support the
welfare state as an analogous form of nurturing government. Our research contributes to CPA
research methods by our use of OLS regression, as well as OLS with Fixed Effects, to establish
that retaining the legitimacy of a country’s nurturing, monarchical heritage when democratizing
promotes positive public attitudes about the welfare state as a contemporary form of nurturing
government. The results illuminate from cross-national, historical, institutional, and causal
perspectives the familial foundations of today’s welfare democracies.

CO-AUTHORS
—
ABSTRACT

Demonstrating causation in the social sciences is inherently difficult, given the complexity of the
relationships about which we do our research. But difficult does not mean we can not try, or that
we should not pursue research designs and methodologies that can give us the strongest inklings
of causation. This introductory speech will address the difficulties for address challenges, and
opportunities, for making causal inferences in comparative public policy analysis.