LIST OF WORKSHOPS

This workshop combines "The merits and demerits of the third paradigm in policy implementation research and beyond: assessment and prospects" and "Learning about implementation through realistic evaluation and design recovery case studies" constituting this new working group.

Learning about implementation through realistic evaluation and design recovery case studies:

Ever since public policies became systematic objects of scientific research, by the middle of the 20th Century, the investigation of what happened between design and outcomes, of how did a policy worked (or not) has occupied a central interest in both fields (Pressman and Wildavsky). Implementation studies, single or multiple, were the cradle of policy analysis and public administration research (Lasswell) and, as we shall argue, should continue to be their meeting point. Yet policy analysis and public administration have often failed to share or interact.

The label “implementation” is more often than not found within the field of policy analysis. But there is good reason to think this is because implementation is often considered a dependent variable in research projects, both because it is naturally uncomfortable to investigate, and because linking design with outcome is perhaps more dramatic and thus fashionable. Studies that provide in-depth understanding of such link is uncommon.

On the public administration side, specialization made implementation studies the exclusive playground of evaluation. A myriad of highly specific tools to frame and study policy, with a legitimate pretention to be “scientific”, are used to answer the classical normative, often prescriptive questions of the field. But evaluations, as it is been known for some time (McLaughlin), generally privilege the study of outcomes, relating them directly to design, and failing to grasp the realization process explaining the links.

In this proposed workshop we seek to combine two traditions that, we shall argue, could greatly benefit of dialogue, and eventually crossings over: public management (when possible, design-oriented) case studies and (when possible, process-oriented) realistic evaluations.

Case studies derive from a commonsense way to draw lessons from experience. But, as suggested by Ragin or Yin, they grew to become a sophisticated tool in political science, as they part from the celebration and diffusion of best practices toward in-depth, qualitative knowledge on the “how” of phenomena. Recently, Michael’s Barzelay design-oriented case study methods provide a way forward for this perspective.

Embedded in a venerable tradition of understanding public organizations as purposeful phenomena –artificial organisms, or “artifacts”– Barzelay draws a genealogy aligning Herbert Simon’s artificial organisms, Henry Mintzberg’s organizational configurations, Eugene Bardach’s smart practices and Mark Moore’s public-value oriented strategic management. The blend produced a compelling array of “questions to asks” and “answers to find” when understanding the role of design in why and how public management artifacts –organizations, campaigns, systems— became “working phenomena” or “functioning wholes”.

Realistic evaluation, as in Pawson and Tilley, is an applied form of scientific research aimed at identifying “the worth and the value” of policy, and to inform it to decision-makers and practitioners. It should seek -at least that is the way it is conceptually presented- the best method for each peculiar circumstance, framing it as a program and condensing results in terms of mechanisms-context-outcomes popularly known today as “theory of change”. Implementation (Love) or process evaluation studies aim at identifying and isolating the causal power of implementation issues as something distinct from design issues.

The point of choosing realistic evaluation as a “partner” to design-oriented case studies is their outstanding sharing (a) the importance of theory-based phenomena analysis (as opposed to solely evidence-based analysis) and (b) the backbone formula for explanation: mechanisms + context = outcome, and (c) the goal of producing “transferable” detailed answers to the questions about why programs worked, for whom, and under what circumstances.

The merits and demerits of the third paradigm in policy implementation research and beyond: assessment and prospects

This workshop provides an arena to discuss and assess the merits and demerits of the so-called third generation of policy implementation research. Policy implementation is a long-standing concern in public policy and administration research. Over the decades, the study of implementation dynamics, the main implementation arrangements and role of implementation in the policy-making process has continued to attract the attention of policy analysts and public administration scholars (see, among others: Sager and Gofen 2022; Casula and Steinebach 2023; Knill et al. 2023). Current scholarship is increasingly interested in understanding the link between implementation and other stages of the policy process (in particular the dynamic policy design/implementation link and its effects in terms of problem solution), as well as in advanced research designs including neighbouring disciplines such as economics and organization studies.

Goggin famously divided the evolution of policy implementation research in three generations. The first generation emerged during the 1970s and was characterized by an explorative, and a-theoretical character, a single case focus, and the use of qualitative methods and approaches; the second generation emerged during the 1980s and it had the merits of proposing the first analytical-theoretical frameworks for the study of policy implementation. Goggin was the promoter of the third-generation research paradigm. In the scale between relevance and rigor, Goggin clearly stood at the rigor-pole. His “third paradigm” claims the rigorous research designs considered insufficient in the first two generations. Goggin et al. (1990, 19) defined the main trait of this third-generation research paradigm in the following way:

- Key concepts and related variables must be clearly defined and operationalized;
- Hypotheses derived from theoretical constructs should guide empirical analysis;
- More use of statistical analysis using quantitative data to supplement qualitative analysis;
- More comparison across different units of analysis within the same policy sector; 
- More comparison across different policy sectors;
- More longitudinal research design (i.e., research time frame of at least 5 to 10 years).
 
After more than 30 years, implementation studies following the third-generation research paradigm clearly highlighted the difficulty (and perhaps impossibility) of arriving at a shared theory of implementation. According to Winter (2012), in fact, implementation studies will improve by: (1) Accepting theoretical diversity rather than looking for one common theoretical framework; (2) Developing and testing partial theories and hypotheses rather than trying to reach for utopia in constructing a general implementation theory.

The aim of this workshop is to provide an assessment of how implementation studies still follow the claims of the third paradigm, how they have changed, and in which direction they are heading today.

The aims are twofold: 1) to reflect on the potentials and pitfalls of the key elements of this paradigm (Goggin 1986; Goggin et al. 1990) when doing implementation research; and 2) to discuss how policy implementation can improve (if it needs to improve) towards overcoming the limits of the third generation research paradigm.

To achieve these aims, we will provide an arena to discuss and assess the merits and demerits of the so-called third generation of policy implementation research requesting analytic rigour in the study of implementation when it comes to theory and research design (Goggin et al. 1990, 19). Doing so, this workshop will be an opportunity to present and discuss contributions able, from a theoretical, analytical or empirical point of view, to show show the study of different implementation dynamics in multiple institutional and organizational setting is changing in modern societies within the third-generation research paradigm, and beyond.

Call for papers

Beside personal invitations to be sent to many colleagues, the workshop will welcome academic, pracademics, professional and students that commit to the rule of submitting fully written documents at an adequate date for collective reading and assessment, taking the following three formats:

1) Public policy / public organizations / public initiatives case studies, single or multiple, in-depth or comparative, where the implementation / management dimensions are a/the central feature of research.

2) Public policy / public organizations / public initiatives fully-developed / in-progress summative / process evaluations or evaluation designs that are centered in (or include an explicit interest in) what links impact to design.

3) Methodological and theoretical discussions on how to design, conduct and/or assess implementation case studies and evaluations, or studies discussing the state of these arts.

In addition to this call, a number of personal invitations have been sent to colleagues actively involved in the related fields of work, in many universities around the world.

Implementation studies is a general label for many different approaches, all sharing the interest of considering the critical relevance of the "how", beside the what and the why, in public administration and policy. This workshp intends to provide a space for exchange to different traditions in implementation studies, with special regard to case studies oriented to design recovery and realistic evaluation. We will be delighted to bring together different strains of work into dialogue for mutual learning.

Michael Barzelay has is Professor of Public Management at the London School of Economics and Political Science since 1995. Professor Barzelay’s academic career included an undergraduate degree at Stanford University, and graduate work at Yale University, where he first earned a Masters’ in Public and Private Management (MPPM) and then a PhD in Political Science, under the supervision of Charles E. Lindblom. He started teaching and researching at Harvard’s John F. Kennedy School of Government, then at University of California, Berkeley. Throughout the past 20 years, Professor Barzelay has been working on research to expand professional knowledge about public organizations and professional practice within them. Milestones of this route have been Preparing for the Future: Strategic Planning in the U.S. Air Force, which received the Louis Brownlow Book Award from the US National Academy of Public Administration; “Learning from Second-hand Experience: Methodology for Extrapolation-Oriented Case Research”; and Public Management as a Design-Oriented Professional Discipline, published by Edward Elgar in September 2019, a manifesto book presenting a full-fledged new design for public management as a professional discipline. Professor Barzelay has served in editorial roles for leading scholarly journals that cover public management: Governance: An International Journal of Policy, Administration, and Institutions, and the International Public Management Journal.  He is presently a co-editor of Perspectives on Public Management and Governance, published by the Public Management Research Association (PMRA) and serves on the editorial boards of Policy Design and Practice and Evaluation and Program Planning. Professor Barzelay was awarded a Doctor Honoris Causa in Political Science by the University of St. Gallen, Switzerland, for his outstanding research in the field of public management and governance, in particular his contributions to theory development and the internationalization of the field as well as his explicit interdisciplinary approach.   

Luciano Andrenacci is a Political Scientist specialized in Public Policy and Administration and Citizenship Studies (BD in the Universidad Nacional de Rosario, Argentina, 1991; MD and PhD in the Université Sorbonne-Nouvelle / Paris III, France, 1998). He is an Associate Professor at the Escuela de Política y Gobierno of the Universidad Nacional de San Martín, Buenos Aires, where he serves as Director of the Postgraduate Program in Public Policy Planning and Evaluation and Co-Director of the State, Public Policy and Public Administration Research Program. He also holds responsibilities at the Argentine chapter of the Facultad Latinoamericana de Ciencias Sociales (FLACSO), where he is the Director of the Postgraduate Program in Social Policy, and the Academic Coordination of the Postgraduate Program in Human Development. He has been an active academic and practitioner since 1998, conducting multiple research projects and authoring widely read publications. Along his professional career, he has been Project Coordinator and Consultant for different agencies of the United Nations System; the Interamerican Development Bank; and the Governments of Argentina (national and subnational levels), Guatemala, Honduras, Haiti and Peru. He is teaching and has taught in university postgraduate programmes in Argentina, Chile, Colombia, New Zealand, Peru, Spain, the United Kingdom, Uruguay and the USA.