T07P04 - The Rise of Policy Labs

Topic : Policy Design, Policy Analysis, Expertise and Evaluation

Panel Chair : Emma Blomkamp - blomkamp.e@unimelb.edu.au

Panel Second Chair : Jenny Lewis - jmlewis@unimelb.edu.au

Panel Third Chair : MIchael Howlett - howlett@sfu.ca

Objectives and Scientific Relevance of the panel

Call for papers

Session 1 Policy Labs and Public Policy I: Crafting New Approaches to Old Problems?

Wednesday, June 28th 14:00 to 16:00 (Manasseh Meyer MM 3 - 2 (70))

Discussants

Jenny Lewis - jmlewis@unimelb.edu.au - University of Melbourne - Australia

Mapping and evaluating behaviour-change policy innovation agendas

Brian Head - brian.head@uq.edu.au - University of Queensland - Australia

This paper traces the rise of new “behavioural” approaches to policy instruments aimed at various forms of “behavioural change”. These approaches are mainly anchored in cognitive psychology and behavioural economics, and are increasingly being applied to public policy design and innovation. Many of the recent initiatives have been associated with ‘choice architecture’ (Nudge, or ‘liberal paternalism’) and the experimentalism and controlled trials (‘test, learn, adapt’) advocated by the Behavioural Insights Team.

This paper attempts four main tasks:

(i) map the main intellectual foundations of these approaches, noting some important earlier traditions that substantially preceded the recent initiatives;

(ii) identify some reasons for the increasing popularity of these approaches, linked primarily to the innovation agenda, the demands for fiscal austerity, and the attractions of voluntarism;

(iii) outline the methodologies used by experimental approaches together with the main types of policy problems and client-targets to which they have been directed;

(iv) assess some of the strengths and limitations of these approaches in relation to both simple problems and complex problems.  

The paper suggests that these approaches are neither new, nor universally applicable, nor insightful in isolation unless harnessed to contextual and institutional analyses and inter-disciplinary knowledge.

The Rise of the Behavioural Insights Team: A Research Agenda

Sarah Ball - s.ball@uq.edu.au - Institute for Social Science Research, University of Queensland - Australia

In 2010 the newly elected Conservative government in the United Kingdom (UK) set up a Behavioural Insights Team within the Cabinet Office to assist government agencies in applying behavioural economics theories to policy development. Since then, the interest in applying these ‘behavioural insights’ has grown rapidly outside the UK, with more than 50 countries boasting a central state-led behavioural policy initiative of one form or another (Whitehead, Jones, Howell, Lilley, & Pykett, 2014, p. 8).

 

Behavioural insights teams work as specialised policy labs, promoting the use of evidence-based policy making, particularly randomised controlled trials, and providing advice on applying behaviourally-informed research findings to policy design and implementation. These behavioural insights initiatives and teams are also commonly defined by their use of ‘nudges’, a policy intervention that “alters people's behavior in a predictable way without forbidding any options or significantly changing their economics incentives (Thaler & Sunstein, 2008, p. 6)”. However whether this means behavioural insights teams can be defined as instrument constituencies or presented as examples of policy transfer appears to be largely reliant on the individual teams themselves. Each team focuses on diverse policy areas, primarily defined by the interests of their governments and the influence of key stakeholders. Beyond this, little is known about the impact of these teams on the policy formulation, design and implementation process and their influence on their government partners.

 

This paper forms the basis for a research agenda into the relationship between behavioural insights teams and government agencies, and their impact on policy formulation, design and implementation more broadly. Summarising the results of a literature review, this paper explores what we currently know about several key behavioural insights teams working within Federal government departments, in the UK, the United States and Australia. First it will consider their role throughout the policy process, second, what forms of knowledge and expertise they draw on and promote and finally, whether there are particular government agencies or areas of policy where behavioural insights approaches appear to be particularly well-received and why. These findings will be used to extend some early hypotheses on the impact of behavioural insights teams on the policy process overall.

 

Thaler, R. H., & Sunstein, C. R. (2008). Nudge:  Improving Decisions about Health, Wealth, and Happiness. New Haven: Yale University Press.

Whitehead, M., Jones, R., Howell, R., Lilley, R., & Pykett, J. (2014). Nudging all over the World. Economic and Social Research Council.

 

Public innovation labs caught between a rock and a hard place: to randomize or to experiment?

Rainer Kattel - rainer.kattel@ttu.ee - Ragnar Nurkse School of Innovation and Governance - Estonia

Veiko Lember - veiko.lember@ttu.ee - Ragnar Nurkse School of Innovation and Governance - Estonia

Piret Tonurist - piret.tonurist@gmail.com - OECD - France

Innovation labs have become increasingly popular in the public sector and in international development organizations as well. While i-labs differ in many ways from typical public organizations (Tõnurist, Kattel, Lember 2017), perhaps most strikingly they use evaluation and design methods markedly differently from 'typical' public sector actors. As these novel organizations are almost inherently fashionable, accordingly they are often seen as avant-garde of public sector developments. At the same time, they are influenced by two diverging methodological trends: top-down randomized control trials/behavioral interventions and bottom-up grassroots, user-based approaches rooted in design thinking. Both require different skill-sets, strategical thinking and action on the part of innovation labs. Thus, it is pivotal to understand the working methodologies used by i-labs: what methods are used, why and to what kind of success. As working methods invariably start to influence the role of innovation labs - by the scope and scale of their activities -, this paper hypothesizes that this will determine the level of systemic change labs will bring to the public sector, not to mention their survival itself. Our study is based on extensive interviews with 11 i-labs globally both functioning and closed down. We show that while in academic studies randomized control trials are increasingly dominant (Duflo and others), in practice i-labs find it difficult to employ fully developed RCTs as they are expensive, conceptually limited, difficult to implement in complex situations and still quite unreliable. Thus, i-labs revert to small-scale experimentation via pilots, prototypes, and similar. This produces quick-wins, provides easy measurement and a proof of concept to their activities, but also subjects them to work in the margins, in the areas of  "low-hanging fruit." This brings legitimacy, but little radical change.

The craft of Policy Labs: an overview of methods applied for development and testing of policy solutions.

Olejniczak Karol - k.olejniczak@uw.edu.pl - EUROREG, University of Warsaw - Poland

Sylwia Borkowska-Waszak - sylwia.borkowska@strath.ac.uk - University of Strathclyde, European Policies Research Centre - United Kingdom

Anna Domaradzka - anna.domaradzka@gmail.com - Institute for Social Studies, University of Warsaw - Poland

Yaerin Park - yrpark7@gwu.edu - George Washington University - United States

PROBLEM STATEMENT

Policy Labs emerge all over the world with a mission to support policy practitioners with innovative solutions, grounded in empirical research. The trend started less than a decade ago (Price, 2015), and recent study have identified 78 policy labs in the European Union alone (Fuller & Lochard, 2016, p. 8-9). While the idea of policy labs  gains high popularity, the name tends to be applied to very different activities and approaches. This could be explained by diverse strands that provided methods for labs: design thinking, ethnography, behavioral insights, collaborative governance, and social entrepreneurship. However, the lack of coherent typology of the labs and their methods, results in a limited possibility to compare their outcomes for academic and practical purposes.

 

By definition "laboratory" is a place providing opportunity for experimentation, observation, or practice in a field of the study (Merriam-Webster Dictionary). While grey literature on labs report extensively on how they aim to generate innovation and produce creative solutions, the discussion about robust testing of those solutions is quite rare. In other words, the methodology used by labs to evaluate their ideas’ effectiveness is often unknown.

 

QUESTION AND METHODOLOGY

We are, thus, motivated to define, systemize, and challenge the idea of policy labs. For this, we will ask: "What methods and tools are policy labs using to develop and test policy solutions?".

 

We address this question by presenting results of the comparative study of 20 well-established policy labs from Western Europe, North America, South America, and Asia. The systematic, deductive content analysis (Hsieh & Shannon, 2005) covers grey literature, documents and web content on lab activities, records from the analyzed sample of policy labs, and structured interviews with their management.

 

ARTICLE STRUCTURE

Paper has three parts. We start with theoretical framework that captures the spectrum of potential lab activities, spanning from exploration and analysis of policy issues, through creative development of solutions, to their testing and application.

 

In the empirical part of the article we apply the framework to map methods and tools applied by 20 well-established policy labs across four continents. We specifically focus on methods used for testing the effectiveness of created solutions. This comparative analysis allows us to provide a tentative typology of policy labs and their methodological approaches across the world.

 

In conclusions, we (1) discuss implications of current methodological shortcomings of policy labs' practice, and (2) offer improvements for labs’ development by creating synergies with a well-established field of evaluation.

 

ARTICLE CONTRIBUTION

The presented research on new phenomena of policy labs will be useful for both academics and practitioners. The examination of methodological practices and orientations of policy labs will contribute to deepening our understanding of the role of policy labs in enhancing evidence-informed policies. For practitioners, the article will provide an overview of innovative methods for designing and testing public policy solutions.

 

 

Session 2 Policy Labs and Public Policy II: Experiences on the Street

Wednesday, June 28th 16:15 to 18:15 (Manasseh Meyer MM 3 - 2 (70))

Discussants

Olejniczak Karol - k.olejniczak@uw.edu.pl - EUROREG, University of Warsaw - Poland

Lab or No-Lab? Exploring institutional trajectories of public innovation teams

Emmanuel Coblence - emmanuel.coblence@isg.fr - Institut Supérieur de Gestion, Paris - France

Frédérique Pallez - frederique.pallez@mines-paristech.fr - Mines Paris-Tech - France

Elsa Vivant - elsa.vivant@univ-paris-est.fr - France

jean-marc weller - jean-marc.weller@enpc.fr - LISIS - France

As public policy issues are frequently considered as "wicked problems" that require the setting up of co-production devices with citizens and users, design-based approaches are increasingly mobilized for the design of public services and policies. One reason is that design is seen as particularly appropriate considering the complexity and uncertainty characterizing contemporary contexts. These few diagnostic elements seem to justify the development of collaborative innovation by design methods within the public sector, and the creation, all over the world, of “policy labs”, defined as structures with an organizational identity and relative autonomy, while being connected to public organizations. However, beyond this generic term, the various organizational forms and practices that are implemented in order to institutionalize these innovation processes, i.e. to make them sustainable and legitimate, have been largely neglected by academic research so far.

 

In this article, we explore the institutionalization process of Innovation-Oriented Teams (IOTs), located at different territorial levels and relying on the use of methods partly inspired by design. This analysis first aims at better understanding the organizational construction of “policy labs”: How do these teams emerge and perpetuate? Do all IOTs eventually become “labs”? Moreover, we investigate the link between the organizational construction processes and the ways of appropriation of design practices: How is the conception of design IOTs manipulate a means or an obstacle to their institutional anchoring and development?

 

Our analysis is based on the empirical study of three IOTs at three different administrative scales in France (National state, Region, Department). These IOTs were first chosen because they all use design-oriented approaches. Secondly, they have the advantage of having several-year existence, which enables us to study an institutionalization process deployed over time. Finally, it seemed interesting to locate our analysis at different territorial levels, the responsibilities of the three administrative host organizations being quite different, which should a priori impact the nature of their activities. We exploit different qualitative sources: interviews with IOT members, administrative officers involved in their birth and projects, with external stakeholders collaborating with these teams; documents (internal or public) relating to their birth, doctrine and the projects carried out; observation of key events that punctuated the trajectories of these IOTs.

 

Beyond similar characteristics of emergence, we show that the anchoring, development and legitimacy of IOTs vary, and largely depend on organizational tactics. Above all, their institutionalization relies on a process of “metabolization” of design methods that is contingent to each IOT and that allows it to enrole key actors such as elected representatives and managers. We conclude by highlighting the pragmatic and careful aspects of their strategy – and their plastic (or instrumental) approach of design – that enables them to move forward, survive and follow an institutionalization trajectory. Contrary to policy labs previously studied by academics, these IOTs have chosen to dissolve in the administrative organization to better disseminate the effects of the doctrines of innovation they carry. To that extend, they would thus work more to hybridize bureaucracies than to replace them.

Harnessing the Capacity and Potential of Cross-Sectoral Collaborations in Driving Policy Change: Griffith University’s Policy Innovation Hub

Tiernan Anne - a.tiernan@griffith.edu.au - Griffith University - Australia

Scholars studying the most central networks at the summit of the core executive have noted an increasing disjuncture between the supply of and effective demand for evidence-based policy analysis and advice. Growing interest in policy innovation, policy design and experimentation reflected in the rise of policy labs and comparable units, seems to be occurring in a universe parallel to the dynamic and volatile policy and decision-making contexts inhabited by political leaders and the advisory systems and ‘courts’ that support them.

 

Concern that the processes of decision-making have become impervious to advice and recommendations offered by career officials, researchers, practitioner and other experts; and recognition that short-term, often reactive policy-making is failing to deliver improved outcomes, have spurred development of new and innovative collaborations aimed at affecting policy change from the outside in.

 

This paper describes the model of experimentation, policy research and policy learning being pursued by Griffith University’s Policy Innovation Hub. It draws on examples of cross-disciplinary, cross-sectoral and place-based collaborations to consider the role that universities can play in supporting and fostering the capacity for innovation, reform and policy change.

Everybody else is doing it so why don’t we? Analysing the rise of the policy lab

Emma Blomkamp - blomkamp.e@unimelb.edu.au - University of Melbourne - Australia

McGann Michael - mmcgann@unimelb.edu.au - University of Melbourne - Australia

Jenny Lewis - jmlewis@unimelb.edu.au - University of Melbourne - Australia

Over the past decade, there has been what Williamson (2015) describes as a ‘“labification” of the policy field’ as governments have increasingly turned to public policy and government innovation labs to address ‘wicked’ policy challenges. The proliferation of such labs, especially in more recent years, has been driven by growing interest in evidence-based policy making and also in the application of design thinking to policy making, with advocates arguing that the rise of labs heralds the emergence of a new era of experimental government. However, critics contend that the turn towards policy labs is emblematic of the ‘reluctant state’ and its tendency to outsource government functions onto contracted-providers and other non-state actors. Much depends, however, on how policy labs are defined and distinguished from other policy actors, such as think tanks, consultancies, and policy teams within government departments.

 

What distinguishes and sets policy labs apart from other public sector institutions and policy actors remains poorly understood despite the enthusiasm shown by governments for their establishment. There has been little detailed analysis of these emerging institutions beyond a handful of overviews and guides published by practitioner groups and a small number of academic working papers. Policy labs remain especially under-examined within the policy sciences literature.

 

This paper sets out to provide an initial map of the field of public policy labs, examining their organisational structure, relationship to government, the policy areas they predominantly work on, and the methodological approaches and tools they deploy. As the paper highlights, one of the key challenges in understanding this emerging field is that there is little agreement or consensus concerning what policy labs are despite near unanimous agreement that they are spreading. A cacophony of different terms is used to refer to the same entities, clouding attempts to differentiate their organisational form. To address this definitional confusion, we focus on a core group of about 20 labs that feature repeatedly in discussions of the rise of policy and government innovation labs. We consider the extent to which these labs share distinct features in terms of their organisational structure and methodological commitments that distinguish them from other actors within policy advisory systems. We also consider the extent to which the formation of public policy labs is motivated by the application of “designerly” thinking to policy making and what this entails for evidence-based and more traditional approaches to policy making.

Export PDF