T10P03 - How to Create Quantitatively Comparable Policy Measures

Topic : Methodologies

Chair : Jody Heymann - jody.heymann@ph.ucla.edu

Second Chair : DAVID Godfrey - dgodfrey@ph.ucla.edu

Third Chair : Arijit Nandi - arijit.nandi@mcgill.ca

General Objectives, Research Questions and Scientific Relevance

Call for papers

Session 1

Wednesday, June 28th 14:00 to 16:00 (Block B 3 - 5 )


Jody Heymann - jody.heymann@ph.ucla.edu - Fielding School of Public Health; WORLD Policy Analysis Center - University of California, Los Angeles - United States

DAVID Godfrey - dgodfrey@ph.ucla.edu - WORLD Policy Analysis Center, UCLA - United States


Estimating the effect of compulsory and tuition-free education policies on financial inclusion in Asian countries

Ni Luh Putu Satyaning Pradnya Paramita - sat.pradnyaparamita@gmail.com - Indonesia

Dikara Alkarisya - dikara.alkarisya@un.or.id - Indonesia

Download the paper

Education has a key role to play in moving towards environmentally sustainable and inclusive economic growth. Inclusive economic growth is not only about expanding national economies but also about ensuring that we reach the most vulnerable people of societies. The “equality of opportunity” and “participation in growth by all” with special focus on the working poor and the unemployed are the very basis of inclusive growth. In the meantime, a recent IMF and NBER Working Paper (Era Dabla-Norris, Yan Ji, Robert Townsend, and D. Filiz Unsal, 2015) shows that elimination of blockages to financial inclusion has significant and unambiguous direct impacts on GDP growth and productivity through smarter allocation of resources and more efficient financial contracting; resulting in stronger entrepreneurial activities and new business start-ups that increase aggregate output.


This paper examines how (i) compulsory and tuition-free education policies affect financial inclusion in Asian countries and (ii) the effects differ across different economic setting based on UN definition of low-income, middle-income and high-income country. We used country-level policies data collected by WORLD Policy Analysis Center, contains the information whether a country has constitutions on making certain levels of education compulsory and/or tuition-free. As a measurement of financial inclusion, we referred to the Global Findex database, the world’s most comprehensive database on financial inclusion, provides in-depth data on how individuals save, borrow, make payments, and manage risks.

However, the causal relationship of compulsory and tuition-free education and financial inclusion is not that simple. There are some unobservable ‘latent’ variables that affects the relationship, includes enrollment rate, literacy rate, household income, financial literacy. We proposed to use Structural Equation Modelling (SEM) to address this complexity and reveal how those variables interact each other. SEM provides a very general and convenient framework for statistical analysis that includes several traditional multivariate procedures, for example factor analysis, regression analysis, discriminant analysis, and canonical correlation, as special cases.


This paper shall contribute on comparison analysis of how education policies and under which condition can affect financial inclusion in low-income, middle-income, and higher-income Asian countries.

Measuring political symbolism for large-n comparative policy studies

Andrew Tanabe - andrew.tanabe@mcgill.ca - McGill University - Canada

Biesbroek Robbert - robbert.biesbroek@wur.nl - Wageningen University & Research - Netherlands

Symbolism plays a pivotal role in the politics and decision making around complex societal issues such as sustainable development. Politicians and policy makers use and create, in both instrumental and expressive sense, symbolic policies to achieve their (own) goals. Many examples of political symbolism are part of the political repertoire (“less government”, “tough on crime”, “the 99%”) and both positive or negative connotations are associated with symbolism. In the context of sustainable development symbolic actions include relabeling existing policies and instruments to fit the prevalent public discourse on sustainability – the mechanism of greenwashing. Symbolism of political rhetoric, symbolic politics, as well as recent symbolism of policy instruments have been widely discussed in the political science and public policy literatures, see for example the works of Edelman, Gusfield, Bludhorn and others. Symbolic policy is the inevitable part of the ‘dynamic interplay’ between symbolic and substantive policy making.


Demarcating symbolic from substantive elements of policies is important as symbolism can create significant measurement error when comparing policy outputs across contexts and over time. Many large-n comparative studies are insensitive to the dynamic interplay of symbolism and substance in policies and fail to adequately address this issue. As yet, no measurement frameworks exist that demarcate symbolic from substantive policy instruments for the purpose of tracking or evaluating policy output (over time and across contexts). 


The aim of this paper is to provide such a measurement framework. We review the main strands of the literature on political and policy symbolism. We propose a framework for measuring the symbolism of a specific policy instrument as a tool to measure how policy has changed and provide insight into the likely impact of a policy portfolio. To measure the symbolism of a policy portfolio, the policy objectives, instrument setting and calibration, are identified. Next, these are compared with a taxonomy of the expected use of governing resources to assess a level of dissonance between the two. When the level of dissonance between expected and empirically identified tools and calibrations reaches a critical level, the policy can be identified as having a high level of symbolism.


We illustrate the applicability of our framework to the example of climate change adaptation policy instruments related to one of the key SDG goals (SDG13) which calls for integrating climate change measures into national policies, strategies and planning. In addition to this SDG goal, the need for systematic approaches for tracking climate change adaptation progress following the Paris Agreement on Climate Change has been highlighted by Ford et al. 2015. The framework can provide more precision to comparative policy studies involved both in measuring changes in policy portfolios as well as comparing policies across space or time. The proposed framework for estimating levels of symbolism advances the current metrics for measuring policy output by allowing for a more nuanced estimation of the potential impact of policy instruments on the policy outcome.

Creating Quantitatively Comparable Policy Measures to Strengthen Equal Rights and Opportunities Worldwide: New Global Public Use Data Being Created on Equal Rights at Work

DAVID Godfrey - dgodfrey@ph.ucla.edu - WORLD Policy Analysis Center, UCLA - United States

Jody Heymann - jody.heymann@ph.ucla.edu - Fielding School of Public Health; WORLD Policy Analysis Center - University of California, Los Angeles - United States

Comparative data can help leaders understand the range of options other countries have taken to address some of the world’s most pressing challenges, and help civil society to urge their lawmakers to adopt similar effective approaches. Furthermore, periodically updating these datasets allows us to track individual countries’ progress towards fulfilling targets like the Sustainable Development Goals (SDGs). 

This paper examines our approach to building a database of quantitatively comparable policy measures relevant to SDGs 5 and 10, which aim to achieve gender equality and empower women and girls and reduce inequalities based on sex, age, disability, race, ethnicity, origin, religion or economic or other status, respectively. We chose to focus on workplace discrimination and sexual harassment laws because workplace equality is fundamental to ensuring everyone has the opportunity to earn a living income in decent working conditions, laying the foundation for health, well-being, and economic growth.


This paper details every step of building a comprehensive quantitative policy database, from conceptualization, to creating variables, to gathering and analyzing legislation in a systematic way that ensures consistency across countries.  We first created a conceptual framework of quantitative indicators based on international agreements and research evidence on this topic and with input from civil society.  Our multilingual team then collected and analyzed thousands of primary sources including laws and policies that make discrimination illegal and prohibit sexual harassment in the workplace from 193 UN countries.  All questions were answered independently by two different researchers who then compared their answers to minimize human errors.  The resulting dataset can then be transformed into accessible, user-friendly resources, including interactive maps, tables, and downloadable datasets. Periodic updates to this dataset over the coming decade will provide a detailed picture of the steps countries have taken to realize these development goals, and will serve to quickly identify leaders and laggards in reducing inequalities.


By sharing some of our lessons learned in gathering and evaluating quantitatively comparable policy data, we hope to encourage and support the work of other organizations in this field.

Export PDF