T03P11 - Bringing Politics to the Analysis of Performance Measurement Programs: Case and Comparative Studies in Health Policy

Topic : Policy and Politics sponsored by Policy & Politics Journal

Panel Chair : Fabiana C Saddi - fasaddi@usp.br

Panel Second Chair : Nick Turnbull - nick.turnbull@manchester.ac.uk

Objectives and Scientific Relevance of the panel

Call for papers

Session 1 Political processes, actors and approaches in pay for performance

Friday, June 30th 10:30 to 12:30 (Block B 5 - 6)


Matthew Harris - m.harris@imperial.ac.uk - Imperial College London - United Kingdom

Nick Turnbull - nick.turnbull@manchester.ac.uk - University of Manchester - United Kingdom

From ideas to policymaking: exploring the global, continental and national processes leading to adoption of performance-based financing in Mali

Lara Gautier - lara.gautier@gmail.com - Institute of Public Health University of Montreal & University Paris-Diderot - Canada

Manuela De Allegri - manuela.de.allegri@urz.uni-heidelberg.de - Institute of Public Health, Heidelberg University - Germany

Ridde Valery - valery.ridde@umontreal.ca - University of Montreal - Canada

Background: Over the past decade, several donors have promoted performance-based financing (PBF) in Africa, in view of increasing the quantity and quality of health services provision. In Mali, after participating in a pilot program, the Ministry of Health engaged in the scale-up of PBF. PBF policymaking in Mali is investigated through an analysis of processes occurring at the global, continental, and national levels. These notably include internal and external idea framing and constitution in PBF networks.


Method: Using public policy analysis and sociology theories, we adopt a mixed methods approach to collect and analyse data. We investigate PBF adoption through the analysis of ideational and network processes via semi-structured interviews with key stakeholders at the international, continental, and national levels (n=50). We further analyse network processes by investigating the constitution in networks with bibliometric analysis of scientific articles (n=115) and major grey literature (n=55), as well as social network analyses of authors (n=257) of 354 online forum threads. The latter are performed using the exponential random graph model and the multiple regression quadratic assignment procedure. Critical discourse analysis is used to analyse interview data. Using a triangulation approach, we confront quantitative results to qualitative interview findings.


Results: We identify a very active PBF policy community covering both global and continental levels. On top of economics theories, the members of this community mobilise popular international relations discourses, such as the ones of “good governance” and “local autonomy” to frame PBF. PBF is also anchored in the language of South-South learning, whereby “flagship countries” (i.e., Rwanda and Burundi) would become success stories from which other African countries could learn. In Mali, national promoters frame PBF as the logical continuation of pre-existing decentralisation policies.

In terms of networks, the PBF international and African community gravitates around a narrower PBF policy elite made up of experts who know each other from earlier versions of PBF in the African Great Lakes region. Financially supported by a specific trust fund and international organisations, the PBF community produces and disseminates multiple forms of PBF knowledge. A few Malian actors (i.e. former government officials and former bilateral agency staff) showed strong commitment to PBF after participating to training and study tours organised by the PBF community. The latter also helps fuel the PBF policy through providing expertise and visibility to African PBF experts. A few of these African experts participate to the scale-up of the policy in Mali, thereby contributing to national policymaking.


Discussion: This is the first research focusing on the multifaceted and multi-level power of ideas and networks towards the diffusion of an innovation (PBF) in Africa, and in a particular setting (Mali). These findings imply that global health communities sharing a common policy framing, accessing material resources, and expanding their networks in successful countries may spark policy adoption faster in other countries.

Contrasting approaches to Primary Care Performance Governance in Denmark and New Zealand

Tim Tenbensel - t.tenbensel@auckland.ac.nz - University of Auckland - New Zealand

Primary care is a crucial component of health systems, and one which governments typically have a strong interest in being able to steer. Increasingly, this steering is taking place through ‘performance governance’ – the incorporation of performance measurement into institutionalised policy processes. Primary care presents many governance challenges because it is predominantly provided by independent practitioners in small organisations. In this article we compare two small, high-income countries with tax-funded health systems - Denmark and New Zealand which have adopted quite different instruments for performance governance. The Danish state governs primary care performance using ‘soft hierarchy’ based on accreditation processes but few strong sanctions, while New Zealand has relied more on a combination of explicit hierarchical targets and financial incentives. To explain this key difference, we use a conceptual framework that charts the connections between: (i) institutional contexts, including the organisational structure of primary care; (ii) governance processes (corporatist or pluralist); and (iii) governance problems such as access, equity, efficiency, quality, and population health. We argue that the specific nature of primary care institutions have a significant impact on regimes of performance governance. Our comparative framework has the potential to be applied across a wider range of countries.

The politics of implementing a performance measurement program (PMAQ) at the front line of primary health care in Goiania, Brazil: a qualitative political analysis

Fabiana C Saddi - fasaddi@usp.br - Federal University of Goias, Brazil - Brazil

Matthew Harris - m.harris@imperial.ac.uk - Imperial College London - United Kingdom

Fernanda Parreira - fernandarparreira@gmail.com - Universidade Federal de Goiás - Brazil

Raquel Pego - RABRA.PEGO@GMAIL.COM - Faculdade de Saúde Coletiva, UNB - Brazil

This paper´s objective is to understand how front line health workers in Goiania evaluate the Brazilian “National Program for Improving Access and Quality of Primary Care” (PMAQ) and from a political perspective.

PMAQ has the objective of inducing the increased access and improvement of primary health care quality, by means of mobilizing and holding responsible all agents of the process, including front line health workers. It has been implemented in every primary health care unity in the country and therefore generated new data and quantitative analyses in primary health care in Brazil. Comparatively, few implementation and qualitative analyses have been developed so far. Semi-structured questionnaires applied by us to front line health professionals (doctors, nurses, community health agents, and local managers) in Goiania have revealed that the program is mostly perceived as another top-down policy, in which all health workers are not involved (nurses and managers mainly), and in different ways consider (and not consider) it important to improve the quality of care, giving the political/rhetorical and organizational questions that arises in a complex implementation context.

In order to better explore and understand those new results related to PMAQ, we have interviewed 25 front line health workers so as to verify: 1) if and in what ways front line actors (and which of them) value the program, 2) which members of the health team effectively participated in the implementation of PMAQ and how it occurred and 3) if and how PMAQ modified the way in which the professionals assess and plan the work process.

The main contents/themes that came out from interviews were associated with political aspects highlighted by implementation theory and the more realistic-political approach of performance measurement studies. These literatures have stressed a list of factors which encourage or deter the implementation of PM (or are prone to foster unintended results). We have adapted their lists and associated them with themes revealed by front line health workers. The seven codes used in the analysis consist of: 1) The politics of  adhesion, 2) Culture and organizational capacity, 3) Culture of assessment/monitoring, 4) Participation in the implementation, 5) Perceived impact of PMAQ, 6) Feedback and uses of results and 7) Ambiguous rhetoric.

Discussion/Results – The analyses of the politics of implementation at the front line can be considered as an strategy to generate new contextualized evidences about PMAQ. The improvement of PMAQ at the front line would mean the initiation (or revision) of a new organizational culture in the implementation of primary health care/PMAQ at the implementation ground, privileging a broader participation and involvement from front line health workers, with higher possibility of creating a (new) assessment culture at the front line and, consequently, guided by a new form of adhesion, involving more feedback and uses of PMAQ’s results during both implementation and assessment, making thus possible to deconstruct rhetorics and ambiguities related to the program, and the construction of a new way of valuing PMAQ and the policy process related to it.

How Do Physician Executives Understand Performance Review and Assessment? A Longitudinal Q-method Analysis in a Public Health Organization

Alberto Asquer - aa144@soas.ac.uk - SOAS, University of London - United Kingdom

During last decades, various reforms informed by New Public Management doctrine have largely affected performance management and personnel policies by introducing, among other schemes, performance review and assessment (PRA) systems. PRA systems are generally expected to result in various positive effects at both the individual and organizational level, such as greater job commitment and satisfaction, employees’ motivation, and performance. When coupled with performance-related pay (PRP) schemes, PRA systems are supposed to trigger efforts to attain individual or organizational objectives because of individuals’ utilitarian expectation of rewards based on positive performance reviews. 


The reception of PRA, however, is controversial, especially in public sector organizations where professionalism norms and political context conditions contribute shaping individual identity and conduct. In the health sector, physician executives (or doctor managers) feel that their decisions should be largely informed by deontological considerations primarily related to the ethical standards of the medical profession rather than to the attainment of individual or organizational objectives. In context conditions where political affiliation matters for recruitment and career prospects, physician executives may sense that their job perspectives are more dependent on party connections rather than demonstrated professional achievements. In such professional and political organisations, what do physician executives think about PRA? How do they reconcile their understanding of performance measurement and appraisal with respect to other deontological principles and pragmatic criteria that orient their behavior?


This study employs a longitudinal Q method to provides some evidence of the subjective viewpoints of physician executives about PRA systems. Q method enables to access the subjective views of physician executives about the role of PRA. Longitudinal Q method permits to detect how subjectivities vary over time. The analysis focuses on data collected among physician executives in a public sector healthcare company in Italy in 2013 and in 2016. The longitudinal Q method analysis (factor analysis and varimax rotation) showed that physician executives hold diverse and fragmented views on the role of PRA, which can be characterized as ‘pragmatic’, ‘holistic’, and ‘disillusioned’. Interpretation of the results takes into account features of the ‘political bargain’ between the state and the medical profession in the public sector,  which included the adoption of accountability and managerial control policies that, in part, eroded the traditional ‘medical dominance’ in the health sector. In part, physician executives try and reconcile their understanding of performance measurement and appraisal with deontological principles that orient their behavior. In part, they may even regard the PRA system consistent with the canons of conduct of the medical profession, especially in the extent to which they consider it aligned with the attainment of health objectives of organizational units. In part, however, they view the PRA system as bearing little if any effects on behavior and performance, although they also consider the PRA system functional to the production of legitimacy for the health organization in the eyes of external stakeholders and political supervising agencies. 

Session 2 Global normative and political perspectives of performance measurement and reward

Friday, June 30th 13:45 to 15:45 (Block B 5 - 6)


Matthew Harris - m.harris@imperial.ac.uk - Imperial College London - United Kingdom

Fabiana C Saddi - fasaddi@usp.br - Federal University of Goias, Brazil - Brazil

Exploring the use of Payment by Results in health and social care in the UK

Chris O'Leary - c.oleary@mmu.ac.uk - Manchester Metropolitan University - United Kingdom

Over recent years, there has been increasing interest in ‘Payment by Results’ (PbR) (Pay for Success in the US) as a model for commissioning services in the public sector. A PbR contract links payment to the outcomes achieved, rather than the inputs, outputs or processes of a service (Cabinet Office 2011). By making some or all of payment to a service contingent on delivering agreed outcomes, PbR supposedly reduces ‘micro-management’ on the part of the commissioner, encourages innovation and transfers risk away from the branch of government commissioning the service towards the service provider because government will only pay if outcomes are achieved. From government’s perspective payments for service are deferred. Given the need to reduce public sector spending, both the transference of risk and deferring payment for services are attractive propositions for government. To date, over £15 billion of services in the UK are subject to PbR contracts (National Audit Office 2015), in areas such as criminal justice, healthcare, and social care. Payment by Results and Social Impact Bonds can be considered as the logical conclusion of outcome-based performance management (OBPM) (Lowe and Wilson, 2015), as they are intended to ensure that financial rewards directly flow from the achievement of specified outcomes. OBPM is a general term used for using outcomes as a means of assessing performance (Lowe, 2013), and different forms of OBPM have emerged since the 1990s. OBPM is associated with New Public Management (NOM) (Hood 1991).

Currently there is very little written in the academic literature on Payment by Results, with the majority of publications to date are policy briefings produced by government departments and Think Tanks. Such publications should be treated with caution because their treatment of the (limited) evidence base is often partial and they tend to ‘gloss over’ theoretical and ideological debates that are not consistent with their agenda. Further, publications in their field to date tend to concentrate on either the UK or the US experience.

This paper seeks to examine the use of Payment by Results in health and social care in the UK. It will draw on a Rapid Evidence Review of the literature on PbR. Although formal evaluations of both PbR and are still limited some evaluation findings are starting to be published and some tentative conclusions on the potential for innovation are drawn from the REA. I will build on and develop the limited theoretical discussion and, in particular, explore two themes: one that PbR drive innovation in the delivery of health and social care; the other that PbR are simply an extension of government outsourcing that ultimately prioritises corporate profits over social goods. I will also consider the impact of these approaches on not-for-profit and smaller players in the market for social outcomes.


Measuring share of drug sales in revenues of health facilities as a performance indicator in China

Chaojie Liu - c.liu@latrobe.edu.au - La Trobe University - Australia

China is the second largest pharmaceutical market in the world. Pharmaceutical sales account for 39% of China’s total health expenditure. In the 1980s and 1990s, China introduced a pricing system that set low service prices but allowed hospitals to make 15% markup from drug sales. This led to over-prescriptions. Drug sales account for 45-70% of hospital revenues.
To bring down medical costs, the government developed a hospital performance measurement system that monitors the share of drug sales in hospital revenues. The government intends to bring it down to 30% in 2017.
Since 2009, China has implemented the Essential Medicines List (EML) policy for primary care with prices of drugs on the EML set by the government at zero mark-up. However, most medicines (>70%) are dispensed from hospitals with prices set after negotiations between the government and manufacturers. Income of hospital staff continues to depend on the revenues they bring to the hospitals with low level of governmental investments. Under the 15% mark-up policy, over-prescriptions are common especially for expensive drugs. But when medical workers are no longer able to obtain financial benefits from drug prescriptions, they quickly shift priorities to other revenue generating activities (eg. intravenous drips) to compensate for the loss.
The drug performance measurement has attracted enormous attention from consumers. The public believe that medical costs would come down if good compliance of the governmental policies is achieved. But when those policies failed to achieve their intended goals, consumers started to blame health providers. Trust in medical practitioners was eroded, exacerbating medical disputes.
Generic drug sales dominate the Chinese market (>80%). However, some medicines are still heavily dependent on overseas suppliers. Most insulin products, for example, are imported. We found that significant differences of insulin availability exist across pharmacy outlets. Over 90% of public hospitals had pre-mixed insulin products. By contrast, insulin availability in community health centers was very low, with 10% to 20% of community health centers having insulin products.
The government sent out a clear signal to the public for its intention of developing an affordable medical services system. But these policies provide perverse incentives to health providers, stimulating profit-seeking behaviors and demand-inducing activities. This, in turn, has damaged the image of health providers, fueled medical disputes, and diminished patient trust in medical workers. The drug policies have also inadvertently placed primary care facilities to a weaker position for providing appropriate care due to low availability of drugs.
Many factors have shaped the current situation in China. Some may argue that culturally Chinese consumers are more likely to accept drug therapy. Others may blame the lack of a stringent medical education system for the poor prescription performance of medical practitioners. But the lack of participation and endorsement of consumers and health providers in the development of the drug performance measurement system is perhaps the fundamental reason undermining the results of those measurements. In a highly fragmented bureaucratic system, a top-down approach is unlikely to deliver a good policy product without meaningful engagement of the public and health providers.

Selection of Performance Measures in Context of Universal Health Coverage

Sundararaman Thiagarajan - sundar2016@gmail.com - Tata Institute of Social Sciences, Mumbai - India

Alok Ranjan - alokranjancmc@gmail.com - Tata Institute of Social Sciences, Mumbai - India

Priyanka Dixit - priyanka.dixit@tiss.edu - Tata Institute of Social Sciences, Mumbai - India

Background: Universal Health Coverage (UHC) has emerged as the major health policy discourse around the globe. Some of its proponents have even proclaimed it as third major transition after demographic and epidemiological transition, whereas others have called it as “old wine in new bottle”. In one sense every country is moving on the path of UHC, some are near the starting line, some are mid-way and some have reached closer to goal. In this context measure of progress towards UHC becomes the central discourse- and how one measures can influence both its meaning and its directions. This study based on the recently released India’s 71st Round National Sample Survey (NSS), 2014, measures progress in three contexts- as a comparative case study- its two most populous states, Uttar Pradesh (population: 199 million) and Maharashtra (population: 112 million) and for all India( population: 1221million).It discusses the implications of the choice of performance indicators with respect to the understanding of progress and the roadmaps.

Methodology: This National Sample Survey, 71st Round, 2014 was done for 65932 households (rural: 36480, urban: 29452) in India which included 3, 33,104 individuals). Also, 7921 and 5403 households were selected from Uttar Pradesh (UP) and Maharashtra, respectively. Insurance coverage, hospitalization rate, reimbursement, Out of pocket expenditure (OOPE), catastrophic health expenditure (CHE) at 10% (CHE-10) and 25% (CHE-25) and impoverishment were calculated for public and private healthcare providers. These indicators were also explored and evaluated through different equity dimensions of gender, caste, income quintile, and geographical location. Cross tabulation, multivariate logistic regression and propensity score matching were main analytical methods.  

Results: Insurance did not have facilitating role in increasing hospitalization rates.  Whereas chances of hospitalization consistently increased for richer category of population in all three contexts. Access to hospitalization was higher in higher income quintiles in both Uttar Pradesh – the states with one of the lowest human development index (HDI) in India and Maharashtra the state with one of the highest HDIs in India. Social group category played determining role in access to hospitalization in India and Maharashtra but not for Uttar Pradesh. Most persons who were insured did not get the benefit of cashless care and average OOPE between insured and non-insured offered some measure of protection in Uttar Pradesh, but not in Maharashtra. Propensity score matching showed government funded insurance schemes reduced CHE incidence for hospitalization at the 25% threshold by a meagre 6% in India. Out of pocket expenditure was significantly lower under public provisioning compare to private provisioning. Access to subsidized public services in contrast was more equitous and had a significant financial protection effect.

Conclusion: Measurement of health performance requires equity dimension integral to it. Government needs to be cautious while choosing insurance coverage as a performance measure in the discourse of UHC. When measuring financial protection both the type of provisioning and the type of financing needs to be studied together. This study fits in given panel topic.

Export PDF