T03P11. Bringing Politics to the Analysis of Performance Measurement Programs: Case and Comparative Studies in Health Policy

Topic : Policy and Politics sponsored by Policy & Politics Journal

Chair : Fabiana SADDI (Universidade de Brasília)

Second Chair : Nick Turnbull (University of Manchester)

Share to Facebook Share to Twitter Share to Linkedin Share by mail

General Objectives, Research Questions and Scientific Relevance

In this panel a selection of papers may be considered for the Policy & Politics journal.

 

 

Co-organized by Fabiana C. Saddi, Stephen Peckham, Nick Turnbull and Matthew J. Harris.

The objective is to gather political-realistic studies focusing on either or both policymaking and implementation processes of performance measurement (PM) programs in health policy in developed and/or developing counties, as a case or comparative study.

PM programs have been adopted in countries with distinct levels of development, and tend to continue to play an important role in policymaking. In this process, the adoption of PM has revealed some challenges during implementation and has therefore, though in different rhythm between countries, been accompanied by the valorization of political-realistic or more post-positivist type of analyses. Those programs are constructed and implemented in political and social environments with distinct organizational capacity and where people hold values and interests that can influence the implementation of rational-based PM programs. This is why concerns based on who are involved in its elaboration and implementation, as well as on where/how those processes have been realized, have recently contributed to enhance the importance of taking the politics, the cognitive/subjective (“alternative logics”) and work task and organizational aspects of PM programs into account. They have also contributed to better understand and unfold some dynamics and regularities that go beyond rational-based concerns. This literature emphasizes aspects such as political system, organizational culture, participation of staff in the implementation, appropriateness of the design, the possibilities of gaming (Bevan and Hood) and cheating and symbolic uses. Also, concerns and consequences regarding performance measurement programs have been categorized as “performance alternative logics” (Pollitt), as the “politics of performance” (Lewis) and as “performance paradox”, as examples. 

When applied to middle and low income countries, studies have given emphases not only to front line staff’s involvement (Songstad et al.) (Chimhutu et al.) (Ssengooba F et al.), but especially to organizational constraints (Olafsdottir et al.), given the fact that the policies still face some contradictory organizational problems (Saddi and Harris et al.). Those works are considered important for having enhanced the knowledge on motivation and impact regarding front line workers in contradictory or problematic contexts, as well as for shedding lights on how to enable the creation of a culture of evaluation in diverse and not always favorable organizational and political environments.

From the policy diffusion perspective, however, we still know little comparatively about the distinctive and politically significant challenges involved in the implementation of PM programs not only across health unities with different configurations in each country, but also across countries with distinctive and similar levels of development. 

If those issues constitute a significant lacuna in the knowledge of comparative health policy and politics, shouldn’t we develop comparative political analyses evaluating how PM have been designed and implemented? What methods could be used to develop meaningful comparisons across countries, taking each reality into account? Could differences be explained in terms of institutional heritages, or by means of using a comprehensive and long-term political analysis? What lessons could be partially and meaningfully transferred from developed to developing countries and vice versa?

Call for papers

Co-organizers and chairs: Fabiana C. Saddi (Federal University of Goias, Brazi - fasaddi@usp.br), Stephen Peckham (London School of Hygiene and Tropical Medicine and University of Kent - s.peckham@kent.ac.uk), Nick Turnbull (University of Manchester, nick.turnbull@manchester.ac.uk), Matthew J. Harris (Imperial College London - m.harris@imperial.ac.uk ).

 

This panel welcomes papers focusing on either or both the policymaking and implementation process(es) of performance measurement programs (PM) adopted in health policy in distinct countries in the last years. We expect papers to take into account the actors, ideas and interests involved in the policymaking and/or implementation phases in diverse institutional setting(s) and macro/micro political context(s). Papers can be applied to either primary health care or specialized health care policies. Analyses should focus on political or political-realistic aspects of policy-making and/or implementation processes, or establish politically significant relationships between both processes. We welcome studies that consider policymaking from the view point of social learning (Hall), policy transfer (Dunlop), feedback (Jacobs), policy regime change (May), state capacity, performance regimes and system of performance (Talbot) and/or as communicative practice (Fischer) (Turnbull) or from other interactive perspective. Implementation analyses that have applied surveys, semi-structured and open interviews, as well as developed focus groups or policy dialogues with front line health workers are highly encouraged. Papers highlighting the inherent problems of measuring performance in health care delivery when comparing those interventions where the medical intervention and professional practice has only a partial effect and where self-care and informal care may play a larger role in success (Peckham) are welcome. Country analyses of PM programs and comparisons across countries employing mixed-methods, qualitative and long-term analyses, as well as political-sociological and institutional type of policy analyses will also be considered. Papers that deal with the theme of this panel in innovative and politically and policy relevant ways will be highly appreciated.

ROOM
Block B 5 - 6
Fri 30th
10:30
Session 1 Political processes, actors and approaches in pay for performance

Discussants : Fabiana SADDI (Universidade de Brasília)

Discussants : Nick Turnbull (University of Manchester)

Responsive image
PDF
How do diffusion agents make their policy travel across international, continental, and national levels? The case of performance-based financing

Lara Gautier (Université de Montréal)

Manuela De Allegri (Institute of Public Health, Heidelberg University)

Ridde Valery (Institute for Research on Sustainable Development)

Context: Over the past decade, several donors have promoted performance-based financing (PBF) in healthcare facilities in Susbaharan Africa, in view of  increasing the quantity and quality of health services provision. PBF supposes a shift from an input-based financial system to an output-based approach, thereby spurring a managerial approach to healthcare provision.

Case selection: The case of PBF, an innovative healthcare financing policy embedded in economics theories and new public management, offers an interesting research opportunity in that it involves policy diffusion processes that, we argue, have unprecedentedly been planned and driven (financially, scientifically, technically, discursively) by a nexus of diffusion agents. These include several North-based individuals, networks, and formal institutions.

Discussion: We provide theoretical reflections on diffusion agents’ critical characteristics, and on their ability – empowered by a facilitating opportunity structure – to deliberately plan, develop and support an apparatus that spark diffusion at the global, continental, and national levels. Based on Hassenfeutel’s typology, we argue that in order to be successful, diffusion agents typically need to gather material, knowledge, political, social, and temporal resources. In the case of PBF, should diffusion agents bring these resources together and benefit from a facilitating opportunity structure, they would be empowered to develop an apparatus that enables: i) framing of PBF through discourse, ii) emulation through the making of a PBF community, iii) learning through the production and dissemination of multiple forms of knowledge, and iv) building consensus through country-level cooperation.

Research value: Bringing together two major fields of research, this paper critically draws from public policy literature on diffusion while taking into account the multilevel and multiactor complexity of global health. This is the first research focusing on diffusion agents’ characteristics and strategies aimed at diffusing a global health innovation (PBF) in Subsaharan Africa.

Responsive image
PDF
Contrasting approaches to Primary Care Performance Governance in Denmark and New Zealand

Tim Tenbensel (University of Auckland)

Primary care is a crucial component of health systems, and one which governments typically have a strong interest in being able to steer. Increasingly, this steering is taking place through ‘performance governance’ – the incorporation of performance measurement into institutionalised policy processes. Primary care presents many governance challenges because it is predominantly provided by independent practitioners in small organisations. In this article we compare two small, high-income countries with tax-funded health systems - Denmark and New Zealand which have adopted quite different instruments for performance governance. The Danish state governs primary care performance using ‘soft hierarchy’ based on accreditation processes but few strong sanctions, while New Zealand has relied more on a combination of explicit hierarchical targets and financial incentives. To explain this key difference, we use a conceptual framework that charts the connections between: (i) institutional contexts, including the organisational structure of primary care; (ii) governance processes (corporatist or pluralist); and (iii) governance problems such as access, equity, efficiency, quality, and population health. We argue that the specific nature of primary care institutions have a significant impact on regimes of performance governance. Our comparative framework has the potential to be applied across a wider range of countries.

Responsive image
PDF
The politics of implementing a performance measurement program (PMAQ) at the front line of primary health care in Goiania, Brazil: a qualitative political analysis

Fabiana SADDI (Universidade de Brasília)

Matthew Harris (Imperial College London)

Fernanda Parreira (Universidade Federal de Goiás)

Raquel Pego (Faculdade de Saúde Coletiva, UNB)

This paper´s objective is to understand how front line health workers in Goiania evaluate the Brazilian “National Program for Improving Access and Quality of Primary Care” (PMAQ) and from a political perspective.

PMAQ has the objective of inducing the increased access and improvement of primary health care quality, by means of mobilizing and holding responsible all agents of the process, including front line health workers. It has been implemented in every primary health care unity in the country and therefore generated new data and quantitative analyses in primary health care in Brazil. Comparatively, few implementation and qualitative analyses have been developed so far. Semi-structured questionnaires applied by us to front line health professionals (doctors, nurses, community health agents, and local managers) in Goiania have revealed that the program is mostly perceived as another top-down policy, in which all health workers are not involved (nurses and managers mainly), and in different ways consider (and not consider) it important to improve the quality of care, giving the political/rhetorical and organizational questions that arises in a complex implementation context.

In order to better explore and understand those new results related to PMAQ, we have interviewed 25 front line health workers so as to verify: 1) if and in what ways front line actors (and which of them) value the program, 2) which members of the health team effectively participated in the implementation of PMAQ and how it occurred and 3) if and how PMAQ modified the way in which the professionals assess and plan the work process.

The main contents/themes that came out from interviews were associated with political aspects highlighted by implementation theory and the more realistic-political approach of performance measurement studies. These literatures have stressed a list of factors which encourage or deter the implementation of PM (or are prone to foster unintended results). We have adapted their lists and associated them with themes revealed by front line health workers. The seven codes used in the analysis consist of: 1) The politics of  adhesion, 2) Culture and organizational capacity, 3) Culture of assessment/monitoring, 4) Participation in the implementation, 5) Perceived impact of PMAQ, 6) Feedback and uses of results and 7) Ambiguous rhetoric.

Discussion/Results – The analyses of the politics of implementation at the front line can be considered as an strategy to generate new contextualized evidences about PMAQ. The improvement of PMAQ at the front line would mean the initiation (or revision) of a new organizational culture in the implementation of primary health care/PMAQ at the implementation ground, privileging a broader participation and involvement from front line health workers, with higher possibility of creating a (new) assessment culture at the front line and, consequently, guided by a new form of adhesion, involving more feedback and uses of PMAQ’s results during both implementation and assessment, making thus possible to deconstruct rhetorics and ambiguities related to the program, and the construction of a new way of valuing PMAQ and the policy process related to it.

Responsive image
PDF
How Do Physician Executives Understand Performance Review and Assessment? A Longitudinal Q-method Analysis in a Public Health Organization

Alberto Asquer (School of Oriental and African Studies, University of London)

During last decades, various reforms informed by New Public Management doctrine have largely affected performance management and personnel policies by introducing, among other schemes, performance review and assessment (PRA) systems. PRA systems are generally expected to result in various positive effects at both the individual and organizational level, such as greater job commitment and satisfaction, employees’ motivation, and performance. When coupled with performance-related pay (PRP) schemes, PRA systems are supposed to trigger efforts to attain individual or organizational objectives because of individuals’ utilitarian expectation of rewards based on positive performance reviews. 

 


The reception of PRA, however, is controversial, especially in public sector organizations where professionalism norms and political context conditions contribute shaping individual identity and conduct. In the health sector, physician executives (or doctor managers) feel that their decisions should be largely informed by deontological considerations primarily related to the ethical standards of the medical profession rather than to the attainment of individual or organizational objectives. In context conditions where political affiliation matters for recruitment and career prospects, physician executives may sense that their job perspectives are more dependent on party connections rather than demonstrated professional achievements. In such professional and political organisations, what do physician executives think about PRA? How do they reconcile their understanding of performance measurement and appraisal with respect to other deontological principles and pragmatic criteria that orient their behavior?

 


This study employs a longitudinal Q method to provides some evidence of the subjective viewpoints of physician executives about PRA systems. Q method enables to access the subjective views of physician executives about the role of PRA. Longitudinal Q method permits to detect how subjectivities vary over time. The analysis focuses on data collected among physician executives in a public sector healthcare company in Italy in 2013 and in 2016. The longitudinal Q method analysis (factor analysis and varimax rotation) showed that physician executives hold diverse and fragmented views on the role of PRA, which can be characterized as ‘pragmatic’, ‘holistic’, and ‘disillusioned’. Interpretation of the results takes into account features of the ‘political bargain’ between the state and the medical profession in the public sector,  which included the adoption of accountability and managerial control policies that, in part, eroded the traditional ‘medical dominance’ in the health sector. In part, physician executives try and reconcile their understanding of performance measurement and appraisal with deontological principles that orient their behavior. In part, they may even regard the PRA system consistent with the canons of conduct of the medical profession, especially in the extent to which they consider it aligned with the attainment of health objectives of organizational units. In part, however, they view the PRA system as bearing little if any effects on behavior and performance, although they also consider the PRA system functional to the production of legitimacy for the health organization in the eyes of external stakeholders and political supervising agencies. 

ROOM
Block B 5 - 6
Fri 30th
13:45
Session 2 Global normative and political perspectives of performance measurement and reward

Discussants : Nick Turnbull (University of Manchester)

Discussants : Fabiana SADDI (Universidade de Brasília)

Responsive image
PDF
Exploring the use of Payment by Results in health and social care in the UK

Chris O'Leary (Manchester Metropolitan University)

Over recent years, there has been increasing interest in ‘Payment by Results’ (PbR) (Pay for Success in the US) as a model for commissioning services in the public sector. A PbR contract links payment to the outcomes achieved, rather than the inputs, outputs or processes of a service (Cabinet Office 2011). By making some or all of payment to a service contingent on delivering agreed outcomes, PbR supposedly reduces ‘micro-management’ on the part of the commissioner, encourages innovation and transfers risk away from the branch of government commissioning the service towards the service provider because government will only pay if outcomes are achieved. From government’s perspective payments for service are deferred. Given the need to reduce public sector spending, both the transference of risk and deferring payment for services are attractive propositions for government. To date, over £15 billion of services in the UK are subject to PbR contracts (National Audit Office 2015), in areas such as criminal justice, healthcare, and social care. Payment by Results and Social Impact Bonds can be considered as the logical conclusion of outcome-based performance management (OBPM) (Lowe and Wilson, 2015), as they are intended to ensure that financial rewards directly flow from the achievement of specified outcomes. OBPM is a general term used for using outcomes as a means of assessing performance (Lowe, 2013), and different forms of OBPM have emerged since the 1990s. OBPM is associated with New Public Management (NOM) (Hood 1991).

Currently there is very little written in the academic literature on Payment by Results, with the majority of publications to date are policy briefings produced by government departments and Think Tanks. Such publications should be treated with caution because their treatment of the (limited) evidence base is often partial and they tend to ‘gloss over’ theoretical and ideological debates that are not consistent with their agenda. Further, publications in their field to date tend to concentrate on either the UK or the US experience.

This paper seeks to examine the use of Payment by Results in health and social care in the UK. It will draw on a Rapid Evidence Review of the literature on PbR. Although formal evaluations of both PbR and are still limited some evaluation findings are starting to be published and some tentative conclusions on the potential for innovation are drawn from the REA. I will build on and develop the limited theoretical discussion and, in particular, explore two themes: one that PbR drive innovation in the delivery of health and social care; the other that PbR are simply an extension of government outsourcing that ultimately prioritises corporate profits over social goods. I will also consider the impact of these approaches on not-for-profit and smaller players in the market for social outcomes.

 

Responsive image
PDF
Measuring share of drug sales in revenues of health facilities as a performance indicator in China

Chaojie Liu (La Trobe University)

China is the second largest pharmaceutical market in the world. Pharmaceutical sales account for 39% of China’s total health expenditure. In the 1980s and 1990s, China introduced a pricing system that set low service prices but allowed hospitals to make 15% markup from drug sales. This led to over-prescriptions. Drug sales account for 45-70% of hospital revenues.
To bring down medical costs, the government developed a hospital performance measurement system that monitors the share of drug sales in hospital revenues. The government intends to bring it down to 30% in 2017.
Since 2009, China has implemented the Essential Medicines List (EML) policy for primary care with prices of drugs on the EML set by the government at zero mark-up. However, most medicines (>70%) are dispensed from hospitals with prices set after negotiations between the government and manufacturers. Income of hospital staff continues to depend on the revenues they bring to the hospitals with low level of governmental investments. Under the 15% mark-up policy, over-prescriptions are common especially for expensive drugs. But when medical workers are no longer able to obtain financial benefits from drug prescriptions, they quickly shift priorities to other revenue generating activities (eg. intravenous drips) to compensate for the loss.
The drug performance measurement has attracted enormous attention from consumers. The public believe that medical costs would come down if good compliance of the governmental policies is achieved. But when those policies failed to achieve their intended goals, consumers started to blame health providers. Trust in medical practitioners was eroded, exacerbating medical disputes.
Generic drug sales dominate the Chinese market (>80%). However, some medicines are still heavily dependent on overseas suppliers. Most insulin products, for example, are imported. We found that significant differences of insulin availability exist across pharmacy outlets. Over 90% of public hospitals had pre-mixed insulin products. By contrast, insulin availability in community health centers was very low, with 10% to 20% of community health centers having insulin products.
The government sent out a clear signal to the public for its intention of developing an affordable medical services system. But these policies provide perverse incentives to health providers, stimulating profit-seeking behaviors and demand-inducing activities. This, in turn, has damaged the image of health providers, fueled medical disputes, and diminished patient trust in medical workers. The drug policies have also inadvertently placed primary care facilities to a weaker position for providing appropriate care due to low availability of drugs.
Many factors have shaped the current situation in China. Some may argue that culturally Chinese consumers are more likely to accept drug therapy. Others may blame the lack of a stringent medical education system for the poor prescription performance of medical practitioners. But the lack of participation and endorsement of consumers and health providers in the development of the drug performance measurement system is perhaps the fundamental reason undermining the results of those measurements. In a highly fragmented bureaucratic system, a top-down approach is unlikely to deliver a good policy product without meaningful engagement of the public and health providers.

Responsive image
PDF
Selection of Performance Measures in Context of Universal Health Coverage

Sundararaman Thiagarajan (Tata Institute of Social Sciences, Mumbai)

Alok Ranjan (Indian Institute of Technology-Jodhpur)

Priyanka Dixit (Tata Institute of Social Sciences, Mumbai)

Background: Universal Health Coverage (UHC) has emerged as the major health policy discourse around the globe. Some of its proponents have even proclaimed it as third major transition after demographic and epidemiological transition, whereas others have called it as “old wine in new bottle”. In one sense every country is moving on the path of UHC, some are near the starting line, some are mid-way and some have reached closer to goal. In this context measure of progress towards UHC becomes the central discourse- and how one measures can influence both its meaning and its directions. This study based on the recently released India’s 71st Round National Sample Survey (NSS), 2014, measures progress in three contexts- as a comparative case study- its two most populous states, Uttar Pradesh (population: 199 million) and Maharashtra (population: 112 million) and for all India( population: 1221million).It discusses the implications of the choice of performance indicators with respect to the understanding of progress and the roadmaps.

Methodology: This National Sample Survey, 71st Round, 2014 was done for 65932 households (rural: 36480, urban: 29452) in India which included 3, 33,104 individuals). Also, 7921 and 5403 households were selected from Uttar Pradesh (UP) and Maharashtra, respectively. Insurance coverage, hospitalization rate, reimbursement, Out of pocket expenditure (OOPE), catastrophic health expenditure (CHE) at 10% (CHE-10) and 25% (CHE-25) and impoverishment were calculated for public and private healthcare providers. These indicators were also explored and evaluated through different equity dimensions of gender, caste, income quintile, and geographical location. Cross tabulation, multivariate logistic regression and propensity score matching were main analytical methods.  

Results: Insurance did not have facilitating role in increasing hospitalization rates.  Whereas chances of hospitalization consistently increased for richer category of population in all three contexts. Access to hospitalization was higher in higher income quintiles in both Uttar Pradesh – the states with one of the lowest human development index (HDI) in India and Maharashtra the state with one of the highest HDIs in India. Social group category played determining role in access to hospitalization in India and Maharashtra but not for Uttar Pradesh. Most persons who were insured did not get the benefit of cashless care and average OOPE between insured and non-insured offered some measure of protection in Uttar Pradesh, but not in Maharashtra. Propensity score matching showed government funded insurance schemes reduced CHE incidence for hospitalization at the 25% threshold by a meagre 6% in India. Out of pocket expenditure was significantly lower under public provisioning compare to private provisioning. Access to subsidized public services in contrast was more equitous and had a significant financial protection effect.

Conclusion: Measurement of health performance requires equity dimension integral to it. Government needs to be cautious while choosing insurance coverage as a performance measure in the discourse of UHC. When measuring financial protection both the type of provisioning and the type of financing needs to be studied together. This study fits in given panel topic.