LIST OF WORKSHOPS

Background

Health program evaluation is recognised as a crucial tool in building the evidence base for appropriate and responsive public health policy.  Despite this, evaluation is frequently overlooked, seen as an add-on rather than central to health policy and program design, or undertaken in such a way that it does not fulfil its potential to contribute to the available body of evidence.  Compounding this, interventions to address public health priorities are increasingly complex, leading to a need for expertise in the development and design of complex evaluations. 

This workshop will focus on the question of how to design evaluations of complex initiatives, programs and policies to improve public health policy and practice.  In particular, we aim to provide space to discuss how best to ensure that complex evaluations respond to the needs of diverse stakeholders, including policymakers, implementers, community members and end users.  Papers focusing on the use of theory in evaluation, the ethics of evaluation, evaluation design, stakeholder management, methodology, evidence-based decision-making and knowledge translation are welcomed.  

This panel aims to take advantage of the location of the first major ICPP initiative to be based in Latin America to strengthen ties with and within this region.  This includes the invitation of potential participants based within the Pan American Health Organisation (PAHO) Department of Evidence and Intelligence for Action in Health (EIH), eminent Latin American scholars currently undertaking evaluations of health initiatives as well as international experts in topics particularly relevant to the region, including evaluating emerging technologies and approaches in communicable disease control and the use of big data in addressing transnational health threats.  As such, we anticipate that this workshop will be a strong contributor to knowledge translation in the field of health program evaluation and policy.

The panel chairs are also representative of this approach. The chairs collectively bring together expertise in Latin American health policy and program research; health policy and management; and the design and implementation of complex health program evaluations. 

Research questions

The workshop will respond to the following questions:

1.       What are the ethical considerations present when conducting complex evaluations in health?  Who has the obligation to respond to these (evaluators, implementors, evaluation commissioners etc)?

2.       How can effective stakeholder management be undertaken to facilitate the use of evidence in public health policy?

3.       What guidance exists for the use of innovative methods in evaluation?  How can these be applied to complex evaluations?

Workshop structure

The workshop will consist of 15-20 presentations; in addition to these individual presentations, a number of roundtable discussions will be scheduled in accordance with the workshop questions.  Discussions will centre around the following themes:

·       The ethics of evaluation

·       Stakeholder engagement for effective application

·       Use of innovative methods in evaluation, including participatory evaluation design and methods

·       Evaluations of complex health initiatives in Latin American contexts

Expected publication

If we are successful in having our panel selected for participation in IWPP2, we intend to approach the following journals about developing a special issue:

·                Health Policy

·                Implementation Science

·                Social Science and Medicine

Potential participants

Potential participants have been selected due to their expertise in their field as well as the relevance of their work to questions of particular import to the Latin American and Caribbean region.  With this selection of potential participants, this panel will represent an exciting opportunity for cross-disciplinary approaches to tackling the challenges of developing responsive and effective evaluations of complex health interventions.

·       Langlois, EV: Embedding implementation research to enhance health policy and systems: a multi-country analysis from ten settings in Latin America and the Caribbean

·       Jarbas Barbosa da Silva: Regional approaches to evidence and intelligence for action in health

·       Volker Amelung: The evaluation of complex initiatives in integrated health systems

·       Margaret Kelaher:  Enhancing effective uptake of evaluation findings into public health policy

·       Pablo Tsukayama: How genomics is being applied to epidemiological surveillance and public health in Latin America

·       Dr Silvia Argimon: Utilising health policy to facilitate genomic public health approaches in Latin America

·       Angeline Ferdinand:  Ethical considerations in evaluating genomics and genetic health initiatives

·       Ana María Oyarce: Epidemiological data collection and the evaluation of national health initiatives in Chile

·       Steven J. Hoffman: Big data approaches to transnational health threats and social inequalities

·       Juan Pablo Villanueva Cabezas: One Health approaches to food insecurity and antimicrobial resistance

·       Steffen Eckhard: The politics of evaluation in international organizations: A comparative study of stakeholder influence potential

Health program evaluation is recognised as a crucial tool in building the evidence base for appropriate and responsive public health policy.  Despite this, evaluation is frequently overlooked, seen as an add-on rather than central to health policy and program design, or undertaken in such a way that it does not fulfil its potential to contribute to the available body of evidence.  Compounding this, interventions to address public health priorities are increasingly complex, leading to a need for expertise in the development and design of complex evaluations.

This workshop will focus on the question of how to design evaluations of complex initiatives, programs and policies to improve public health policy and practice.  In particular, we aim to provide space to discuss how best to ensure that complex evaluations respond to the needs of diverse stakeholders, including policymakers, implementers, community members and end users. 

This workshop is designed to foster a community of scholars working on the evaluation of complex health programs, policies and initiatives.   As such, the workshop will bring together academics undertaking evaluation in a variety of public health fields as well as experts in the science of evaluation.  Papers relating to the evaluation of health programs in Latin America or in fields of particular relevance to Latin America (e.g. communicable disease control in low to middle resource settings, achieving health equity, regional approaches to transnational health threats) are especially welcome.

We envision papers centered around (but not limited to) the following themes:

• The ethics of evaluation

• Stakeholder engagement for effective application

• Use of innovative methods in evaluation, including participatory evaluation design and methods

Papers developed successfully as a result of the workshop will be invited to participate in one or more special issue(s) in relevant internationally ranked journals.

This workshop aims to provide space to discuss how best to ensure that complex evaluations respond to the needs of diverse stakeholders, including policymakers, implementers, community members and end users. 

The workshop will respond to the following questions:

1.            What are the ethical considerations present when conducting complex evaluations in health?  Who has the obligation to respond to these (evaluators, implementers, evaluation commissioners etc)?

2.            How can effective stakeholder management be undertaken to facilitate the use of evidence in public health policy?

3.            What guidance exists for the use of innovative methods in evaluation?  How can these be applied to complex evaluations?

Angeline Ferdinand

Ms Ferdinand graduated from Cornell University with a BA in biological sciences (genetics) before undertaking proteomics research with Harvard Medical School.  Subsequently, Ms Ferdinand went on to focus on developing research that is applicable to the challenging and complex social issues that perpetuate inequitable health outcomes.  Over the past decade, she has undertaken health program evaluations that span the breadth of local, national and international approaches.  Ms Ferdinand’s career thus reflects her belief that both social and biomedical approaches to health are necessary and can be complementary.

Ms Ferdinand’s research includes evaluations of complex place-based programs to improve the mental health of migrant and Aboriginal communities by addressing racism; a national-level evaluation to assess the delivery of disability services for Aboriginal families and communities; and an examination of Indigenous community participation in health from a global perspective.  Her dissertation, entitled Models of Indigenous community-controlled health care in Chile, evaluates and critiques local models of Indigenous community participation in health and the extent to which national and international legislative and policy environments support their goal of Indigenous self-determination.  This research has strong roots in complexity science, as it investigates the phenomenon of community participation as a system affected by different categories of stakeholders (service providers, government agencies and patients). 

Ms Ferdinand is currently undertaking an internationally groundbreaking evaluation of the application of microbial genomics in public health surveillance and outbreak control.

Volker E. Amelung was born in 1965. He studied Business Administration at the Universities of St. Gallen and Paris-Dauphine. After his graduation, he has been working at the University for Economy and Political Science in Hamburg, and had been a visiting scientist at the Columbia University in New York for several years.

In 2001 Volker Amelung has been appointed professor at the Medical University of Hannover with focus on health management and international health services research. He also has lectureships at the Medical University and Economy University of Vienna, at the Columbia University (NY, USA), at the TiasNimbas Business School (NL), at the advanced technical college of Carinthia (AT), the European Business School and the Technical University of Braunschweig (DE). His main research

focuses on managed care, integrated health care, health services research and health management. He has co-authored several books, and has published more than 100 papers on managed care, health care policy, new technologies and integrated delivery systems.

Since 2007 Volker Amelung is also the president of the German Managed Care Association (BMC), Berlin. The BMC is one of the leading independent health care associations, elaborating innovative concepts in health care management and health care policy. In 2011 he founded the private institute for applied health research (inav) in Berlin.

Ana María Oyarce

Professor Ana María Oyarce is an anthropologist internationally recognised for her expertise in Indigenous health, and particularly known for her focus on cross-cultural concepts of health, the valorisation of Indigenous health knowledge and the articulation between traditional Indigenous models of health and biomedical care.  She is a pioneer in her field of intercultural health and ethnic disparities in health and has authored dozens of reports, manuscripts and books on these topics.    In 1997, Professor Oyarce was awarded a Rockefeller Foundation Fellowship to obtain a Master's degree in Health Social Science in Australia, and she was awarded the Fulbright Fellowships in 2000 to undertake her PhD in Medical Anthropology in the United States. 

In addition to her position at the University of Chile, Professor Oyarce has been an international consultant to the Economic Commission for Latin America (CEPAL) and the World Health Organisation for many years.  As such, she has conducted major international comparative studies regarding Indigenous demography and health policy across Latin America and internationally.  Her work has contributed to improving the identification of Indigenous peoples in demographic and epidemiological data and the collection of Indigenous health data more generally, which has been crucial for the development of appropriate Indigenous health policy across the continent.  Within Chile, she is a national consultant on intercultural health and has been instrumental in shaping the country’s approach to Indigenous health policy and the official recognition and validation of Indigenous traditional health care.  Professor Oyarce is co-author of Medicines and Cultures in the Araucanía, which continues to be the definitive text on the traditional health beliefs and culture of the Mapuche indigenous people of Chile, and the intersection between the two. 

Margaret Kelaher

Margaret Kelaher is Head of Evaluation and Implementation Science at the Centre for Health Policy, Melbourne School of Population and Global Health, University of Melbourne. Margaret has established an international reputation in research to improve health equity. Her work has a unique focus, bringing an action orientated and intervention-based approach to understanding complex health issues and informing theory and policy. She is one of the few health researchers whose work encompasses the continuum of population health from social determinants of health to health service delivery. Her main contributions include demonstrable policy change and methodological innovation to better inform policy. For example, her research on access to medicines among clients of remote area Indigenous health services led to changes in the Pharmaceutical Benefits Scheme to enable the approval of medicines specific to the needs of Aboriginal people and provided the impetus for programs to improve access to medicines among regional and urban Indigenous people. She is currently leading major research to ensure that genomics and genetic health services meet the needs of Aboriginal and Torres Strait Islander people.

She has attracted over $34 million in funding through competitive grants and has published over 150 peer reviewed journal articles and 130 books and reports. She has published in the leading journals in her field including, Social Science and Medicine, Health Policy and the International Journal of Epidemiology.

Her achievements have been recognised with an NHMRC Sidney Sax Fellowship, Australian Young Tall Poppy award, NHMRC Career Development award, VicHealth senior research fellowship and an ARC Future Fellowship.