Format

PR03. Learning from Evidence: Rethinking Monitoring and Evaluation through Qualitative and Ethnographic Insights

Practice
PANEL CHAIR(S)
Padmini Ram
Urban Ethnographers, India
Main chair
CATEGORISATION
POLICY TOPIC
Practice
SECTOR
—
KEYWORDS
—
GENERAL OBJECTIVES, RESEARCH QUESTIONS AND SCIENTIFIC RELEVANCE

Monitoring and Evaluation (M&E) has become central to India’s development governance architecture, shaping decisions on program design, performance management, and accountability across sectors. Yet an increasing dependence on quantitative indicators, digital dashboards, and standardised metrics often constrains the ability to understand how policies actually unfold in diverse implementation contexts. This panel approaches M&E not solely as a technical apparatus but as an institutional and social process shaped by frontline discretion, community engagement, everyday administrative routines, and local political economy.

Aligning with the 2026 IPPN conference theme on public policy praxis in the Global South, the panel argues for evidence systems that capture contextual complexity and lived experience, elements essential for coherence, adaptability, and resilience in rapidly changing policy environments. Recent research from health, nutrition, education, water governance, and women’s empowerment illustrates how frontline actors interpret, negotiate, and sometimes reconfigure monitoring expectations in response to local constraints. These micro-practices influence outcomes as substantively as formal indicators, yet they are seldom recognised within existing M&E frameworks.

By foregrounding qualitative and ethnographic insights, the panel asks how evaluation systems can better incorporate experiential knowledge, gendered perspectives, and informal practices that shape implementation. It aims to shift attention from compliance-driven measurement toward learning-oriented evaluation that supports inclusive and adaptive public policy.

Research Questions
How do qualitative and ethnographic approaches reshape the understanding of how M&E systems influence development outcomes and institutional learning in India?
What forms of local and experiential knowledge are overlooked by current monitoring frameworks, and how might they be integrated into adaptive policymaking?
How can participatory and ethnographic evaluations bridge the gap between measurement, accountability, and reflexive learning in development practice?

CALL FOR PAPERS

This panel invites contributions that demonstrate how qualitative, interpretive, and ethnographic approaches can reshape Monitoring and Evaluation (M&E) systems to better support institutional learning and adaptive governance. As scholars of development have long argued, numerical indicators often obscure the social, relational, and political processes through which policies are implemented on the ground (Eyben, 2015; Mosse, 2005). In India’s complex administrative landscape, frontline discretion, community interactions, and everyday bureaucratic practices shape outcomes as profoundly as formal program designs, making qualitative inquiry essential for understanding how policies actually work in practice (Gupta, 2012; Puri, 2021).

To this end, the panel seeks empirical case studies that illuminate the lived realities of implementation, whether in health, education, nutrition, water governance, or social protection. Submissions may explore how frontline workers navigate monitoring expectations, reinterpret performance metrics, or negotiate accountability demands in resource-constrained settings (Chambers, 2017). Participatory, embedded, or collaborative evaluations that centre community voices and experiential knowledge are also welcome, especially those that highlight gendered or intersectional dimensions of program delivery.

We additionally encourage methodological contributions comparing qualitative, mixed-methods, and ethnographic evaluation approaches, or reflecting on the challenges of integrating qualitative insights into state evidence systems that remain primarily quantitative and compliance-driven. Analyses that examine how governments, civil society organisations, or donors incorporate, or fail to incorporate, contextual knowledge, tacit expertise, or informal practices into decision-making are particularly relevant.

These are the potential panellists:

Dr. Aarti Madhusudhan, Founder, Governance Counts 
Maya Menon, Founder, The Teacher Foundation 
Dr. Prashanth N. Srinivas, Institute of Public Health (IPH)
Together, these contributions aim to advance a more reflective, context-sensitive, and learning-oriented vision of M&E in India


References:

Chambers, R. (2017). Can we know better? Reflections for development. Practical Action Publishing.
https://doi.org/10.3362/9781780449449

Eyben, R. (2015). International aid and the making of a better world: Reflexive practice. Routledge.

Gupta, A. (2012). Red tape: Bureaucracy, structural violence, and poverty in India. Duke University Press.
https://doi.org/10.1515/9780822394709

Mosse, D. (2005). Cultivating development: An ethnography of aid policy and practice. Pluto Press.

Puri, J. (2021). Integrating qualitative methods in impact evaluations: Tackling complexity in development programmes. Journal of Development Effectiveness, 13(3), 265–280.
https://doi.org/10.1080/19439342.2021.1947802

ACCEPTED
Format
01 - Workplace Ethnography for Emerging Indian Labour Market
Lakshmi Parvathy (IIM Sirmaur)
CO-AUTHORS
—
ABSTRACT

 
The labour contracting landscape in India had relied mainly on the informal individual labour contractor (jobber, thekedaars, maistries, kangaanies and so on) operating in their little geographies to source workers. This has changed post-2000 with the entry of manpower supply companies, temping companies, flexi-staffing industry, or human resource companies (HRC). Although the latter entity can be seen as being evolved from the former to become more formal, in the face of it, they both are performing the same task – recruiting casual labour or meeting the supply-demand gaps and ensuring a good fit of required skills. In doing so, they smoothen the volatility of labour force requirements in industries prone to the vagaries of global demand trends. In this process of brokering, the labour market intermediaries reduce various transaction costs such as search, verification, monitoring, compliance. To understand how the HRC addresses these costs, we do a workplace ethnography working in an HRC for eight-month long period understanding their processes. Methods such as interviews, participant observation and unstructured interviews were employed to collect the relevant data over and above the daily field notes.
After studying the organization structure closely, we realized that within the workforce deployed by Wergo, staffing business of unskilled and semi-skilled labour deployment (referred to as generalized staffing) made up 90% of their revenue base. Although they supply workers to various sectors including banking and financial services, telecom and technology and consumer durables, we study their retail, ecommerce and quick commerce (RECom) sector to understand the process of staffing. The RECom, primarily the q-commerce industry instantly delivered daily groceries, vegetables, and fruits to customers' doorsteps. To ensure quick delivery, they have various warehouses and hubs spread out through the cities called as ‘dark-stores’. Once a customer places an order on app, the algorithm notifies the nearest dark store. In the dark store, this sets off the process of the picker finding the items as ordered and packing them within a few minutes with minimum error. Then, the order is transferred to the delivery boy waiting outside the dark store, who transports it to its final destination - the customer. The delivery boys are hired on gig-model but most of the employees in dark stores are hired through HRCs. Given the high attrition of workers, the role of the intermediary becomes crucial in sustaining a constant supply of pickers and packers. Drawing on extensive ethnographic data, the study provides nuanced insights into the intermediation processes and the functions of the intermediary in making the labour available as an on-tap service – liquifying labour force. We also argue that, in a context where comprehending labour market is hampered by incongruent methods and definitional discordance in employment datasets, taking the vantage point of one intermediary and ethnography enables answering our questions on what, how and why labour market intermediation thrives in India and, thereby, draw insights on the country's dynamic labour market. The fine-grained data on the intermediation process validate the impossibility of studying this new trend any other way.

ACCEPTED
Format
02 - Digitalization and Public Policy Implementation: An Empirical Study of a Village in Varanasi, Uttar Pradesh
suchit Yadav (NLU Prayagraj )
CO-AUTHORS
—
ABSTRACT

Abstract

In the contemporary welfare state, public policy functions as a key instrument for addressing social inequality and uplifting marginalized sections of society. In India, successive governments have introduced a range of targeted welfare policies in sectors such as health, housing, food security, and education. In recent years, digital technologies have become central to the implementation of these policies, reshaping the relationship between the state and its beneficiaries. While digitalization is often presented as a means to enhance efficiency, transparency, and inclusion, its uneven spread has produced new forms of exclusion. Due to the persistent digital divide, significant sections of society are adversely affected by the digitalisation of welfare delivery. This paper examines the everyday experiences of villagers in accessing and engaging with digitally mediated public policies, highlighting the practical challenges and social barriers they encounter. Many beneficiaries lack access to essential digital infrastructure such as smartphones and stable internet connectivity, or the digital literacy required to navigate these systems. Based on an empirical study conducted in a village in Varanasi, Uttar Pradesh, the paper argues that public policy in India needs to reconceptualise their approach. It emphasises the need for policymakers to reconsider their orientation towards policy subjects, modes of implementation, and the broader assumptions underlying digital governance in order to ensure that digitalisation contributes to social inclusion rather than reinforcing existing inequalities.

Keywords – public police, welfare state, digitalisation, Inclusion,

ACCEPTED
Format
03 - Examining the promise of realist methods and policy relevant evaluations
Pragati Hebbar (University of Maastricht)
CO-AUTHORS
—
ABSTRACT

The traditional quantitative indicator dominated evaluations and surveys seemed to be hitting a dead-end for the system actors at the state and national level in the area of tobacco control. Numbers do not capture the complexity of the phenomena of program implementation and do not explain the ‘why’ aspect. In the case of tobacco control although the national program for tobacco control (NTCP) 2008 and The Cigarettes and Other Tobacco Products Act (COTPA) 2003 have been in effect since two decades the implementation has been varied across India.

 

We used realist research to explain the implementation of tobacco control in India. Eliciting initial program theories is central to realist methods and we conducted a realist review to articulate how tobacco control policies work. Following the literature review a realist evaluation was conducted choosing three Indian states with diverse implementation outcomes of COTPA measured using proxy implementation indicators. Using these contrasting case studies the initial program theory was refined to explain how and for whom tobacco control policies work.

 

Through this paper we reflect on the application of realist methods in India over the past few years when it was sparsely applied. One of the upfront challenges is the expectations that the method raises for decision makers due to the framing as ‘what works for whom and why under what circumstances?’ Hence understanding and setting expectations with the policy actors is an important preliminary step guiding the research cycle and further enquiry.

 

Using an implementation research approach to the enquiry ensured stakeholder engagement at each stage of the research from developing the research question to dissemination of the findings. Regional consultations engaging stakeholders from 20 Indian states helped test and refine the theories that were being developed as part of this research in a manner that gave them an open space to share their challenges and successes. This significantly added to the depth of explanations.

 

Traditional evaluation methods fail to capture complexity which is inherent in social programs and often produce simplistic answers to causal questions. Realist methods embrace this complexity and produce context specific causal pathways thereby enhancing usability of findings. This also makes evaluations of the kind resource and time intensive along with high costs of building capacity within the team.

 

The lack of worked examples and expensive training opportunities further reduces the uptake of such approaches since these methods are not prescriptive and can be used in a variety of ways.

These gaps are being bridged by various informal networks and Communities of Practice which seek to make these methods more accessible and focus on teaching the application of them especially to novice researchers.

 

This practice paper will explore the use of realist methods to inform policy relevant questions through extensive stakeholder engagement and building trust with bureaucrats and implementers through engagement across the research cycle. While explaining the challenges and benefits of applying realist methods we examine critically the promise this theory driven enquiry holds for tackling complex health policy and systems research questions.

CO-AUTHORS
—
ABSTRACT

 
The labour contracting landscape in India had relied mainly on the informal individual labour contractor (jobber, thekedaars, maistries, kangaanies and so on) operating in their little geographies to source workers. This has changed post-2000 with the entry of manpower supply companies, temping companies, flexi-staffing industry, or human resource companies (HRC). Although the latter entity can be seen as being evolved from the former to become more formal, in the face of it, they both are performing the same task – recruiting casual labour or meeting the supply-demand gaps and ensuring a good fit of required skills. In doing so, they smoothen the volatility of labour force requirements in industries prone to the vagaries of global demand trends. In this process of brokering, the labour market intermediaries reduce various transaction costs such as search, verification, monitoring, compliance. To understand how the HRC addresses these costs, we do a workplace ethnography working in an HRC for eight-month long period understanding their processes. Methods such as interviews, participant observation and unstructured interviews were employed to collect the relevant data over and above the daily field notes.
After studying the organization structure closely, we realized that within the workforce deployed by Wergo, staffing business of unskilled and semi-skilled labour deployment (referred to as generalized staffing) made up 90% of their revenue base. Although they supply workers to various sectors including banking and financial services, telecom and technology and consumer durables, we study their retail, ecommerce and quick commerce (RECom) sector to understand the process of staffing. The RECom, primarily the q-commerce industry instantly delivered daily groceries, vegetables, and fruits to customers' doorsteps. To ensure quick delivery, they have various warehouses and hubs spread out through the cities called as ‘dark-stores’. Once a customer places an order on app, the algorithm notifies the nearest dark store. In the dark store, this sets off the process of the picker finding the items as ordered and packing them within a few minutes with minimum error. Then, the order is transferred to the delivery boy waiting outside the dark store, who transports it to its final destination - the customer. The delivery boys are hired on gig-model but most of the employees in dark stores are hired through HRCs. Given the high attrition of workers, the role of the intermediary becomes crucial in sustaining a constant supply of pickers and packers. Drawing on extensive ethnographic data, the study provides nuanced insights into the intermediation processes and the functions of the intermediary in making the labour available as an on-tap service – liquifying labour force. We also argue that, in a context where comprehending labour market is hampered by incongruent methods and definitional discordance in employment datasets, taking the vantage point of one intermediary and ethnography enables answering our questions on what, how and why labour market intermediation thrives in India and, thereby, draw insights on the country's dynamic labour market. The fine-grained data on the intermediation process validate the impossibility of studying this new trend any other way.

CO-AUTHORS
—
ABSTRACT

Abstract

In the contemporary welfare state, public policy functions as a key instrument for addressing social inequality and uplifting marginalized sections of society. In India, successive governments have introduced a range of targeted welfare policies in sectors such as health, housing, food security, and education. In recent years, digital technologies have become central to the implementation of these policies, reshaping the relationship between the state and its beneficiaries. While digitalization is often presented as a means to enhance efficiency, transparency, and inclusion, its uneven spread has produced new forms of exclusion. Due to the persistent digital divide, significant sections of society are adversely affected by the digitalisation of welfare delivery. This paper examines the everyday experiences of villagers in accessing and engaging with digitally mediated public policies, highlighting the practical challenges and social barriers they encounter. Many beneficiaries lack access to essential digital infrastructure such as smartphones and stable internet connectivity, or the digital literacy required to navigate these systems. Based on an empirical study conducted in a village in Varanasi, Uttar Pradesh, the paper argues that public policy in India needs to reconceptualise their approach. It emphasises the need for policymakers to reconsider their orientation towards policy subjects, modes of implementation, and the broader assumptions underlying digital governance in order to ensure that digitalisation contributes to social inclusion rather than reinforcing existing inequalities.

Keywords – public police, welfare state, digitalisation, Inclusion,

CO-AUTHORS
—
ABSTRACT

The traditional quantitative indicator dominated evaluations and surveys seemed to be hitting a dead-end for the system actors at the state and national level in the area of tobacco control. Numbers do not capture the complexity of the phenomena of program implementation and do not explain the ‘why’ aspect. In the case of tobacco control although the national program for tobacco control (NTCP) 2008 and The Cigarettes and Other Tobacco Products Act (COTPA) 2003 have been in effect since two decades the implementation has been varied across India.

 

We used realist research to explain the implementation of tobacco control in India. Eliciting initial program theories is central to realist methods and we conducted a realist review to articulate how tobacco control policies work. Following the literature review a realist evaluation was conducted choosing three Indian states with diverse implementation outcomes of COTPA measured using proxy implementation indicators. Using these contrasting case studies the initial program theory was refined to explain how and for whom tobacco control policies work.

 

Through this paper we reflect on the application of realist methods in India over the past few years when it was sparsely applied. One of the upfront challenges is the expectations that the method raises for decision makers due to the framing as ‘what works for whom and why under what circumstances?’ Hence understanding and setting expectations with the policy actors is an important preliminary step guiding the research cycle and further enquiry.

 

Using an implementation research approach to the enquiry ensured stakeholder engagement at each stage of the research from developing the research question to dissemination of the findings. Regional consultations engaging stakeholders from 20 Indian states helped test and refine the theories that were being developed as part of this research in a manner that gave them an open space to share their challenges and successes. This significantly added to the depth of explanations.

 

Traditional evaluation methods fail to capture complexity which is inherent in social programs and often produce simplistic answers to causal questions. Realist methods embrace this complexity and produce context specific causal pathways thereby enhancing usability of findings. This also makes evaluations of the kind resource and time intensive along with high costs of building capacity within the team.

 

The lack of worked examples and expensive training opportunities further reduces the uptake of such approaches since these methods are not prescriptive and can be used in a variety of ways.

These gaps are being bridged by various informal networks and Communities of Practice which seek to make these methods more accessible and focus on teaching the application of them especially to novice researchers.

 

This practice paper will explore the use of realist methods to inform policy relevant questions through extensive stakeholder engagement and building trust with bureaucrats and implementers through engagement across the research cycle. While explaining the challenges and benefits of applying realist methods we examine critically the promise this theory driven enquiry holds for tackling complex health policy and systems research questions.