Format

P05. Integration of Data and Evidence from a Range of Sources to Inform Policy

PUBLIC PROBLEM AND AGENDA SETTING
PANEL CHAIR(S)
Elizabeth Burgess-Pinto
MacEwan University
Main chair
Navjot Lamba
PolicyWise for Children & Youth
Second chair
CATEGORISATION
POLICY TOPIC
PUBLIC PROBLEM AND AGENDA SETTING
SECTOR
—
KEYWORDS
—
GENERAL OBJECTIVES, RESEARCH QUESTIONS AND SCIENTIFIC RELEVANCE

Increasingly, policymakers, researchers, and planners are recognizing the interconnectedness of social needs at the population level. Accordingly, there is increasing demand for policies that require input from multiple government as well as public sectors. This panel will explore the potential of integrating data and evidence produced in the academic domain and government sectors to inform the policy outcomes of populations at risk of inequities such as children, women, immigrants/refugees, and indigenous groups.  Linked administrative data can provide insight into the patterns of government use across sectors; however, these data do not address root causes.  Evidence from academic research can help tease out root causes.  Together, the two sources of data can inform and support policy changes. The following examples illustrate the potential

 

In one Canadian province, data from the Ministry of Health and the Ministry of Children’s Services were linked to inform the mental health service use patterns of children in care. The policy reform (and funding) resulting from these findings allowed children and youth in care to access mental health services within weeks of being assessed as opposed to months and years. The value of integrating data at the government service level is access to population data. However, the details of the root causes leading to mental health service use among this population is less well understood.  This is where academic research on use of mental health services and root causes could enhance analysis and decision making. 

 

In the Province of Ontario, the McMaster Health Forum (2016) held public citizen meetings with a view to exploring the integration of person-level data across sectors and reviewed examples of data integration from several countries.  It was found that an integrated approach to analysis of data can produce tangible benefits for populations that are often termed “vulnerable” or “high priority” but an overarching policy framework is essential to ensure successful data integration is sustainable over time. 

 

 

CALL FOR PAPERS

This panel seeks papers that both theoretically and empirically examine, compare, and critically interrogate approaches to data integration. Submissions that are in the process of data and evidence integration or have considered the value of doing so are invited.  We are interested in papers that address the following questions:  What is the impact of the intersection of academic and government administrative data and evidence on policy for populations at risk for inequities?  What is the value of co-creation of knowledge from different ways of knowing in the development of policies, for example Indigenous and Western knowledge?  What are the best approaches for integrating data from various sources across sectors?  Are there frameworks that would enhance sustainability of these processes?

 

 

ACCEPTED
Format
01 - Legislative and collaborative approaches to improve data access in the US
Amy O'Hara (Georgetown University)
CO-AUTHORS
—
ABSTRACT

US researchers use linked administrative data sources to study economic outcomes, effects of policy changes, and longitudinal trends.  Examples of such studies are highlighted, including research on intergenerational mobility, food security, racial fluidity, access to higher education, and the opioid epidemic.  These studies face the privacy, security and sensitivity challenges of using person-level, population-wide data.  Such research is constrained by misaligned incentives between government and proprietary data holders and analysts to use data for policy development and evaluation.  There are perceived and actual legal barriers and security risks surrounding data access.  The laws, methods, and platforms currently enabling administrative data research are described. 

 

Efforts are underway to improve governance models and data access to produce robust, timely, replicable evidence for policymakers in the US.  From 2016 to 2017, the Commission for Evidence-based Policymaking gathered input from the public and private sectors to determine whether and how to use administrative data to build evidence.  The commission proposed 21 recommendations within the following major themes: (1) Improve access to data for program administrators and researchers, including linkages across datasets; (2) Stronger privacy protections, including greater uniformity across government and development of methods that address the changing risks associated with the ever-growing volume of data available; and (3) Increasing capacity to produce evidence, including establishing a National Secure Data Service.  To begin implementing these recommendations, the Foundations for Evidence-based Policymaking Act of 2017 (PL 115-435) was passed in January 2019.  This law’s requirements for federal agencies will be discussed in the context of an emerging US Federal Data Strategy. 

 

Beyond legally mandated (but unfunded) activities at the federal level, there are initiatives to promote the secure and responsible use of administrative data for research and policymaking in state and local government agencies.  States and counties are building integrated data systems, and partnerships with academic and non-profit teams support use of the data.  The Administrative Data Research Initiative (ADRI) aims to connect these efforts by identifying and addressing data access issues, developing reusable governance approaches, templates, toolkits, and training.  Improving data capacity in federal, state, and local governments aligns with parallel efforts in the social sector.  ADRI sits within the McCourt School of Public Policy at Georgetown University, taking advantage of its location in the US capital to network with federal agencies, think tanks, and national organizations representing specific populations and jurisdictions.  This talk outlines ADRI’s efforts to strengthen the network of data intermediaries across various domains including labor, housing, health, justice, innovation, and education. 

ACCEPTED
Format
02 - The politics of administrative data: interpreting demand for special educational needs and disability provision in the South West of England.
Hannah Durrant (Wales Centre for Public Policy, Cardiff University)
CO-AUTHORS
Emily Rempel (University of Bath)
Julie Barnett (University of Bath)
ABSTRACT

This paper examines the relationship between how data are collected, assembled, analysed and interpreted, and what this means for what data can represent and signify. It presents the case of Special Educational Needs and Disability (SEND) in a Local Authority area in the South West of England. It is grounded in approaches to the study of evidence informed policymaking that are concerned with what counts as evidence and the processes by which evidence is aligned in political efforts to frame problems and solutions (Cairney, 2016; Parkhurst, 2017). Integrated data, as one form of evidence for policymaking, has attained a privileged position in policy discourse, as a result of expansive claims made for its potential (Shakespeare, 2013). In particular, its potential to reveal and predict social need at scale, refine policy options and evaluate implementation (Höchtl et al., 2015).

 

In this context, and in response to local policymakers calls for data-derived insights to inform policy, we collaborated on a project with a Local Authority in the South West of England to understand rising demand for SEND provision (2016-2018). Tracking educational need at the population level is a priority for local governments with a statutory duty to meet these needs, plan and commission services, ensure equitable access to appropriate provision and lessen inequalities between groups.

 

The project involved integration and analysis of educational data, generated by a range of local providers. Initial analysis was conducted by the Local Authority and the findings were discussed at a workshop attended by data analysts, education service and commissioning managers. This led to the development of 6 hypothesis to explain rising demand for SEND. Our role was to conduct further analysis; integrating evidence and interpretative capacity to test these.

 

Our findings reveal the significance of education policy and practice for defining data collection and storage architectures and the extent to which this delimits and skews what data analytics can reveal. We align with critical data studies scholars in recognising that the frameworks by which data is produced, curated, manipulated and made sense of, determine what is counted and what can be revealed (Iliadis & Russo, 2016; boyd & Crawford, 2012). This matters in policy contexts, particularly where it determines the distribution of resources. The intersection of data and evidence and the collaboration between policymakers and academic researchers was critical to understanding and revealing the politics of administrative data in this case.

 

We supplemented our analysis of data with semi-structured interviews with policy officials to understand how data-derived insight is taken up and used in policy and commissioning decisions. Our findings revealed that SEND were only increasing when the population was viewed from a high need/high cost perspective. Indeed, this policy framing obscured and reinforced significant counter trends among lower need/lower cost groups. We reflect on the decision not to involve education practitioners and families in data practices and deliberative processes to understand patterns of service demand, and the implications for the privileging of certain ways of knowing over others in policymaking.

 

References:

 

boyd, D., & Crawford, K. (2012). Critical questions for big data. Information, Communication & Society, 15(5), 662–679.

 

Cairney, P. (2016). The politics of evidence-based policymaking. London: Palgrave Pivot

 

Höchtl, J., Parycek, P., & Schöllhammer, R. (2015). Big data in the policy cycle: Policy decision making in the digital era. Journal of Organizational Computing and Electronic Commerce, 26(1/2), 147–169.

 

Iliadis, A., & Russo, F. (2016). Critical data studies: An introduction. Big Data & Society, 3(2), 1–7.

 

Parkhurst, J. (2017). The politics of evidence: from evidence based policy to the good governance of evidence. Abingdon: Routledge.

 

Shakespeare, S. (2013). Shakespeare review: An independent review of public sector information. London: Department for Business, Innovation and Skills. Retrieved from https://www.gov.uk/government/publications/shakespeare-review-of-public-sector-information

 

ACCEPTED
Format
03 - Indebtedness and deprivation of basic needs: Using a national survey and public service user interviews
Jung Min Park (Seoul National University)
CO-AUTHORS
—
ABSTRACT

This study examines the relationship of personal debt with the degree of deprivation of basic needs in the areas of material hardship and health.
Data for the analyses was drawn from 3-10 waves of the Korea Welfare Panel Study, collected from 2008 to 2015 for a nationally representative sample. The degree of deprivation was estimated using the fuzzy set theory. Another set of data were collected annually from 2016 to 2018 for a sample of individuals who used publicly-funded debt relief programs (e.g., personal bankruptcy). The sample, 209 individuals, was recruited from the Bankruptcy Court and government-funded credit counseling and recovery agencies in South Korea. Measures of material hardship focused on individuals’ ability to meet basic needs, including items stating as to whether the household did not pay the full amount of the rent or was late with a payment because it could not afford to pay, the household was not able to pay the full amount of the gas, water, or electricity bills, The household was food insecure, and the family had problems paying medical bills. Measures of health included indicators of overall health status, chronic illness, and mental health.  
Major results are as follows: First, the low-income group had the debt-to-income ratio almost 2 times higher and the proportion of subprime loan 2-6 times higher, compared to the middle- and high-income groups. Second, among the lower income individuals (income deciles 1-3), the degree of deprivation in terms of material hardship and housing was much higher for those with indebtedness. Such differences were not observed for the middle- and high-income groups.
The debt relief program users group demonstrated a substantially higher level of material hardship and a lower level of subjective well-being compared to the general and low income populations. In wave 1, the proportion of people who suffered from hunger was 17.9% in the debtors group compared to 1.6% in the low income group from a national dataset. The proportion of those who had had suicidal ideation was 57.9% in the debtors group compared to 19.2% in the low income group and 2.7% in the general population.
The results suggest that a combination of low-income and indebtedness increases the risk of deprivation in basic needs, over-indebtedness adversely affects individuals’ well-being, and that there is a need for coordinating social safety net programs and financial counseling, management, and recovery services.

CO-AUTHORS
—
ABSTRACT

US researchers use linked administrative data sources to study economic outcomes, effects of policy changes, and longitudinal trends.  Examples of such studies are highlighted, including research on intergenerational mobility, food security, racial fluidity, access to higher education, and the opioid epidemic.  These studies face the privacy, security and sensitivity challenges of using person-level, population-wide data.  Such research is constrained by misaligned incentives between government and proprietary data holders and analysts to use data for policy development and evaluation.  There are perceived and actual legal barriers and security risks surrounding data access.  The laws, methods, and platforms currently enabling administrative data research are described. 

 

Efforts are underway to improve governance models and data access to produce robust, timely, replicable evidence for policymakers in the US.  From 2016 to 2017, the Commission for Evidence-based Policymaking gathered input from the public and private sectors to determine whether and how to use administrative data to build evidence.  The commission proposed 21 recommendations within the following major themes: (1) Improve access to data for program administrators and researchers, including linkages across datasets; (2) Stronger privacy protections, including greater uniformity across government and development of methods that address the changing risks associated with the ever-growing volume of data available; and (3) Increasing capacity to produce evidence, including establishing a National Secure Data Service.  To begin implementing these recommendations, the Foundations for Evidence-based Policymaking Act of 2017 (PL 115-435) was passed in January 2019.  This law’s requirements for federal agencies will be discussed in the context of an emerging US Federal Data Strategy. 

 

Beyond legally mandated (but unfunded) activities at the federal level, there are initiatives to promote the secure and responsible use of administrative data for research and policymaking in state and local government agencies.  States and counties are building integrated data systems, and partnerships with academic and non-profit teams support use of the data.  The Administrative Data Research Initiative (ADRI) aims to connect these efforts by identifying and addressing data access issues, developing reusable governance approaches, templates, toolkits, and training.  Improving data capacity in federal, state, and local governments aligns with parallel efforts in the social sector.  ADRI sits within the McCourt School of Public Policy at Georgetown University, taking advantage of its location in the US capital to network with federal agencies, think tanks, and national organizations representing specific populations and jurisdictions.  This talk outlines ADRI’s efforts to strengthen the network of data intermediaries across various domains including labor, housing, health, justice, innovation, and education. 

CO-AUTHORS
Emily Rempel (University of Bath)
Julie Barnett (University of Bath)
ABSTRACT

This paper examines the relationship between how data are collected, assembled, analysed and interpreted, and what this means for what data can represent and signify. It presents the case of Special Educational Needs and Disability (SEND) in a Local Authority area in the South West of England. It is grounded in approaches to the study of evidence informed policymaking that are concerned with what counts as evidence and the processes by which evidence is aligned in political efforts to frame problems and solutions (Cairney, 2016; Parkhurst, 2017). Integrated data, as one form of evidence for policymaking, has attained a privileged position in policy discourse, as a result of expansive claims made for its potential (Shakespeare, 2013). In particular, its potential to reveal and predict social need at scale, refine policy options and evaluate implementation (Höchtl et al., 2015).

 

In this context, and in response to local policymakers calls for data-derived insights to inform policy, we collaborated on a project with a Local Authority in the South West of England to understand rising demand for SEND provision (2016-2018). Tracking educational need at the population level is a priority for local governments with a statutory duty to meet these needs, plan and commission services, ensure equitable access to appropriate provision and lessen inequalities between groups.

 

The project involved integration and analysis of educational data, generated by a range of local providers. Initial analysis was conducted by the Local Authority and the findings were discussed at a workshop attended by data analysts, education service and commissioning managers. This led to the development of 6 hypothesis to explain rising demand for SEND. Our role was to conduct further analysis; integrating evidence and interpretative capacity to test these.

 

Our findings reveal the significance of education policy and practice for defining data collection and storage architectures and the extent to which this delimits and skews what data analytics can reveal. We align with critical data studies scholars in recognising that the frameworks by which data is produced, curated, manipulated and made sense of, determine what is counted and what can be revealed (Iliadis & Russo, 2016; boyd & Crawford, 2012). This matters in policy contexts, particularly where it determines the distribution of resources. The intersection of data and evidence and the collaboration between policymakers and academic researchers was critical to understanding and revealing the politics of administrative data in this case.

 

We supplemented our analysis of data with semi-structured interviews with policy officials to understand how data-derived insight is taken up and used in policy and commissioning decisions. Our findings revealed that SEND were only increasing when the population was viewed from a high need/high cost perspective. Indeed, this policy framing obscured and reinforced significant counter trends among lower need/lower cost groups. We reflect on the decision not to involve education practitioners and families in data practices and deliberative processes to understand patterns of service demand, and the implications for the privileging of certain ways of knowing over others in policymaking.

 

References:

 

boyd, D., & Crawford, K. (2012). Critical questions for big data. Information, Communication & Society, 15(5), 662–679.

 

Cairney, P. (2016). The politics of evidence-based policymaking. London: Palgrave Pivot

 

Höchtl, J., Parycek, P., & Schöllhammer, R. (2015). Big data in the policy cycle: Policy decision making in the digital era. Journal of Organizational Computing and Electronic Commerce, 26(1/2), 147–169.

 

Iliadis, A., & Russo, F. (2016). Critical data studies: An introduction. Big Data & Society, 3(2), 1–7.

 

Parkhurst, J. (2017). The politics of evidence: from evidence based policy to the good governance of evidence. Abingdon: Routledge.

 

Shakespeare, S. (2013). Shakespeare review: An independent review of public sector information. London: Department for Business, Innovation and Skills. Retrieved from https://www.gov.uk/government/publications/shakespeare-review-of-public-sector-information

 

CO-AUTHORS
—
ABSTRACT

This study examines the relationship of personal debt with the degree of deprivation of basic needs in the areas of material hardship and health.
Data for the analyses was drawn from 3-10 waves of the Korea Welfare Panel Study, collected from 2008 to 2015 for a nationally representative sample. The degree of deprivation was estimated using the fuzzy set theory. Another set of data were collected annually from 2016 to 2018 for a sample of individuals who used publicly-funded debt relief programs (e.g., personal bankruptcy). The sample, 209 individuals, was recruited from the Bankruptcy Court and government-funded credit counseling and recovery agencies in South Korea. Measures of material hardship focused on individuals’ ability to meet basic needs, including items stating as to whether the household did not pay the full amount of the rent or was late with a payment because it could not afford to pay, the household was not able to pay the full amount of the gas, water, or electricity bills, The household was food insecure, and the family had problems paying medical bills. Measures of health included indicators of overall health status, chronic illness, and mental health.  
Major results are as follows: First, the low-income group had the debt-to-income ratio almost 2 times higher and the proportion of subprime loan 2-6 times higher, compared to the middle- and high-income groups. Second, among the lower income individuals (income deciles 1-3), the degree of deprivation in terms of material hardship and housing was much higher for those with indebtedness. Such differences were not observed for the middle- and high-income groups.
The debt relief program users group demonstrated a substantially higher level of material hardship and a lower level of subjective well-being compared to the general and low income populations. In wave 1, the proportion of people who suffered from hunger was 17.9% in the debtors group compared to 1.6% in the low income group from a national dataset. The proportion of those who had had suicidal ideation was 57.9% in the debtors group compared to 19.2% in the low income group and 2.7% in the general population.
The results suggest that a combination of low-income and indebtedness increases the risk of deprivation in basic needs, over-indebtedness adversely affects individuals’ well-being, and that there is a need for coordinating social safety net programs and financial counseling, management, and recovery services.