LIST OF WORKSHOPS

Machine learning (ML) is a branch of artificial intelligence that allows computers to learn from data and improve their outputs/performance without being explicitly programmed. Natural language processing (NLP) leverages ML modelling and enables computers to understand and attempt to make sense of the human language. Both ML and NLP have become powerful tools in social sciences and in policy analysis more specifically. They are capable of efficiently navigating through and extracting insight and knowledge from large datasets. This goes hand in hand with abundance of data (e.g., text) that are helpful or in some cases crucial to explain social and policy phenomena.

This workshop builds on the premise ML and NLP as computational tools have the potential to enhance methodological grounding of quantitative public policy analysis (see Grimmer 2015; Mason et al. 2014). This premise builds on ML’s overall focus on prediction and its inductive-flexible modelling that entails a more lenient approach to both technical assumptions and interpretation of models (see Bryman et al. 2008). As such, ML arguably (to an extent) addresses drawbacks (e.g., focus on p-values and statistical significance; replication crisis; publication bias; statistically underpowered hypotheses) of prevalent quantitative research that utilises linear models as a benchmark (see Schrodt 2014).

Although usage of ML and NLP in social sciences and in public policy is relatively novel, innovative and compelling empirical applications have been on the rise. The applications stem from quantitative text analyses including opinion mining, thematic analysis, topic modelling (see Yantseva and Kucher 2021); to informing public policy issues such as crime rates (see Reier Forradellas et al. 2020) and providing insight on online governmental services (see Alexopoulos et al. 2019). In addition, there are compelling investigations of potential of NLP and large language models (LLMs) for policy ranging from policy evaluation to production of policy briefs (see Singh et al. 2024; Safaei and Longo 2024). Overall, the applications have demonstrated methodological power of ML and NLP as well as their potential to extend and build on prevalent quantitative and qualitative techniques.

Against this backdrop, the workshop aims to demonstrate the potential of ML and NLP techniques as applied to public policy concerns. In addition, the workshop aims to showcase synergies between ML approach to modelling data and established techniques for quantitative public policy analysis (e.g., experiments, linear modelling, panel data analysis, time series forecasting, cost-benefit analysis, etc.).

The following research questions broadly guide and inform the workshop’s discussion:

1.    To what extent and for what purpose does public policy utilise ML and NLP (LLMs)?

2.    How do ML and NLP enhance policy outcomes as well as impact?

3.    To what extent ML and NLP modelling enhance the existing methodological grounding of quantitative public policy?

4.    How can ML and NLP work in synergy with other methodological approaches and techniques?

5.    What are best practices in applying ML and NLP for public policy analysis?

6.    What are the ongoing challenges (e.g., ethical concerns) regarding ML and NLP applications in public policy and how to address them?

Building on the above, regarding ML, we invite contributions focusing on:

·      Empirical demonstrations of supervised (classification, prediction) and/or unsupervised (pattern recognition) ML applied to public policy issues; all public policy fields/sectors, geographical contexts as well as levels of analysis are welcome

·      Utility and limitations of ML modelling in analyses of public policies

·      Synergies between ML and established/conventional quantitative and qualitative techniques

·      Review of and critical assessment on applied ML for public policy including reflections on the current and future methodological setup of quantitative public policy

Regarding NLP more specifically, we are particularly interested in papers that address one or more of the following themes:

·      Types of textual data used for policy analysis (e.g., legislative texts, regulations, policy briefs, political manifestos, speeches, and parliamentary debates) including reflections how these datasets are processed, the challenges of working with complex legal or political language, and the potential for expanding the range of data used in policy analysis

·      Demonstration and/or evaluation of different methodologies for analysing policy-related data including text classification, topic modelling, sentiment analysis, causal mapping, predictive analytics, and the use of LLMs for advanced language tasks; authors are encouraged to address limitations and ethical concerns, such as biases within models, transparency, interpretability, and the challenges of deploying LLMs in policy contexts.

·      Practical applications of NLP and LLMs in policymaking (e.g., automated production of policy briefs using LLMs to summarize legislative texts, real-time monitoring and evaluation of policy implementation, or predictive modelling to forecast policy outcomes, etc.) and their role in informing policy decisions, improving policy design and enhancing the understanding of complex policy issues; authors are also encouraged to critically assess to what extent can NLP and LLM technologies replace or complement human analysts and policymakers

Overall, we encourage contributions from both researchers and practitioners who are exploring the intersection of ML, NLP (LLMs), and public policy. The above lists of topics for contributions are not exhaustive. Submissions may present empirical studies, methodological innovations, case studies, or theoretical (or critical) reflections. The workshop aims to foster an interdisciplinary dialogue on how these technologies are reshaping the field of public policy, the practical challenges they face, and the opportunities for further development.

References:

Alexopoulos, Charalampos, Zoi Lachana, Aggeliki Androutsopoulou, Vasiliki Diamantopoulou, Yannis Charalabidis, and Michalis Avgerinos Loutsaris. 2019. ‘How Machine Learning Is Changing E-Government’. Proceedings of the 12th International Conference on Theory and Practice of Electronic Governance, April 3, 354–63. https://doi.org/10.1145/3326365.3326412.

Bryman, Alan, Saul Becker, and Joe Sempik. 2008. ‘Quality Criteria for Quantitative, Qualitative and Mixed Methods Research: A View from Social Policy’. International Journal of Social Research Methodology 11 (4): 261–76. https://doi.org/10.1080/13645570701401644.

Grimmer, Justin. 2015. ‘We Are All Social Scientists Now: How Big Data, Machine Learning, and Causal Inference Work Together’. PS: Political Science & Politics 48 (01): 80–83. https://doi.org/10.1017/S1049096514001784.

Mason, Winter, Jennifer Wortman Vaughan, and Hanna Wallach. 2014. ‘Computational Social Science and Social Computing’. Machine Learning 95 (3): 257–60. https://doi.org/10.1007/s10994-013-5426-8.

Reier Forradellas, Ricardo Francisco, Sergio Luis Náñez Alonso, Javier Jorge-Vazquez, and Marcela Laura Rodriguez. 2020. ‘Applied Machine Learning in Social Sciences: Neural Networks and Crime Prediction’. Social Sciences 10 (1): 4. https://doi.org/10.3390/socsci10010004.

Safaei, Mehrdad, and Justin Longo. 2024. ‘The End of the Policy Analyst? Testing the Capability of Artificial Intelligence to Generate Plausible, Persuasive, and Useful Policy Analysis’. Digit. Gov.: Res. Pract. 5 (1): 4:1-4:35. https://doi.org/10.1145/3604570.

Schrodt, Philip A. 2014. ‘Seven Deadly Sins of Contemporary Quantitative Political Analysis’. Journal of Peace Research 51 (2): 287–300. https://doi.org/10.1177/0022343313499597.

Singh, Chandan, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. 2024. ‘Rethinking Interpretability in the Era of Large Language Models’. arXiv:2402.01761. Preprint, arXiv, January 30. https://doi.org/10.48550/arXiv.2402.01761.

Yantseva, Victoria, and Kostiantyn Kucher. 2021. ‘Machine Learning for Social Sciences: Stance Classification of User Messages on a Migrant-Critical Discussion Forum’. 2021 Swedish Workshop on Data Science (SweDS), December 2, 1–8. https://doi.org/10.1109/SweDS53855.2021.9637718.

This workshop builds on the premise that machine learning (ML) and natural language processing (NLP) as computational tools have the potential to enhance methodological grounding of quantitative public policy analysis. It aims to demonstrate the potential of ML and NLP techniques as applied to public policy concerns. In addition, it aims to showcase synergies between ML approach to modelling data and established techniques for quantitative public policy analysis (e.g., experiments, linear modelling, panel data analysis, time series forecasting, cost-benefit analysis, etc.).

Building on this, regarding ML we invite contributions focusing on:

·      Empirical demonstrations of supervised (classification, prediction) and/or unsupervised (pattern recognition) machine learning applied to public policy issues; all policy realms, geographical context as well as levels of analysis are welcome

·      Utility and limitations of machine learning modelling in analyses of public policies

·      Synergies between machine learning and established/conventional quantitative and qualitative techniques

·      Review of machine learning practices and applications in public policy as well as critical reflections on the current and future methodological setup of quantitative public policy

Regarding NLP more specifically, we are particularly interested in papers that address one or more of the following themes:

·      Types of textual data used for policy analysis (e.g., legislative texts, regulations, policy briefs, political manifestos, speeches, and parliamentary debates) including reflections how these datasets are processed, the challenges of working with complex legal or political language, and the potential for expanding the range of data used in policy analysis

·      Evaluation of different methodologies for analysing policy-related data including text classification, topic modelling, sentiment analysis, causal mapping, predictive analytics, and the use of LLMs for advanced language tasks; authors are encouraged to address limitations and ethical concerns, such as biases within models, transparency, interpretability, and the challenges of deploying ML and LLMs in policy contexts.

·      Practical applications of NLP and LLMs in policymaking (e.g., automated production of policy briefs using LLMs to summarize legislative texts, real-time monitoring and evaluation of policy implementation, or predictive modeling to forecast policy outcomes, etc.) and their role in informing policy decisions, improving policy design and enhancing the understanding of complex policy issues; authors are also encouraged to critically assess to what extent can NLP and LLM technologies replace or complement human analysts and policymakers

Overall, we encourage contributions from both researchers and practitioners who are exploring the intersection of ML, NLP (LLMs), and public policy. Submissions may present empirical studies, methodological innovations, case studies, or theoretical (or critical) reflections.

This workshop explores the growing significance of Machine Learning (ML) and Natural Language Processing (NLP), including Large Language Models (LLMs), as computational tools for enhancing quantitative public policy analysis. ML and NLP offer a departure from traditional linear models by focusing on prediction and employing inductive-flexible modelling approaches, which can address some limitations of prevalent quantitative political science research.  ML and NLP are already being applied in social sciences and public policy, inter alia focusing on quantitative text analyses like opinion mining and topic modelling, policy evaluation, production of policy briefs, predicting crime rates. The workshop aims to demonstrate the utility of ML and NLP in public policy and explore their synergy with established quantitative and qualitative analysis techniques. An overarching research questions that guides the workshop focuses on the extent and purpose of ML and NLP utilization in analyses of public policy and their potential impact on policy outcomes and decision-making.

The workshop welcomes contributions from academics and practitioners. The submissions may examine empirical demonstrations of supervised and unsupervised ML applied to policy issues; discussions on the utility and limitations of ML for public policy; types of textual data used in NLP for policy analysis; evaluation of various NLP methodologies (e.g., text classification, sentiment analysis, causal mapping); practical applications of NLP in public policy. The workshop encourages and aims to foster interdisciplinary dialogue on how these technologies are reshaping public policy, addressing practical challenges, and exploring future opportunities.

Igor Tkalec is Assistant Professor (Lecturer) in social data science at UCL's Social Data Institute (SODA) with a background in political economy. Prior to joining UCL, Igor was a research fellow at the European University Institute in Florence working on the GlobalStat project. Igor's research interests intersect EU socio-economic governance, sustainability and applied data science. His current research focuses on retirement policy, data storytelling, and the utility of applied machine learning models for answering social issues. He is co-editing 'The Elgar Companion to Data and Indicators for the Sustainable Development Goals' (forthcoming 2026) and developing a monograph on “Navigating the Wicked Problem of Pensions in the EU: the Role of Pension Funds”.

Antoine Lemor is a postdoctoral fellow jointly affiliated with the Réseau francophone international en conseil scientifique (RFICS) and the Centre de recherche sur la science et la technologie (CIRST) at the Université de Sherbrooke, under the supervision of Professor François Claveau. He is also a postdoctoral researcher within the ENDURE project, supervised by Professor Philippe Bourbeau, and a researcher on digital sovereignty with Professor Guillaume Beaumier at the École nationale d’administration publique (ENAP). His research applies computational methods, natural language processing (NLP), and machine learning to analyze global challenges such as international public health crises, climate change, digital sovereignty, and online radicalization. He specializes in the development of large-scale research infrastructures that integrate computational social science and policy analysis to address interconnected global problems.

Louis-Robert Beaulieu-Guay is an Assistant Professor at the Johnson Shoyama Graduate School of Public Policy (University of Saskatchewan). His research bridges computational social science and public policy, focusing on how governments and citizens produce and communicate policy-relevant knowledge. He builds and analyses large-scale text corpora, including Canada’s federal regulatory activities and public consultation reports from Montreal. In addition to his scholarly works, he co-authored the Machine-Readable Regulatory Impact Analysis Statements report for the Treasury Board Secretariat of Canada and 20 Years of Advocacy for the Preservation of Montreal’s Heritage: A Quantitative Analysis of Public Consultations for the City of Montreal.