NLP and ML to Contextualize News for Policy Insights

Details

Mission and objectives

As the United Nations lead agency on international development, UNDP works in 170 countries and territories to eradicate poverty and reduce inequality. We help countries to develop policies, leadership skills, partnering abilities, institutional capabilities, and to build resilience to achieve the Sustainable Development Goals. Our work is concentrated in three focus areas; sustainable development, democratic governance and peace building, and climate and disaster resilience.

Context

Afghanistan development context is very dynamic. Country faces multiple interconnected challenges. To staying abreast of the dynamic nature of the country is essential for policymakers to make informed decisions, by tracking multiple news and putting them into context. The continuous flow of news presents a challenge in efficiently identifying relevant topics and connecting to a contextual information pertinent to policy analysis. To address this challenge, Afghanistan Policy and Knowledge Unit explores the Natural Language Processing (NLP) and Machine Learning (ML) techniques. These technologies offer the capability to process large volumes of text data, extract meaningful insights, and provide a structured representation of information crucial for policy formulation and analysis.
You will support this work by developing scripts for applying NLP and ML techniques to news and contextual infromation datasets. Through this work you harness NLP and ML techniques, learn about their application for policy analysis, and understand development challenges in Afghanistan and broader region.

Task description

• Data Structuring and Cleaning. Compile a corpus of news articles and clean the data. Compile a corpus of contextual information and clean the data.
• Use NLP techniques such as LDA to identify key topics within the news corpus.
• Implement Named Entity Recognition (NER) algorithms to extract important entities like locations, people’s names, and project names from the news articles to update contextual information corpus
• Link identified topics with extracted entities to provide context and understanding of each topic
• Set up a flow for preparing the mapped contextual information to generate actionable insights for policy analysis