Result of Service
Final Report: Inter-Agency Synthesis of evaluative evidence of the humanitarian response in Ukraine
Work Location
Remote, home-based
Expected duration
6 months
Duties and Responsibilities
Introduction: The Inter-Agency Evaluation Humanitarian Steering Group (IAHE SG) – an Inter-Agency Steering Committee (IASC)-associated body, prepared the commission of an Inter-Agency Humanitarian Evaluation (IAHE) of the humanitarian response in Ukraine, in line with IASC Scale-Up Protocols . However, due to ongoing security risks in a fast-changing context and a high workload for humanitarian actors on the ground, the IAHE SG decided in June 2023 to undertake a synthesis of evidence from agency evaluations of the response to the Ukraine crisis instead. The Synthesis aims to fulfil the learning needs of the collective humanitarian response in Ukraine. Background: The Synthesis of Evaluative Evidence on Humanitarian Response in Ukraine (hereinafter referred to as “synthesis”) will be carried out under the auspices of the IAHE SG, which is chaired by the Office for the Coordination of Humanitarian Affairs (OCHA) and consists of the Evaluation Directors of IASC member organizations. An Inter-Agency Management Group is being set up to manage the synthesis exercise. Members are ICVA/DRC, IFRC, OCHA (Chair), OHCHR, UNHCR and WFP. The objective of the synthesis is to map the existing IASC members’ evaluative evidence on the humanitarian crisis in Ukraine, synthesize the evidence according to the criteria and questions outlined in the TOR, and to offer lessons learned and actionable recommendations for improvement of the collective humanitarian response in Ukraine, and for the humanitarian system overall. OCHA is seeking to recruit an evaluation specialist to undertake the inter-agency synthesis of evaluative evidence on the humanitarian response in Ukraine. The data analyst will be teamed up with an evaluation specialist (Synthesis Consultant). Indicative Synthesis questions are: RELEVANCE How well does the evidence show that the collective IASC response in Ukraine since 2022 has served the best interests (short and longer-term) of vulnerable people in the country? [Specifically, to what extent has it enabled them to avoid, withstand and recover from threats to their well-being and security?] Sub-questions: R1: Scale-up and adaptive capacity: To what extent does the evidence find that IASC/HCT member agencies were able to anticipate the crisis, the changes in the context and adjust their capacities to respond? R2: Needs and data: To what extent was the response based on comprehensive analyses of needs of vulnerable groups in Ukraine, including children, people with disabilities, the elderly and minority groups affected by the conflict? To what extent did response objectives target these needs? R3: Response design: To what extent was the collective response logic clear and well-defined? Were the goals set and resource-raising approaches applied, appropriate and based on realistic assumptions regarding the context in Ukraine? R4: Security at operational level: To what extent does the evidence indicate that the response took into account, and adjusted for, security issues in Ukraine? R5: Civil society engagement: To what extent was the collective response aligned with and relevant to the direct engagement by citizens and local actors in alleviating the effects of the war in local communities? EFFECTIVENESS To what extent does the evidence find that IASC members’ collective efforts were able to effectively respond to the humanitarian crisis in Ukraine, especially in addressing the needs ofthe most vulnerable? Sub-questions: EFFECT1: Results: To what extent does the evidence show that the collective response generated significant positive or negative, intended or unintended effects for the affected population, collectively? EFFECT2: Equity, inclusion and human rights: To what extent did the collective assistance delivery reach all identified target groups and specifically women and girls, minorities and people living with disabilities? To what extent did it help realise their rights? EFFECT3: Factors: What specific factors enabled or inhibited the effectiveness of the response and how? Including: a) IASC tools and protocols such as scale-up activation b) Funding, volume and profile c) Leadership and management arrangements, particularly in the early phases of the response EFFECT4: Humanitarian access: What efforts did evaluations find were made by IASC members to secure humanitarian access in constrained areas of the country, and how successful were these? EFFECT5: Humanitarian principles: To what extent and how were humanitarian principles prioritised in the collective response? How successfully were any trade-offs or tensions between principles navigated? EFFECT6: Cross-cutting issues: To what extent does the evidence show that PSEA, gender, disabilities and AAP were mainstreamed throughout the response? SUSTAINABILITY To what extent has the collective response delivered benefits to Ukraine that will last? Sub-questions: SUST1: Nexus: To what degree has the response connected the humanitarian dimensions of the collective response with resilience and reconstruction efforts in Ukraine? SUST2: National systems: To what extent, and how, does the evidence find that the collective response supported national systems in Ukraine and particularly those related to social protection? PARTNERSHIPS/LOCALISATION To what extent does the evidence find that the collective response in Ukraine made use of available national capacities, including localization and partners to deliver assistance to affected people? Sub-questions: P/L1: Partnerships and localisation: To what extent did the collective response seek out, and build on, national and local actors both as ‘first responders’ to the crisis, and subsequently as partners in the ongoing effort to combat the effects of the crisis on the vulnerable? P/L2: Capacities: To what extent were the diverse local response capacities in Ukraine utilized and integrated at the coordination and response level? P/L3: Systemic fit: To what extent was the cluster system able to work with Ukrainian civil society outside of established organisations such as local NGOs? COORDINATION To what extent do evaluations find the co-ordination model in Ukraine fit for purpose in the context? To what extent did it support operational delivery? Sub-questions: COORD1: Internal coherence: To what extent does the evidence find that the collective response strategy was internally coherent across sectors? To what extent did it support complementarity and mutual reinforcement between sectoral/agency/geographically targeted interventions? COORD2: The cluster system: How effectively has the cluster system, including its strategies, such as area-based coordination model, besides others, worked in Ukraine, and has it retained its relevance over time? COORD3: Operational co-ordination: How well did IASC member organizations coordinate their efforts responding to the humanitarian needs in the varying conditions of the different areas of Ukraine? COORD4: Joint advocacy: To what extent have IASC agencies been able to improve operating conditions through joint advocacy/coordinated action (through the UN or otherwise)? Synthesis methods: The synthesis’ main approach is learning-oriented, as opposed to accountability-aimed. The rationale for this largely stems from scoping phase consultations with stakeholders. The humanitarian actors stressed on the constantly changing context in Ukraine and challenges to aggregate results in broader terms, thus rendering a synthesis exercise inconducive to an accountability lens. Proposed data collection methods include: Sample collation: The synthesis Management Group collated a preliminarily list of evaluative evidence that falls under the inclusion criteria (see bullet point below). Further identification of suitable evidence will continue until the synthesis’ inception phase. The Inception Report will contain the final list of evaluative evidence that will be included in the systematic document review. Systematic document review of IASC members’ individual evaluative evidence on humanitarian crisis in Ukraine: the evaluative evidence will be identified/ confirmed based on the following inclusion criteria (i.e. criterion sampling): a) evaluative evidence that complies with the UNEG definition of evaluation; b) the evidence is focused on a humanitarian actor’s response to Ukraine crisis; c) the quality of the sample document is appraised either externally or internally, or both. Data extraction based on deductive approach, whereby evidence will be coded and extracted against the synthesis’ themes and questions but allowing for emergent coding for patterns of data that were not foreseen (inductive approach). Data coding may be manual or electronic (or a combination of both). In case of the use of Artificial Intelligence (AI), the method, as well as its ethical considerations, will be described in the Inception Report. The anticipated steps of the synthesis are desk review, preparation of a short inception report, data collation, extraction and coding, data analysis, draft and final synthesis report and presentation of results. It is expected that the data will be complemented with a limited number of interviews with key stakeholders (estimated 15 interviews), to fill in evidence gaps and verify information when necessary. The consultant will be paid upon deliverables of key milestones (Final Inception Report, Draft Synthesis Report, Final report) Duties and Responsibilities of the data analyst: The data analyst will be primarily responsible to support the Synthesis consultant to collect, analyze and interpret data to provide insights to the Synthesis questions: – Collect and organize available data in a database, identify gaps – Develop and apply various analysis (statistical and/or other analytical) tools – Analyze and interpret datasets to identify patterns, trends and relationships between data in line with the areas of inquiry (synthesis questions) – Communicate findings in a clear and concise manner – Provide insights for relevant sections of the Synthesis (methods & limitations, findings, conclusions and recommendations) – Proven experience with data quality assurance mechanisms – Experience contributing to evaluation or synthesis reports – Maintain a record of analysis conducted – Collaborate with Synthesis consultant Duties and Responsibilities of the synthesis management group: – Advice and support the data analyst when necessary – Provide document repository and facilitate access to additional data where possible and required
Qualifications/special skills
Advanced university degree (Master’s degree or equivalent) in social science, evaluation, humanitarian affairs, public administration, international development, economics, political science, or related areas is required. A minimum of 7 years of professional experience in research and data analysis for evaluation with an excellent understanding of social research and evaluation principles and methodologies, including capacity in an array of qualitative and quantitative data analysis, and UNEG Norms and Standard are required. Previous experience in designing, conducting and leading data analysis, or comparable products (meta-evaluations, systematic reviews), including in the humanitarian context is required. Adequate technical skills in coding, analyzing large sets of qualitative data and qualitative data analysis software as well as analytical skills with an ability to identify patterns and divergences in large data sets and attention to detail is required. A proven record of delivering timely and high-quality results, i.e., credible data analysis that are used in research papers/articles and/or evaluation reports is desirable. Previous experience of working in Europe and Eastern Europe, together with understanding of the context of the war in Ukraine, is desirable.
Languages
Fluency in English (both oral and written) is required.
Additional Information
Communication: The data analyst shall obtain the Management Group’s written approval prior to issuing any press releases or making any public statements concerning the Synthesis. All decisions with regards to products or documents and other materials, which bear a direct relation to, or are produced, prepared, collected during the conduct of the synthesis, shall be made by the Management Group. The synthesis consultant shall not publish any confidential information or draft deliverables. The consultant is expected to declare any conflict of interest and sign a non-disclosure agreement as part of the contract. Other: – The applicant might point out in their application if they are applying as a team together with the Synthesis Consultant (See JO 243877) – The applicant is expected to conduct several UN mandatory trainings (before and during the length of the contract
No Fee
THE UNITED NATIONS DOES NOT CHARGE A FEE AT ANY STAGE OF THE RECRUITMENT PROCESS (APPLICATION, INTERVIEW MEETING, PROCESSING, OR TRAINING). THE UNITED NATIONS DOES NOT CONCERN ITSELF WITH INFORMATION ON APPLICANTS’ BANK ACCOUNTS.
How to apply
Apply here