Consultancy-Final Evaluation of “OQ Integrated & Inclusive Humanitarian WASH and Protection Response (INT10403)

tendersglobal.net

Description

BACKGROUND:

Oxfam is a global movement of people working together to end the injustice of poverty. Together we save, protect and rebuild lives. And we won’t stop until every person on the planet can defeat poverty or any type of inequality or discrimination. We are an international confederation of 20 organisations (affiliates) working together with partners and local communities in the areas of humanitarian, development and campaigning, in more than 90 countries. All our work is led by three core values: Empowerment, Accountability, Inclusiveness.

The Iraq 2021 Humanitarian Needs Overview (HNO) identified 2.4 million individuals with acute humanitarian needs[1], while COVID-19 has increased the severity of humanitarian needs by 35%[2] by reducing livelihood opportunities and access to basic services, which were inadequate prior to the pandemic. While the overall number of people in need remain comparable to 2020, the number of people in acute need increased from 36% for 45%. According to the clusters, returnees and internally displaced

persons (IDPs) are classified as the most vulnerable groups, with the proportion of returnees facing acute need alone increasing from 28% to 38%[1]. Iraq also has one of the largest populations of Persons Living with Disabilities (PLWDs) in the world[2]. The situation worsened in October 2020 when the Government of Iraq (GoI) began closing IDP camps, forcing thousands of households (HHs) to return to areas where they had not lived for years, where there are no social services to support them.

Against this acute crisis backdrop, Oxfam proposed to improve equitable access and use of gender-responsive WASH facilities and protection services in Ninewa and Kirkuk, with a focus on HHs affected by recent camp closures. Targets districts in Kirkuk (Daquq, Hawija, and Kirkuk) and Ninewa (Telafar, Sinjar, Ba’aj, and Hatra) Governates have been selected based on urgent needs and the opportunity to build on GAC’s prior humanitarian and development investments[3]

PROJECT INTERVENTION AND TARGET GROUPS:

The project has the following goal, outcomes, outputs and key activities.

Ultimate outcome:

Ultimate outcome 1000: ‘Increased and maintained human dignity, wellbeing, safety, and lives saved for crisis affected IDP, returnee and host communities, particularly women, girls and IDPs affected by camp closures in Kirkuk and Ninewa Governorates of Iraq

Intermediate Outcomes:

Intermediate Outcome 1100: Increased and equitable use of disease-preventing, appropriate and gender-responsive WASH materials, facilities and services for crisis-affected women, men, boys, and girls in Kirkuk Governorate

Indicators: –

% of individuals reporting accessing at least 80 liters/person/day of safe potable water per day (disaggregated by sex)
% of individuals accessing functioning sanitation facilities (disaggregated by sex)
% of individuals applying improved hygiene practices (disaggregated by sex)

Intermediate Outcome 1200: 1200 Reduced barriers to, and enhanced use of, lifesaving protection services and community-based humanitarian interventions for vulnerable conflict-affected women, men, girls, and boys, including Persons Living with Disabilities (PLWDs) in Kirkuk and Ninewa Governorates

Indicators: –

% of targeted vulnerable individuals reporting reductions in barriers to accessing quality SGBV and protection services (disaggregated by sex, age, ability)
% of targeted vulnerable HHs reporting improvements to their wellbeing and safety (disaggregated sex of HH head)

Immediate Outcomes:

Immediate Outcome 1110: Increased and equitable access to inclusive, safe domestic water for IDP, returnees and host communities, particularly for women, girls and PLWD

Indicators: –

% of target HHs reporting satisfaction with the quality of water at the end of the project (disaggregated by sex of HH head)
% of the target HHs reporting improved equitable access to safe domestic water particularly for women, girls and PLWDs (disaggregated by sex of HH head)

Immediate Outcome 1120: Improved access to gender-responsive, safe sanitation services designed by and for women, girls, and PLWDs through community-based solutions.

Indicators: –

% target institutions with gender-sensitive sanitation facilities accessible to women, girls and PLWDs
% of sanitation facilities in target institutions properly maintained at the end of the project

Immediate Outcome 1130: Improved capacity of targeted communities to exercise health-promotion and dignified hygiene and environmental practices, including COVID-19 Infection Prevention and Control (IPC)

Indicators: –

% of HH representatives that can articulate at least 3 key hygiene & environmental practices that can prevent spread of diseases. (Disaggregated by sex)
% of HH representatives that reported improved health and hygiene practices. (Disaggregated by sex)

Immediate Outcome 1210: Increased access to SGBV and lifesaving protection services for conflict-affected women, men, girls and boys, particularly HHs affected by camp closures.

Indicators: –

% targeted vulnerable people reporting safe access and timeliness of SGBV and emergency protection services (disaggregated by sex and age)
% of vulnerable people reporting decreased protection, physical and mental health risks due to CfP and unstructured psychosocial support (PSS) services (disaggregated by sex and age)

Immediate Outcome 1220: Improved community capacity to monitor, identify, mitigate and respond to emergency SGBV and protection risks and gaps.

Indicators: –

% of trained women and youth and CBPC members equipped with skills to monitor, identify, and mitigate SGBV and protection risks and gaps within their communities (disaggregated by sex, age, ability)
% of targeted women and men and CBPC members reporting improvement to their understanding and practice of SGBV and Protection core standards (disaggregated by sex, age, ability)

Outputs:

1111 Existing water supply systems upgraded, rehabilitated, and maintained.

1112 Gender-balanced and inclusive water management committees and other community-level stakeholders trained and linked to Directorate of Water (DoW) for WASH facilities’ operation and maintenance (O&M)

1113 Asset Management Plan and water quality analysis conducted with DoW for sustainable and safe water supply O&M

1121 Gender-appropriate latrine & handwashing facilities serving women, men, girls, boys and PLWDs, expanded and upgraded in the target communities and institutions.

1122 Health facilities and CHVs provided with PPEs and medical supplies for COVID-19 IPC

1123 Appropriate safe solid and hazardous wastes practices established in targeted health facilities.

1131 Community Health Volunteers (CHVs) and women’s groups mobilized and supported to promote inclusive and safe community-based environmental sanitation processes.

1132 Community awareness raising sessions delivered on COVID-19 IPC and WASH practices with emphasis on integrating men to share responsibilities for health management at HH level.

1133 Essential non-food items delivered to the most vulnerable HHs, with a focus on female- headed HHs, PLWDs, and HHs affected by recent camp closures.

1211 SGBV and protection services mapped and shared with communities, clusters and government actors.

1212 Timely, confidential and safe case management, legal counselling and services provided to SGBV survivors and other vulnerable HHs facing protection risks.

1213 Psychosocial support (PSS) services and recreational activities established targeting vulnerable girls, with a focus on those affected by camp closures.

1221 Inclusive Community Based Protection Committees (CBPC) trained on Protection and humanitarian standards and peaceful co-existence activities, especially on those meeting the specialized needs of women, children and PLWDs.

1222 Roundtable discussions conducted and facilitated by Oxfam and CBPC with local authorities and other stakeholders to support collaboration on inclusive and lifesaving Protection solutions.

1223 Inclusive humanitarian response cycle, quality and accountability capacity development program developed and delivered to local humanitarian actors.

OBJECTIVES OF THE FINAL EVALUATION

This GAC funded project will end in June 2024 and an external final evaluation is planned to take place on July and August 2024.

The overall objective of the GAC final evaluation is to systematically review the overall progress of the project towards the project’s outcomes, assess the learnings from project design and project implementation from our work with partners and communities, and provide recommendations on overall design of the project, modifications, and specific actions that might be taken into consideration in the future during the design of future projects of a similar nature. The scope for evaluation is determined using OECD-DAC criteria for evaluating humanitarian action. Relevant criteria are associated with a number of key questions (under section 5) that are to be addressed and explored.

Specific Objectives of the Evaluation:

To assess the design, planning, delivery and management of the project by Oxfam and its partners in accordance with Oxfam Programme Standards and donor requirements.
To identify and assess key internal and external factors that have contributed, affected, or impeded the achievements of the project, and how Oxfam and the partners have managed these factors.
Review the project outcome indicators and update the final/endline values in the log frame.
To assess the beneficiary targeting, verification and selection processes/approaches of the project
To draw key lessons and learning from the project and make recommendations that will help inform Oxfam’s formulation and design of future projects that will benefit the Iraqi displaced, host and returnee communities affected by the armed conflict.
To assess the relevance, effectiveness, efficiency, impact and sustainability of the project on IDPs, returnees and remainees on Oxfam and its partners.
Describe and assess efforts of stakeholders in support of the implementation of the project.
Assess the likelihood of continuation and sustainability of project outcomes and benefits after completion of the project.
Examine the extent to which the impact of the project has reached the intended beneficiaries.

EVALUATION CRITERIA AND KEY QUESTIONS TO BE ADDRESSED IN THE EVALUATION:

The project evaluation should be organized taking into consideration of Oxfam’s key evaluation criteria, and program quality standards. The evaluation will also consider key sector and sub-sector indicators for assessing the performance. The following provides a guide to the questions to be addressed by this evaluation, under each of the criteria below:

Relevance and Appropriateness:

Have Oxfam and its partners selected relevant operational areas for their work?
Have Oxfam and its partners targeted the most vulnerable people, including vulnerable men, women, boys and girls?
To what degree is the intervention addressing the WASH and livelihoods needs of the targeted women, men, boys and girls and contributing to reduced vulnerability?
Was project design consistent with the Do No Harm principle? and with a Safe Programming approach?
What was the level and quality of participation of the beneficiaries and the partners in project design?
How responsive were project activities to the needs of targeted women, men, boys and girls in targeted communities (if possible, please distinguish between displaced/returnee and host communities) given their circumstances and priorities?
To what extent have the key contextual changes, threats and opportunities that arose during implementation been influencing and informing project implementation?
How and to what extent were monitoring and evaluation findings used to inform decision-making and the improvement of project implementation by Oxfam and partners?
Was the project adapted appropriately to changing context and needs in a timely manner? What changes took place in the implementation of the project differed from the original design?

Efficiency:

Was the project implemented based on the best use of existing resources/capacity, e.g. the capacity of the partners and the internal capacity and expertise of Oxfam itself? What key limitations existed on this front? What could Oxfam’s future projects do to increasingly develop and invest in existing resources?
How cost-effective was the intervention? What cost-effective alternatives could have been used?
Were the resources (financial, human, etc) for running all the activities available, adequate and was this the best use of resources to achieve the results?

Effectiveness:

Has the project proceeded well towards the achievement of its outcomes?
Is there anything about this project that has had any impact on the beneficiaries (e.g. predictability of aid, etc.)? (positive or negative)
Has this project had any impact on the partners? (positive or negative)
What were the main challenges of the project and how well have they been addressed? Is there anything in this project that made these challenges more/less manageable?
How were beneficiaries selected and were they informed of the selection criteria?
How effective were the selection criteria in reaching out to the most vulnerable populations?
How inclusive and culturally sensitive were the activities carried out in terms of the approach, quality of participation, information and its dissemination?
To what extent has gender been mainstreamed into the project? How could gender be better mainstreamed in a future project?
What unintended consequences (if any), whether positive or negative, did the intervention have on women, men, boys and girls so far (IDPs/returnees/host communities)?
What system and mechanism are in place (at Oxfam and partners) to ensure accountability to the beneficiaries and how well have they been working throughout the project?
Have the ways of working with the partners been contributing to building the capacity of the partners?
Were there any risks inherent to the duration of the project, either in the course of or towards the end/after the end of the intervention?
What factors have contributed to achieving or not achieving intended outcomes and set objectives of the project? Identify any exceptional experiences/achievements of the project substantiated with evidences, such, but not limited to case studies, best practices, etc.

Impact:

What the positive and negative, intended and unintended, changes produced by the project?
Have these changes contributed to any identified changes according to aspirations of project?
Analyse the contribution of the project to any observed impact (intended, unintended, positive, negative) and analyse what other actors and factors contributed to the impact. What real difference has the intervention made to the beneficiaries?
How many people have been affected? – Are the local communities and stakeholders more resilient than before?
How likely is it that any positive changes may be sustained in the short- and medium-term?

Sustainability:

To which extent did the planning and implementation of the interventions take longer-term and integrated problems into account?
How did the project consider adequately to the context and the nexus of between humanitarian action, recovery and development?
Did the project plan and implement an adequate transition and exit strategy that ensures longer-term positive effects and reduces risk of dependency?
How likely will critical services and effects be sustained beyond the duration of the project?
How well are the project’s outputs linked to more long-term focused objectives?
Assess capacity of key actors to contribute to sustaining the positive changes according to the developed ToC.
What were/are the major factors which influenced the achievement or non-achievement of sustainability of the project?
To what extent did the benefits of the project continue after donor funding ceased? What were the major factors which influenced the achievement or non-achievement of sustainability of the project?

METHODOLOGY:

Oxfam recommends mixed methods (a combination of both quantitative and qualitative methods) that will improve the evaluation findings by ensuring that the limitations of one type of data are balanced by the strength of another. Therefore, the data collection for this evaluation will contain various tools and methodologies (both quantitative and qualitative approaches). The consultant will be responsible for designing of tools, data collection, data analysis and report writing using both quantitative and qualitative methods.

For the quantitative approach, the consultant will conduct the data collection activities of the endline HH survey in the selected intervention areas to find the endline values of outcome level indicators, which will be compared later against the baseline survey values which was already done at the beginning of the project. Quantitative data using structured questionnaire can be used to see how the project’s intervention bring change to the life of conflict affected communities. The endline survey findings will be also integrated with the evaluation findings according to the key evaluation criteria (relevance, effectiveness, efficiency, sustainability, and impact).

Regarding the qualitative approach, the consultant is expected to develop a through methodology (to be endorsed by Oxfam) for this evaluation based on the expected deliverables & OECD-DAC criteria. The consultant will use qualitative data collection methods, such as, focus group discussions/FGD/ with different groups (men, women), key informant interview/KII/, with local government authorities, Oxfam staff, community leaders, partners and other stakeholders, case studies, etc).

The consultant(s) will also do desk study/ literature review, which includes review of different documents, such as baseline and other surveys, project proposals, progress reports, etc and analysing secondary data. The evaluation methodology should make use of baseline and other data already generated by Oxfam’s monitoring and evaluation team. The information and findings of the desk review should be integrated with the data and findings from primary data collection and analysis, and not placed in a separate section.

We anticipate that this evaluation will be a participatory review and learning exercise. Thus, it requires the consultant(s)/firms to be experienced in participatory approaches to learning and inquiry, and especially in seeking the views and perceptions of key stakeholders that include:

Targeted beneficiaries
Partners and actors directly involved in the project at different levels:
The local implementing partners.
Community leaders (if applicable) and representatives of the affected population.
Local authorities.
Oxfam staff involved in the implementation of the project (e.g. AMs, Technical Leads, and field program and MEAL team.)

The consultant should develop all the necessary data collection tools and sampling methods and share them with the Oxfam MEAL & PQ team for review and approval. And the data collection tools proposed by the consultant should be able to address the key evaluation questions mentioned above.

The evaluators/external consultants should consider the following methodological topics at a minimum:

The evaluator’s understanding of the evaluation questions
Description of the phases related to the suggested evaluation methodology.
Sources of information for gathering primary and secondary data
Sampling techniques, such as sample size, sampling methodology, the area and population group the sample represents.
Instruments to be used for data collection.
The type of software or ICT tool used for data analysis.
Reference indicators and benchmarks for each evaluation question (if relevant).

Immediate Outcomes:

Immediate Outcome 1110: Increased and equitable access to inclusive, safe domestic water for IDP, returnees and host communities, particularly for women, girls and PLWD

Indicators: –

% of target HHs reporting satisfaction with the quality of water at the end of the project (disaggregated by sex of HH head)
% of the target HHs reporting improved equitable access to safe domestic water particularly for women, girls and PLWDs (disaggregated by sex of HH head)

Immediate Outcome 1120: Improved access to gender-responsive, safe sanitation services designed by and for women, girls, and PLWDs through community-based solutions.

Indicators: –

% target institutions with gender-sensitive sanitation facilities accessible to women, girls and PLWDs
% of sanitation facilities in target institutions properly maintained at the end of the project

Indicators: –

% of HH representatives that can articulate at least 3 key hygiene & environmental practices that can prevent spread of diseases. (Disaggregated by sex)
% of HH representatives that reported improved health and hygiene practices. (Disaggregated by sex)

Immediate Outcome 1210: Increased access to SGBV and lifesaving protection services for conflict-affected women, men, girls and boys, particularly HHs affected by camp closures.

Indicators: –

% targeted vulnerable people reporting safe access and timeliness of SGBV and emergency protection services (disaggregated by sex and age)
% of vulnerable people reporting decreased protection, physical and mental health risks due to CfP and unstructured psychosocial support (PSS) services (disaggregated by sex and age)

Immediate Outcome 1220: Improved community capacity to monitor, identify, mitigate and respond to emergency SGBV and protection risks and gaps.

Indicators: –

% of trained women and youth and CBPC members equipped with skills to monitor, identify, and mitigate SGBV and protection risks and gaps within their communities (disaggregated by sex, age, ability)
% of targeted women and men and CBPC members reporting improvement to their understanding and practice of SGBV and Protection core standards (disaggregated by sex, age, ability)