Consultancy on Performance Evaluation for assessing the impact of Multi-Sectoral Humanitarian Response South Sudan project

Terms of Reference (TOR)

Consultancy on Performance Evaluation for assessing the impact of Multi-Sectoral Humanitarian Response South Sudan project

Bureau of Humanitarian Assistance (BHA), Award (# 720BHA23GR00097)

Who is the Danish Refugee Council

Founded in 1956, the Danish Refugee Council (DRC) is a leading international NGO and one of the few with a specific expertise in forced displacement. Active in 40 countries with 9,000 employees and supported by 7,500 volunteers, DRC protects, advocates, and builds sustainable futures for refugees and other displacement affected people and communities. DRC works during displacement at all stages: In the acute crisis, in displacement, when settling and integrating in a new place, or upon return. DRC provides protection and life-saving humanitarian assistance; supports displaced persons in becoming self-reliant and included into hosting societies; and works with civil society and responsible authorities to promote protection of rights and peaceful coexistence.

Danish Refugee Council is a leading, international humanitarian displacement organization, supporting refugees and internally displaced persons during displacement, in exile, when settling and integrating in a new place or upon return. DRC provides protection and life-saving humanitarian assistance. DRC has been operational in South Sudan since 2005, implementing multi-sectoral responses including Protection, Camp Coordination and Camp Management (CCCM), Economic Recovery, Shelter and Non-Food Items (SNFI), Humanitarian Disarmament and Peacebuilding (HDP). DRC is implementing projects in Upper Nile, Unity, Jonglei, Western Bahr El Ghazal, Eastern Equatorial States and country wide through static and mobile response teams.

Purpose of the consultancy

The main purpose of the Performance Evaluation for DRC’s BHA-funded Multi-Sectoral Humanitarian Response South Sudan is to measure the overall impact of the project on the target population and the broader community. This includes evaluating changes in key indicators, such as access to protection services, the effectiveness of CCCM services, and livelihood opportunities for beneficiaries.

Background

The BHA-funded Multi Sectoral Humanitarian Response South Sudan project aims to provide lifesaving humanitarian assistance through integrated multi-sector assistance of protection, shelter, livelihood and basic needs through Mobile Response Team (MRT) across the country and community-based programing in the targeted locations. The project foster accountability and quality humanitarian assistance through the state-level camp coordination and camp management coordination leadership that empowers the crisis-affected people to make decision and actions to address humanitarian needs. The integrated approach within each sector and sub-sector of the project is designed to mainstream protection, social cohesion and conflict sensitivity.

The purpose of the program is the following:

1. Women, men, boys, and girls in the most affected locations across South Sudan receive emergency, medium, and longer-term protection assistance to prevent, address, and recover from the effects of human rights violations.
2. Effective coordination, information management, and assistance are provided to displacement-affected communities in and out of camp-like settings in a conflict-sensitive way that supports transition out of emergency status.
3. Basic, secure, private, habitable, safe living spaces and lifesaving NFIs are provided to new and long-term displacement-affected communities to allow households to resume critical social and livelihood activities and reduce the severity and prevalence of protection-associated risks caused by lack of shelter.
4. Displacement-affected communities have improved their ability to undertake income-generating activities, access financial services, and strengthen market systems, contributing to a reduction in the severity and probability of protection risks.

The proposed Theory of Change (ToC) states that: If the lifesaving needs of displacement-affected populations are addressed, and if early recovery interventions are supported, and if the political, security and economic environment remain conducive, then the probabilities, severity, and prevalence of protection risks will be reduced, empowering displacement affected communities to be more self-reliant and access transitional pathways out of displacement or vulnerable statuses.

Final Evaluation Summary

Summary of Performance Evaluation

Program Title

Multi-Sectoral Emergency Response South Sudan

Activity Goal

Displacement-affected communities in South Sudan have reduced protection risks associated with displacement and are able to live in safety and dignity.

Geographic Coverage

South Sudan: Upper Nile and Unity States, Country-wide

Target Counties

Unity State: Rubkona, Koch, Guit; Upper Nile: Malakal and Fashoda; Countrywide- MRT

Assessment Type

Performance Evaluation

Assessment Purpose

To measure the overall impact of the project on the target population and the broader community. This includes evaluating changes in key indicators, such as access to protection services, shelter, the effectiveness of CCCM services, and livelihood opportunities for beneficiaries

Methodology

A desk review, quantitative and qualitative method will be used to gather information from project participants in the target area.

Proposed data collection starts and end dates: August 01 to 20, 2024

Final report submission date to DRC: September 20, 2024

Objective of the consultancy

The specific objectives of this Project Performance Evaluation will include the following:

1. Assess whether the protection prevention and response services, shelter support, CCCM services, and livelihood support are effectively addressed the needs of the affected population.
2. Assess the effectiveness of Women and Girls Safe Spaces (WGSS) in fostering social interactions and the formation of new social networks, accessing timely information and support on relevant issues, including response to protection violations, enhancing their decision-making abilities, and improving their overall psychosocial wellbeing through structured and unstructured activities.
3. Assess the effectiveness of cash for work vs mere cash provision in shelter maintenance and construction.
4. Evaluate how effectively women targeted by the program have utilized their training-acquired knowledge and skills to craft dignity kits from locally sourced materials. Additionally, illustrate the benefits they have gained from this process, including earning income through e-vouchers.
5. Identify best practices, including successful strategies and interventions, within each sector that can be replicated or scaled up in future projects. This involves gathering success stories, and documenting lessons learned and best practices to inform future programming and improve the effectiveness of humanitarian interventions.
6. Identify strengths, any gaps or weaknesses in the project design or implementation that may or preventing the achievement of desired outcomes or inadvertently thwart improved capacities at the individual and community levels

Scope of work, evaluation questions and Methodology

An external Consultant will be required to prepare a detailed methodology and work plan indicating how the objectives of the project will be achieved, and the support required from DRC.

Scope of Performance evaluation

The Performance Evaluation will focus on assessing the achievements of the BHA project from the start of the program to date (May 2023 to August 2024) in the targeted counties of Rubkona, Koch and Guit in Unity State, Malakal and Fashoda in Upper Nile State and accessible hard-to-reach areas by the MRT. The key stakeholders to be involved in the Performance Evaluation process will be DRC project staff (Area Managers, Project Managers, Team Leaders, Technical Coordinators, Supply Chain and Finance team members), beneficiaries (women, men, girls, and boys), community leaders, chiefs, women’s representatives, youth representatives, local authorities, UN Agencies, cluster and relevant Area of Responsibility (AoR) leads, including Protection, CCCM, and Shelter/NFI clusters and the Gender-Based Violence AoR, and other NGOs operating in the target locations.

Evaluation Questions

The key impact questions and criteria are as follows.

• Protection: To what extent do the implemented protection activities adequately address specific protection risks faced by the affected populations, particularly women and girls, reducing vulnerabilities to these threats and enhancing capacities against them at the individual and community levels, in the context of the humanitarian crisis?
• Shelter: How effectively does the provision of shelter support meet the immediate and longer-term housing needs of the affected population, including ensuring safety, privacy, and dignity?
• CCCM Services: To what degree do the CCCM services facilitate efficient and equitable access to essential services, promote community participation, and address governance and security concerns within the camp settings?
• Livelihoods: How does livelihood support influence the economic resilience and self-reliance of communities, especially vulnerable groups?
• Integration: To what extent do beneficiaries benefit from integrated and/or multi-sectoral activities, providing holistic support to their needs?

1. Did the project intervention address the protections risks of the most vulnerable individuals of the target population?
2. Were the approaches to shelter used effective in meeting shelter and shelter related protection needs?
3. How have women targeted by the program utilized their training-acquired knowledge and skills to produce dignity kits from locally sourced materials, what tangible benefits, have they derived from this process?
4. What specific best practices, successful strategies, and interventions within each sector have demonstrated the highest potential for replication or scaling up in future projects, and how effectively have they been documented to inform future programming and improve the effectiveness of humanitarian interventions?
5. What strengths, gaps, or weaknesses in the project design or implementation have been identified that may be hindering the achievement of desired project outcomes and/or inadvertently contributing to further harm or increased risk?

5.3 Performance Evaluation Methodology

The Performance Evaluation data collection will use desk review and qualitative and quantitative mixed methods of data collection, including focus group discussions (FGDs), assessments, transect walks, key informant interviews (KIIs), individual success stories, case studies, and observation of project activities. The desk review will include a detailed reading and assessment of key program documents, including work and MEAL plans, budgets/expenditures, semi-annual performance reports, etc. DRC will share these documents with the selected consultant to support the preparation and submission of the inception report.

5.3.1 Desk Review

The DRC will provide to consulting firm relevant program documents including activity proposal; MEAL plan; baseline report, workplans, semi-annual and annual performance reports; Indicator Progress Tracking Table (ITT), post distribution monitoring reports, intention and satisfaction reports and; success stories and other data that may be relevant for the desk review.

5.3.2 Quantitative household survey

The consultant will conduct a survey of a representative sample of households to gather comparative data on the protection, shelter, CCCM, and economic recovery activities, before the crisis and current situation. The consultant will also review existing data from programmatic work e.g. the MEAL reports, previous assessments and other relevant documents. The consultant will detail the above information or approaches in the inception report and submit it to DRC for feedback and review.

5.3.3 Sample design

A statistically representative sample from the study population will be generated to determine the final sample size of respondents to be selected for the performance Evaluation. The sampling approach should be guided by the context and the existing logistical realities.

5.3.4 Qualitative Data Collection Method

The FGDs will be conducted with male and female beneficiaries in relevant age categories separately in each targeted county for each of the proposed key programme outcomes. KIIs will include key DRC staff members (Technical Coordinators, Project Managers, Head of Programme, Grants team, Area Managers, Project Officers, M&E team), local authorities, youth representatives, community leaders, women’s representatives, and other relevant stakeholders.

Once selected, the consultant will also be required to provide additional details on their qualitative research methods. This will include:

• Identifying specific categories of respondents for each question and a rationale must be provided as to why those respondents were selected;
• Clear justification for selected data collection methods, including criteria for selection of participants for FGDs and KIIs.

Deliverables

The consultant shall submit the following deliverables, according to the ToR and within the agreed timeline:

1. Inception report – outlining the key scope of the work, training manuals, intended Performance Evaluation questions, detailed methodology and data collection tools, data sources and analysis approach, roles and real time frame, and suggested work plan. The inception report will be submitted to DRC five (5) days after signing the contract. This is to ensure that both the consultant and the DRC team have a common understanding of the assessment.
2. Implementation plan – will include the detailed assessment process and timeline with a detailed calendar of key activities and milestones.
3. Draft Performance Evaluation report: the consultant should submit the draft report to DRC according to the recommended structure within the agreed timeline.

• Executive summary
• Brief background (as directly relevant to the report’s analysis and conclusions)
• Methodology (including challenges and data collection approach)
• Key findings and conclusions by sector/result (supported with relevant data and data sources)
• Key best practices with details of how they might be replicated, and recommendations for implementation of future projects and wider learning, by sector.
• Annexes, including a list of all data collection tools, Performance Evaluation Dataset (in excel or CSV), work schedule, and any other relevant information

The report will be reviewed to ensure that it meets the required quality criteria. Comments from relevant DRC program staff will be provided within five (5) working days after reception of the draft report.

1. Final report: (15 to 20 pages): The consultant will be required to address all comments provided by DRC program staff. The report should be finalized seven (7) working days after receiving the revised draft from DRC. The consultant should maintain the requirements for the content, format, depth and length, overall quality, and approved timelines. Once the report has been finalized, it will be disseminated within DRC teams initially. DRC teams then to cascade the key findings and highlights from the evaluation in different coordination forums to reach other humanitarian agencies and stakeholders.

Duration, timeline, and payment

The Performance Evaluation will start on 1 August 2024 and end on 20 September 2024. The final report will be submitted by DRC to BHA on 30 October 2024. The tentative schedule is outlined in Table 2 below.

Inception report: draft inception report, including methodological design, data collection instruments, field manuals, and data collection manuals shared by the consultant.

First draft report: Data cleaning, data analysis, produce and share the first draft report with DRC for their comments.

Second draft report incorporating all comments and inputs shared by the consultant

Final report submitted by the consultant

Budget

The budget allocation for the Performance Evaluation will cover consultant fees, travel expenses, data collection, analysis tools, and any other relevant costs. The consultant will provide a financial proposal for the provision of these services. The cost of travel and other expenses should be included in the overall quotation for the service. Agreed rates will be based on prevailing market competitive rates and value for money. The consultant is responsible for the following:

• The consultant will be responsible for payment of any tax or other fees related to this assignment.
• The consultant is responsible for any insurance, including medical and travel, related to this assignment.
• The consultant is responsible for his/her working tools, such as computers, mobile phones, tablets for data collection, etc.
• Payments will be in three installments: 30% after submission of the inception report, 30% after submission of the draft report, and 40% after submission of the final approved report to DRC.
• Please note that a partial payment hold-back will be in effect until a final report has been approved by DRC

Proposed Composition of Team

• Lead consultant/Team Leader
• Programmer
• Field Research Coordinators
• Data Analyst
• Enumerators

Eligibility, qualification, and experience required

• Demonstrated experience in conducting project Performance Evaluation, case studies, and producing communication materials, particularly in humanitarian settings. At least 7 years of experience in carrying out qualitative research, and impact measurement. Proven experience in conducting primary data collection, including in fragile/developing contexts. Strong experience in questionnaire development, interview techniques and managing focus group discussions
• Expertise in protection, livelihoods, and/or camp coordination and management.
• Strong analytical skills and proficiency in both qualitative and quantitative research methods.
• Excellent communication and report-writing skills.
• Familiarity with the context and challenges of displaced populations and humanitarian response.
• A postgraduate qualification in Social Sciences, Journalism, Development studies or other related disciplines is also strongly required.
• Strong understanding of the South Sudan context, the humanitarian system, protection risks, and conflict dynamics in South Sudan.
• Willingness and ability to travel to remote field locations.
• Expected composition of the evaluation team one lead consultant, two field research coordinators, one data analyst, thirty data collectors/enumerators. This could be adjusted based on the capacity of the bidder or consultancy firm.
• Strong skills in qualitative (drill down approach) and quantitative data analysis.

Technical supervision

The selected consultant will work under the supervision of the BHA Project Manager with the technical support of the MEAL Manager or MEAL Coordinator.

DRC’s BHA Project Manager will lead the overall coordination and facilitation of the Performance Evaluation, including enumerator selection and hiring, providing lists of program sites and numbers of direct beneficiaries for all activities, sharing relevant program documents, advising about local protocols, making logistical arrangements, including accommodation and transportation for field data collection, connect the assessment team with the beneficiaries and key informants, arrange meetings between the consulting team and beneficiaries, community members, authorities, and community leaders.

Location and support

• The locations for the Performance Evaluation will include Rubkona and Guit in Unity state and Malakal, Fashoda counties of Upper Nile state

Travel

• DRC will facilitate only all in country travels to the fields related to this Performance Evaluation including accommodation however, the supplier shall provide for their own meals)
• Consultants contracted by DRC shall also be covered by their own travel insurance when travelling
• DRC, the organisation still has a responsibility and duty of care for the consultants, especially in contexts with a high Safety Level. The consultant is expected to follow and adhere to all governing Safety and Security procedures and rules The consultant firm is liable for their own liability insurance for any damage or injury the consultant may inflict on any persons or objects during the performance of work.

Evaluation of bids

• The RFP shall go under three stages of evaluation namely Administrative, Technical and Financial evaluation respectively.
• Only vendors that qualified at technical evaluation will be contacted for an interview with the panel to ensure their understanding of the consultancy services.

Please find complete bidding documents in the following link: RFP-SSD-JUB-2024-04 – Consultancy for conducting Impact Assessment-survey to measure the impact of the project on the targeted beneficiaries

How to apply

Bids can be submitted by email to the following dedicated, controlled, & secure email address: [email protected]

When Bids are emailed, the following conditions shall be complied with:

• The RFP number shall be inserted in the Subject Heading of the email
• Separate emails shall be used for the ‘Financial Bid’ and ‘Technical Bid’, and the Subject Heading of the email shall indicate which type the email contains
  • The Financial Bid shall only contain the financial bid form, Annex A.2
  • The Technical Bid shall contain all other documents required by the tender as mentioned in section A. Administrative Evaluation, but excluding any pricing information
• Bid documents required, shall be included as an attachment to the email in PDF, JPEG, TIF format, or the same type of files provided as a ZIP file. Documents in MS Word or excel formats, will result in the bid being disqualified.
• Email attachments shall not exceed 4MB; otherwise, the bidder shall send his bid in multiple emails.

Failure to comply with the above may disqualify the Bid.

DRC is not responsible for the failure of the Internet, network, server, or any other hardware, or software, used by either the Bidder or DRC in the processing of emails.

DRC is not responsible for the non-receipt of Bids submitted by email as part of the e-Tendering process

Bids can be submitted in one of two ways; hardcopy or electronically. If the Bidder submits a Bid in both Hardcopy and electronically, DRC will choose the version that is the most advantageous to DRC.