Consultant – Project Final Evaluation

1. BACKGROUND TO THE ASSIGNMENT:
International Medical Corps (IMC) is a global, humanitarian, non-profit organization dedicated to saving lives and relieving suffering through health care service delivery, training, relief, and development programs. IMC has been present in Yemen since 2012 addressing the critical needs of internally displaced (IDPs) and host communities affected by the protracted armed conflict in seven governorates— Sana’a, Ibb, Taiz, Hodaidah, Aden, Lahj, and Al Dhale’s. Key components of our lifesaving humanitarian assistance to affected populations include:

• Provision of integrated Health, Nutrition, and WASH services for affected populations in Yemen.
• Support hospitals, health facilities, and mobile health teams with pharmaceuticals and medical equipment, health information management, and financial support to healthcare workers.
• Provision of cash assistance to households with children suffering from severe or moderate acute malnutrition.

Since April 2022, IMC has been implementing a 25-month program in Taiz and Lahj governorates— funded by Global Affairs Canada (GAC) to reduce excess mortality and morbidity through the provision of integrated health and nutrition interventions to conflict-affected IDPs and Host Communities in the target locations. The project targets 69,964 individuals— (19,228 women, 19,833 men, 15,275 girls, 15,628 boys), including women of childbearing age, children under the age of five years (U5), the elderly, people with disabilities (PWDs), persons with special needs, and marginalized groups among other categories of most vulnerable people. In 2022, before program implementation, IMC conducted a thorough gender analysis (Annex XXX) as a basis for informing the program design and setting performance targets. A baseline assessment was also conducted (Annex XXX). A desk review of these Annexes, including the full proposal, and MEAL tools, among others will be shared with selected consultant only as essential to inform this end-line evaluation.

2. PURPOSE OF THE ASSIGNMENT:
The Evaluation will assess the entire program period since its inception in April 2022 and will assess its completeness and effectiveness, deliverables, and achievements realized, and take stock of attributed positive and unintended negative changes. Moreover, the Evaluation consultant shall refer (DAC) criteria to the relevance, effectiveness, efficiency, impact, coherence, and sustainability of the implemented project. Additionally, the evaluation should also assess gender equality outcomes. The project evaluation should include recommendations for further actions necessary in the field. Therefore, the Consultant is expected to provide information about success factors and recommendations for improvement areas.

3. SCOPE OF THE ASSIGNMENT:
The evaluation process will be directly implemented and managed by the selected consultancy/firm in collaboration with International Medical Corps Yemen. The MEAL Coordinator, with technical support and guidance from country sector Leads, Program Director, and the Technical Unit at HQ, will monitor its implementation and provide quality assurance throughout the process.
Key Roles and Responsibilities of the Consultant

• Design an appropriate evaluation methodology based on an understanding of the expectations of these terms of reference (ToR).
• Conduct extensive document reviews, meetings with IMC staff, perform data analysis, produce high-quality report, and possibly a learning meeting.
• Mobilize resources to implement the evaluation and achieve all its deliverables.
• Develop the end-line evaluation plan, including developing a detailed methodology and data collection and analysis tools. Incorporate feedback from the IMC as required. This will be submitted to IMC as an inception report that MUST be approved by IMC before proceeding with data collection.
• Conduct meetings with key IMC Program and Monitoring, Evaluation, Accountability & Learning (MEAL) staff to further understand the project and evaluation objectives.
• Undertake a detailed desk review of program documents including available secondary data.
• If a Yemeni based consultant or can travel to Yemen, the consultant will train enumerators and selected IMC staff who will be contributing to the evaluation. In this case, all costs related to hiring, training, allowances/salaries of the enumerators should be included in the consultant’s financial offer.
• If non-Yemeni consultant and tasks will be completed remotely, train IMC focal points who will then train hired enumerators with remote participation of the consultant. In this case, all costs related to hiring, training, allowances/salaries of the enumerators MUST not be included in the consultant’s financial offer. These costs would be covered separately by IMC.
• The consultant will lead field data collection, management, and analysis.
• Write and share inception and final reports that incorporate feedback from reviewers.
• Present key findings and recommendations to IMC senior management team.
• Final evaluation report that is presentable to the donor and key stakeholders, with all comments from the SMT team addressed.

Roles and Responsibilities of International Medical Corps

• IMC will be responsible for information sharing and arranging meetings of different stakeholders (excluding the beneficiaries) as will be determined in the inception report.
• IMC will also lead or support the hiring of local enumerators and as well as coordination with all sampled stakeholders including GHO, DHO, public figures, and communities.
• IMC teams will further provide quality assurance during fieldwork.

4. DELIVERABLES DESCRIPTION:

• Inception meeting
• Desk Review completed
• Draft Inception Report and data collection tools submitted to IMC
• Final inception report and data collection tools submitted to IMC
• Research assistants/Enumerators completed
• Quantitative and qualitative data Collection Completed
• Data Analysis and Draft first report
• Revision of the draft report and submission of the final version
• Presentation of Final findings/Report

5. IMPLEMENTATION OF THE ASSIGNMENT DELIVERABLES

Methodology
Overall Approach
A mixed method of qualitative and quantitative approach should be considered for this evaluation. However, the consultant will determine or advise on specific methods and tools appropriate to the local context to guarantee quality of data and the overall report. This will range from desk review of secondary data, structured questionnaire survey, and focused semi structured or guided interviews such as Focus Group Discussions and Key Informant Interviews (KIIs). Attempts should be made to collect data from different sources by different methods for each evaluation question and findings be triangulated to draw valid and reliable conclusions. The gender dimension should be considered as a cross-cutting concern throughout the methodology, deliverables, and final report of the evaluation.

Sampling Design and Sample size

Questionnaire Survey
The sampling frame is the entire catchment population of the targeted locations, estimated at 69,964, and the sample size should achieve a 95% confidence level and a 5% margin of error, appropriate design effect, and calculating the sample size for different study variables, with selection of the maximum sample size applicable. The consultant should develop a comprehensive sampling strategy using a statistically valid formula that collects representative data of the population of interest. The sample size should be gender-balance (50% female and 50% male).

Sampling for Key Informant Interviews

• IMC Program and Management (6 KII) ; Deputy Director- Health, Nutrition Coordinator, Program Coordinator, Program Manager (Health and Nutrition), 3 field staff involved in direct implementation of the program.
• Health facility-based Workers (6 KII): Doctors, Nurses, and midwife in Lahj governorate.
• Governmental stakeholders (3 KII): 1 GHO, and 3 DHOs in Lahj governorate.
• Community and IDPs leaders/ public figures(3 KII): 2 community and 1 IDP leaders in Lahj governorate.

Sampling for Focused Group Discussions

• 10 FGDs in Lahj with male and female both host communities and IDPs

Data Collection Methods and Tools
Selected consultant should provide an elaborate description of most appropriate data collection methods and tools to be used, with possible considerations of IMC’s initial tools used for baseline assessment.

Data Quality Management
Data quality and implementation quality assurance approaches are anticipated to be key part of the methodology. The selected consultant should provide an elaborate description of measures that will be taken to ensure data quality, including mitigating foreseen biases. Suggested measures include but not limited to ensuring that enumerators are properly trained and supervised, leveraging technologies and mobile-based platforms, data validation techniques, piloting data collection tools, and pretesting prior to fieldwork among others.

Data Analysis
Selected consultant should provide an elaborate description of methods and tools that will be applied to analysis statistical (quantitative) and qualitative data, including how results and findings will be presented.

Possible limitations based on context
Selected consultant should provide an elaborate description of limitations associated with the proposed methods, tools, and approaches, and measures that will be put in place mitigate the influence/impact of those limitations. Note, this section should not dwell on describing challenges!

Evaluation Criteria
The following broad DAC evaluation criteria must be addressed in the evaluation framework.
Relevance

• To what extent did the project respond to the diverse needs and priorities of women, men, and gender-diverse individuals within the target population?
• Were the project objectives aligned with the priorities of the donor and the specific health and nutrition challenges faced by the target communities?
• To what extent did the project consider gender disparities and promote gender equality within its health and nutrition interventions?
• Was climate change considered in project design?

Effectiveness

• To what extent were the set project outcomes and intermediate outcomes met, and what internal and external factors influenced the achievement or non-achievement of those outcomes?
• Did the project effectively address the specific health and nutrition challenges faced by women, children, and other vulnerable groups in conflict-affected areas?
• To what extent did the project contribute to addressing gender-related barriers? Do results (effects of activities and outputs) affect women, men, girls, and boys differently? If so, why and in which way?
• To what extent did the project incorporate gender-sensitive approaches to enhance the effectiveness of health and nutrition services and outcomes?
• To what extent did the project adopt environmentally- friendly strategies/solutions to achieve its objectives, there has been evidence exists regarding deliverables and achievement?

Efficiency

• To what extent demonstrated/delivered value for money? Have resources (financial, human, time, expertise, etc.) been allocated strategically and efficiently to achieve the expected results?
• Have the project funds and activities been delivered promptly? If not, why? What steps were taken to mitigate the problems?
• The extent to which the project has leveraged resources/collaborated with other projects?

Sustainability

• To what extent are the benefits of the project likely to be sustained after the completion of this project? How has the project supported the government and other partners in developing capacities and mechanisms to ensure ownership and scale-up of interventions?
• How effective were the exit strategies, and approaches to phase out assistance provided by the project including contributing factors and constraints – what are the recommendations for similar support in the future? (NB: The recommendations should provide comprehensive proposals for future interventions based on the current evaluation findings).
• Is the intervention transformative – does it create enduring changes in norms – including gender and disability norms – and systems, whether intended or not?

Coordination / Coherence

• How did the project take advantage of existing coordination mechanisms and complementarities/synergies with other partners in the field to avoid gaps, duplication, and conflicts?
• What gaps were identified and addressed by partners in the field?

Impact

• What were the intended and unintended positive and negative impacts of the project as perceived by beneficiaries?
• Did all the intended target groups, including the most disadvantaged and vulnerable, (women and people living with disabilities) benefit equally from the intervention?

Lessons learned and replicability

• What are some of the key lessons learned because of this project that can be shared and replicated?
• How was data, information, learning, or knowledge of the project managed during the project life?

Consultant Specifications
To perform this consultancy successfully, an individual must be able to perform each task with or without reasonable accommodation.

• An advanced university degree in Public Health and other research-focused field.
• Minimum of ten (10) years of field experience in leading multisectoral humanitarian evaluations— focused on health, and nutrition is required.
• Experience and background in gender-sensitive programming with sound knowledge of gender-sensitive evaluation methods.
• Strong skills in the collection and analysis of quantitative and qualitative data.
• Strong interpersonal communication skills.
• Ability to synthesize and interpret data.
• Excellent report writing skills.
• Fluency in English, while Arabic fluency is added advantage.
• Experience in Yemen and/or other countries in the Middle East preferred.
• Knowledge of tools for mobile data collection (Mainly Kobo, ONA).

Other Supporting Documents:

• Profile of the lead consultant and 2 other key staff highlighting the bidder’s qualifications and relevant experience.
• An outline of the evaluation plan. Please include if this evaluation will be completed remotely and/or using existing contacts in target locations within Yemen.
• A description of how the bidder will address the evaluation objectives and evaluation criteria.
• A detailed description of the evaluation methodology including sampling design, data collection methods and tools, data quality management, data analysis, reporting, and anticipated limitations.
• A detailed work plan with specific dates as to when key deliverables will be achieved.
• Include at least two sample reports from similar evaluations that were accomplished by the bidder in the last three years with particular emphasis on projects of similar scope and effort. The following information should be included in the reports:
  • Name of client
  • Title of the project
  • Locations
  • Year the evaluation was completed:
  • Summary of the work completed:
  • References/contact details (emails and telephone numbers) of organizations where the consultancy was completed.

Code of Conduct

As applicable to this position, an individual must promote and encourage a culture of compliance and ethics throughout the organization and maintain a clear understanding of International Medical Corps’ and donor compliance and ethics standards and adheres to those standards. Staff are also responsible for preventing violations to our Code of Conduct and Ethics, which may involve Conflicts of Interest, Fraud, Corruption or Harassment. If you see, hear or are made aware of any violations to the Code of Conduct and Ethics or Safeguarding Policy, you have an obligation to report. If this is a supervisory position, one must set an example of ethical behavior through one’s own conduct and oversight of the work of others; ensure that those who report to you have sufficient knowledge and resources to follow the standards outlined in the Code of Conduct & Ethics; monitor compliance of the people you supervise; enforce the Code of Conduct & Ethics and International Medical Corps’ policies, including the Safeguarding Policy and the Protection from Harassment, Bullying and Sexual Misconduct in the Workplace Policy, consistently and fairly; support employees who in good faith raise questions or concerns.

Safeguarding

It is all staff shared responsibility and obligation to safeguard and protect populations with whom we work, including adults who may be particularly vulnerable and children. This includes safeguarding from the following conduct by our staff or partners: sexual exploitation and abuse; exploitation, neglect, or abuse of children, adults at risk, or LGBTI individuals; and any form of trafficking in persons.

Equal Opportunities

International Medical Corps is proud to provide equal employment opportunities to all employees and qualified applicants without regard to race, color, religion, sex, sexual orientation, national or ethnic origin, age, disability or status as a veteran.