Consultant- FINAL EVALUATION FOR ECHO FUNDED PROJECT

TERMS OF REFERENCE FOR A FINAL EVALUATION

FOR ECHO FUNDED PROJECT

Project Title: Comprehensive emergency physical rehabilitation, psychosocial support, mine risk education services to vulnerable communities (including those affected by Herat earthquake, IDP, returnees in Herat, Kunduz, Nimroz, Kandahar, Nangarhar, Kabul) and provision of support for Disability Inclusion Working Group.

1. About Humanity and Inclusion (HI)

Handicap International/ Humanity & Inclusion (HI) is an international organization specialized in the field of disability and inclusion. As a non-governmental, non-religious, non-political and non-profitmaking organization, it works alongside persons with disabilities, whatever the context, in response to humanitarian crises and the effects of extreme poverty. HI implements programs of assistance to persons and local organizations, mainly on inclusion and disability thematic. HI runs projects in almost 60 countries, with the support of a network of 8 national associations (Germany, Belgium, Canada, United-States, Luxembourg, United Kingdom and Switzerland). The organization employs almost 3300 people worldwide, 330 of whom work in France and in its European and North American sections.

HI has been present in Afghanistan since 1996, supporting the most vulnerable communities across the country with over 500 staff in Kandahar, Nimroz, Herat, Kunduz, Nangarhar, Bamyan and Kabul. HI has been delivering humanitarian and longer-term recovery programs ensuring access to basic services for displaced populations, host communities and returnees. The key areas of HI intervention include physical rehabilitation, training of professional physiotherapists, psychosocial support, and socio-economic inclusion through inclusive livelihood activities, mine risk education, disability mainstreaming, logistic services and advocacy. These activities have not only focused on the provision of services but also on mainstreaming a right-based approach and ensuring that disability does not remain a separate and stand-alone work stream, especially in emergency situations.

2. Project background

Considering the needs and gaps, HI has implemented a project untitled “Comprehensive emergency physical rehabilitation, psychosocial support, mine risk education services to vulnerable communities (including those affected by Herat earthquake, IDPs, returnees in Herat, Kunduz, Nimroz, Kandahar, Nangarhar, Kabul) and provision of support for Disability Inclusion Working Group”. The aim is to respond to humanitarian needs for IDPs, refugees, and host communities, especially people with disabilities in the five provinces. In particular, the project was implemented in Herat, Kunduz, Nimroz, Kandahar, Nangarhar and Kabul provinces. The project responded to different sectorial needs such as Protection (EORE), Health (Rehabilitation & MHPSS), Inclusive Humanitarian Action (IHA) andlogistical support services for IDPs, refugees and host communities specially for people with disabilities in coordination with the relevant humanitarian clusters, working groups and government duty-bearers.

The action was formulated to achieve the objective of reducing vulnerabilities and critical health risks of highly vulnerable women, girls, men and boys, with a specific focus on people with disabilities in order to prevent long-term physical and/or psychosocial disabilities in the

project target locations. The project was implemented from 1st November 2021 to 30th June 2024.

Project’s objectives and expected results:

Overall objective: To mitigate and reduce vulnerabilities and critical health risks of highly vulnerable women, girls, men and boys, with a specific focus on people with disabilities in order to prevent long-term physical and/or psychosocial disabilities

Specific objective:

1. To ensure the availability and increase sustainability of critical and comprehensive rehabilitation and PSS services relieving the impact of physical impairments and psychological distress for CW/IED survivors and other persons with physical impairments living in conflict and crises-affected areas of Kandahar, Herat, Kunduz, and Nimroz.
2. Reduce vulnerability of people, especially children, exposed to the risks of death and injury from mines, IEDs and ERW through risk awareness sessions targeting IDPs, Returnees and host communities
3. Increase capacity of humanitarian actors through Disability Inclusion Working Group and increase understanding about disability and the human rights approach to disability.

Expected results

Expected result 1: Quality, comprehensive and accessible rehabilitation services are available for CW/IEDs survivors, occupational trauma, and other people with disabilities at the Kandahar Physical Rehabilitation Centre

Expected result 2: Increased access to rehabilitation care services and to PSS services is enhanced through scaled-up deployment of emergency Project teams equipped with dedicated outreach social workers, and physical rehabilitation and positive coping mechanisms for IDPs, returnees, conflict and earthquake-affected host communities are strengthened in Kabul, Kunduz, Kandahar, Herat, Nimroz and Nangarhar Provinces

Expected result 3: The vulnerability of people, especially children, exposed to the risks of death and injury from explosive devices including mines, IEDs and ERW is reduced through Project teams’ risk awareness sessions targeting IDPs, returnees, and host communities.

Expected result 4: The access to basic and/or specialised services for IDPs, returnees and host community members in need is improved through the provision of cash assistance and through robust referral pathways, including sensitisation on inclusive humanitarian approach and sharing of technical standards on specialised services for public and private health service providers

Expected result 5: Inclusive Humanitarian Action (IHA) is promoted through capacity building of partners and the strategic development of the Disability Inclusion Working Group (DIWG) at sectoral level.

Project Summary

Project Title: Comprehensive emergency physical rehabilitation, psychosocial support, mine risk education services to vulnerable communities (including those affected by Herat earthquake, IDP, returnees in Herat, Kunduz, Nimroz, Kandahar, Nangarhar, Kabul) and provision of support for Disability Inclusion Working Group

Duration: 1st November 2021 – 30th June 2024

Sectors: Health (rehabilitation & MHPSS), Inclusive Humanitarian Action (IHA) & Protection (EORE)

Partners: N/A

Target Area: Provinces: Herat, Kunduz, Nimroz, Kandahar, Nangarhar and Kabul

Districts in Herat: Herat city, Injil, Zindajan, Karukh, Guzara, Kushk Robat Sangi, Kohsan.

Districts in Kunduz: Kunduz city, Emam Sahib, Khan Abad, Ali Abad, Chahar Dara.

Districts in Kandahar: Kandahar city, Dand, Daman, Arghandab, Panjwai, Zhari, Maiwand.

Districts in Nimroz: Zaranj City

Districts in Kabul: Kabul City

Districts in Nangrahar: Jalalabad city, Momand Dara (Torkham)

Target Groups: Most vulnerable and marginalised people including IDP, refugees, returnees people with disabilities and vulnerable host community. Estimated number of direct beneficiaries = 533,763 individuals (details to be provided for inception phase)

3. Justification of the evaluation

The purpose of this final evaluation is to establish the extent to which the project has achieved its aims and objectives and to determine the relevance, coherence, effectiveness, efficiency, impact and sustainability of project outcomes. The knowledge generated by the evaluation will also feed in the design of relevant future intervention and contribute to documenting management and delivery approaches.

4. Scope of the evaluation

The evaluation will be conducted in the five provinces as mentioned above. The consultant is expected to travel to project locations of the field work component of the consultancy. It is expected that consultant work will take a total of 45 working days spread over July and August 2024.

The consultancy firm will be required to undertake the following specific tasks:

• In consultation with HI, the consultant will be expected to design standardized tools and finalize methodology for the collection of quantitative and qualitative data for the final evaluation, taking into account the objectives mentioned in the TOR.
• The consultant will share an inception report about the consultancy.
• The consultant will collect and analyze available secondary data related to the project objectives.
• The consultant will conduct primary data collection in targeted project locations in the five provinces to measure key indicators.
• The consultant will analyze and interpret data to develop a comprehensive final evaluation report, learning briefs and success stories.
• The consultant will share key findings and insights with the client staff and stakeholders through presentations.
• The consultant will demonstrate accountability for the funding received from ECHO.

5. Overall objective of evaluation

The overall aim is to assess the project’s level of achievement and the extent to which it brought positive changes and added value for the beneficiaries and other relevant stakeholders. This summative evaluation will also assess the quality of project approaches, strategies and interventions as stated in HI Project Quality Framework. Furthermore, this evaluation will take into account the constraints and challenges faced during project implementation, as well as adaptations made in response to the earthquake in Herat in October 2023 and the influx of Afghan refugees returning from Pakistan, which led to project response in Nangarhar province. The consultant also needs to consider the change in implementation method from mobile teams to static centers that occurred in August 2023 based on the DFA instruction. In addition, the evaluation will analyze the impact of project implementation to identify activities that have worked well and/ or need to improve in the future as part of lessons learned document of the project.

Specifically, the evaluation aims to:

• Analyze the extent to which the project has contributed to bringing about medium and long-term positive effects in the target population.
• To examine and analyze the achievements of project expected results against targets.
• To document any lessons learned and changes within and outside the project that impacted project delivery, impacts and sustainability of benefits to targeted beneficiaries.

Finally, the final evaluation will provide the donor (ECHO) with sufficient information to:

• Assess and analyze the project results, efficiency and relevance of the implementation strategy.
• Review, analyze, provide evidence and document the impact of the intervention through its progress, challenges, barriers and lessons learned with recommendations for a logical and potential next phase.

6. Evaluation criteria and questions

Evaluation questions should be developed/revised as per below evaluation criteria. The consultant will articulate the analysis around a set of evaluation questions related to Changes, Efficiency, Relevance, Effectiveness and Sustainability. Some questions are listed below. These evaluation criteria and questions are not exhaustive and will be reviewed by the consultant at the beginning of the inception report stage and planning. The following criteria and questions should be looked into, though other criteria can be suggested by the consultant:

Relevance/Appropriateness: appropriateness of project objectives to the problems it was supposed to address, including assessment of the quality of the project preparation and design. Specifically:

• Does the project meet the needs of the target population?
• Is the project design appropriate to the specific context?
• Are the mechanisms and approaches developed in coherence with existing plans and policies of HI?
• Were there agreed structures and respected timeframes in place to investigate and resolve complaints? How effectively were complaints received and dealt with?

Coherence: The need to assess security, developmental, trade and military policies, as well as humanitarian policies, to ensure that there is consistency and, in particular, that all policies take into account humanitarian and human-rights considerations.’

• Was the intervention informed by and aligned with the fundamental humanitarian principles and the CHS?
• Was the Red Cross Code of Conduct respected?

Connectedness: refers to the need to ensure that activities of a short-term emergency nature are carried out in a context that takes longer-term and interconnected problems into account.’

• How were longer-term problems given consideration during implementation of emergency activities?
• Partnerships: how they came into being and were supported, including the extent to which local capacity was supported and developed.

Effectiveness: refers specific assessment of the benefits accruing to target groups:

• To what extent does the project achieve the expected results?
• Did the project contribute to improvement of beneficiary wellbeing and enhance capacities on inclusive humanitarian action?
• How does the team adjust the project and its methodologies to the constraints faced during the implementation phase to achieve the expected results?
• Do the project’s activities designed and implemented address the needs of both men and women on an equal basis?
• Did the project’s implementation have any unexpected consequences which should be considered in the design of future projects with a similar program logic
• What internal and external factors affected (positively or negatively) the effectiveness of the project?
• What strategies were employed to adapt the multi-purpose cash interventions to the specific needs of the target population?

Efficiency: refers comparing alternative approaches to achieving the same results, to see whether the most efficient process has been adopted:

• Are the methodologies and tools appropriate to meet the project’s objective?
• Is the organizations and project team profile efficient for implementing the project in the area covered?
• Were there any barriers to the efficient implementation of the key activities?
• Did the strategy and approaching method enable the achievement of the results in a cost-effective manner?
• Have activities and results been cost efficient compared with results? Could the results and activities have been attained with fewer resources?
• To what extent has the project design, planning and implementation taken gender integration into consideration.

Impact/Outcome: looks at the wider effects of the project – social, economic, technical, environmental – on individuals, gender- and age-groups, communities and institutions. Impacts can be intended and unintended, positive and negative, macro (sector) and micro (household).’To what extent can it be said that the effects/impacts are attributable to project interventions? Are there other external factors which have played a role in the effects/impacts during the project period?

• For those effects/ impacts which are attributed to the project, what have been the processes, component or qualities of the project intervention which have led to the change?
• What modification/reorientation should be made in areas of intervention and activities to better achieve the expected effects/ impacts? The evaluator should consider both the short term and the longer-term directions for future strategy.
• Did the project have any negative impacts or is it likely to have in future?
• What is the project’s impact on its wider environment (social level impact, policy level impact, economic level impact, technical level impact)?

Sustainability: assess the likelihood of benefits produced by the project to continue to flow after external funding has ended, and with reference to:

• What were the major factors influencing the project’s achievement or non-achievement of sustainability?
• To what extent is the sustainability of outcomes, contingent on economic self-reliance (or vice versa)?
• How have the approaches applied contributed to the sustainability of the project achievements?
• To what extent are the changes in outcomes of individuals under this intervention still being felt/observed after exit from direct support? What factors have facilitated or hindered sustainability of outcomes?

Additionally, evaluation criteria will also include partnership, compliance, gender, inclusion and accountability to affected population.

7. Methodology

The project implemented a participatory monitoring system based on HI’s Project Quality Policy (PQP) which is been developed during the inception period of the project. The evaluation methodology is expected to review the PQP, as far as possible, and provide data for the project indicator targets which will allow comparability with baseline. The evaluation methodology will be further developed by the consultant / evaluator(s) and should be well articulated in the response to the ToR. The methodology should demonstrate robustness and rigor in addressing the evaluation objectives and questions. Interested consultants will be expected to propose the design and methodology to be used in this evaluation as part of their proposals.

The methodology proposed by the consultant shall be discussed, finalized and approved by the evaluation steering committee before commencing data collection as part of the quality assurance process. However, the methodology is expected to use a mixed-methods approach including qualitative and quantitative components. The data collection methodologies should use a participatory approach engaging all relevant stakeholders, including community leaders, private sector, government counterparts and partners. The consultant should demonstrate how they will conduct document and system reviews, household surveys, FGDs, KIIs and field visits/observations as appropriate. Moreover the evaluation team will assess integration of crosscutting issues in particular gender, disability and accountability to affected population.

8. Stakeholders involved in the evaluation

HI Operations Manager, Technical Head of Programme, Emergency Program Manager, Technical Specialists, Inclusive Governance Technical Officer, Grants Specialist, Project Managers, MEAL Managers, Beneficiaries, Local authorities, IHA Specialist and other key stakeholders will be involved in the process. A team coordinator will be appointed. The team coordinator will be ultimately responsible for the entire work and all deliveries; s/he will also act as the contact person for the evaluator.

9. Ethics

HI is committed to ensuring that the rights of those participating in data collection or analysis are respected and protected. All applicants should include details in their proposal on how they will ensure ethics and child protection in the data collection process. The evaluation team will need to demonstrate the following:

• How they have considered the protection of children and vulnerable groups (including persons living with disabilities).
• How appropriate, safe, non-discriminatory participation of all stakeholders will be ensured through the different evaluation stages, including recruitment and training of research staff, data collection, data analysis and report writing.
• How confidentiality and anonymity of participants will be guaranteed.
• Experience of putting safeguarding approaches and procedures in place, as well as how they ensure compliance from their staff (including enumerators), at all times.

Research ethics plan: bidders are required to set out their approach to ensuring complete compliance with international good practice with regards to research ethics and protocols particularly with regards to safeguarding children and vulnerable groups (including girls and people with disabilities). Consideration should be given to:

• Administrative, technical and physical safeguards to protect the confidentiality of those participating in research
• Safeguards for those conducting research
• Child-safe physical safeguards for children participating in research
• Adherence to good practice guidelines on researching violence against women and girls
• Appropriate time allocated to engage with children participating in the research
• Data protection protocols and secure maintenance procedures for personal information
• Parental consent concerning data collection from children or collation of data about children
• Age- and ability-appropriate assent processes based on reasonable assumptions about comprehension for the ages of children and the disabilities they intend to involve in the research
• Appropriate spaces and methodologies tailored in consideration of unique needs of girls and boys, including those with disabilities
• Appropriate language and communication for different ages and the disabilities of children involved in the research
• Age-appropriate participation of girls, including in the development of data collection tools

10. Data Quality Assurance

Quality assurance plan: The bidder is required to submit a quality assurance plan that sets out the systems and processes for quality assuring the evaluation process and deliverables of the project, from start to finish. This plan should include the proposed approaches to:

• Piloting of all research activities
• Training of enumerators and researchers conducting the mixed-methods primary research, including in research ethics
• Logistical and management planning
• Field work protocols and data verification including back-checking and quality control by supervisors
• Data cleaning and editing before any analysis
• Only work that meets these standards will be signed off.

11. Deliverables

In accordance with the timetable, the consultant will produce:

• An inception report refining the proposed evaluation design, methodology, survey instruments, detail work plan. This inception report will have to be validated by the Steering Committee.
• A field work implementation plan (to be submitted before field work begins). This field work plan should be presented to HI for comments and revised as necessary prior to commencing field work.
• A comprehensive draft evaluation report of approximately 30-40 pages maximum including a clear set of actionable recommendations.
• A final evaluation report incorporating feedback from HI, implementing partners and key stakeholders
• Draft learning report: draft of 2-3 pages lessons learned brief report and at least 3 success story reports.
• Summary presentation of findings to HI and stakeholders: the Consultant shall present initial findings to HI for review, comment and feedback as per agreed timeline. A PowerPoint presentation and brief report (maximum of 2-3 pages) shall be prepared for the presentation through a workshop to the project stakeholders preferably via in person or online platform. The Consultant shall consider HI and stakeholder comments and revise the draft report as appropriate.
• Annexes including interview notes and list of resource documents, lists of interviewed people, work plan, data collection tools, outputs of statistical analysis, lists of documents reviewed, presentations. The Consultant shall provide HI summaries of all key meetings, and discussions conducted during the course of the evaluation and copies of any relevant documents and reports gathered during the evaluation period.
• Raw collected data and cleaned datasets: the consultant’s needs to submit the data generated in the process of this study in excel as well as other field related photos during field observation in a well-organized format (revised, reformatted, if required) that can be easily used.

HI will oversee the process and be responsible for accountability and guidance throughout all phases of execution, and approval of all deliverables.

12. Key documents of reference

The evaluation team should also refer to relevant country data and information that is currently available, including previous evaluation reports of another projects, quarterly reports, log frame, MEAL Plan, project proposal, as required, to prepare the survey proposal.

13. Timeframe

The consultant will be required to undertake the exercise within a period of 45 working days after signing the agreement which will be from 21st July-12th September 2024.

Timetable for evaluation:

Tasks–> Deadline–> Responsible–>

Develop inception report and submit to HI–>3rd week of July 2024–>Consultant

Collect and analyze available secondary data related to project objectives–>3rd week of July 2024–>Consultant

Develop evaluation tools covering the require scope of work as well as brief field work implementation plan and submit an inception report to HI–>1st week of Aug 2024–>Consultant

Conduct primary data collection in targeted project locations–>2nd & 3rd week of Aug 2024–>Consultant

Draft evaluation report and submit to HI for comments–>4rth week of Aug & 1st week of Sept 2024–>Consultant

HI to provide comments and feedback–>1st week of Sept 2024–>HI

Incorporate comments, presentation and submit final version of the report to HI–> 2nd week of Sept 2024–>Consultant / MEAL team.

14. Consultant’s qualifications and requirements

The survey consultant or firm should meet the following requirements:

• Extensive expertise in evaluations of complex programmes in humanitarian contexts and minimum of 5+ years of experience in conducting evaluation studies with particular experience in people with disabilities, returnees, IDPs and host community context for integrated programmes that include inclusive health (physical rehabilitation, MHPSS), mine risk education and disability inclusion sectors.
• Skills and experience in quantitative and qualitative data collection and analysis, drawing findings from multiple sources and handling potential contradictions between data sets
• Gender, age and disability sensitive research methods and experience measuring changes in gender attitudes
• Proven experience in conducting project endline evaluation studies with international humanitarian organizations.
• Experience in conducting research/studies on people living with disabilities, including children: experience with the Washington Group methodology would be particularly appreciated
• Having qualified staff with advanced degree in social sciences, health sciences, statistics, economics or another relevant academic discipline with extensive experience in assessing and evaluating projects in humanitarian fields.
• Proven experience of using participatory methods for data collection and analysis.
• Knowledge of ICT tools for mobile data collection
• Strong experience on conducting assessments for health, projection and disability mainstreaming in similar contexts.
• Excellent spoken and written communication skills in English. The communication between HI and the consultant will be in English and the report should be prepared in English language, however, the evaluation will be conducted using local languages (Dari and Pashto) and therefore, knowledge of languages of the local communities (Dari and Pashto) for the data collectors is a must.
• Excellent skills and ability to articulate ideas in a clear and accurate manner including the ability to prepare quality reports. Good data presentation and visualization skills.
• Strong interpersonal skills, analytical skills and ability to establish and maintain effective working relations
• An evaluation team should demonstrate technical expertise including health and protection background and skills, designing of evaluation methodologies and previous proven skills and experience in undertaking evaluations of similar projects;
• Excellent knowledge of the Afghanistan context, especially in terms of security, and culture is required
• Strong knowledge of Core Humanitarian Standards
• Practical experience on disability inclusion, gender issues and gender integration analysis
• The consultancy firm / individual consultant must be registered under the law of the country of the assignment.
• Working experience in the study locations is an added advantage.

15. Survey budget and payment

Financial proposal must be indicated all-inclusive costs for conducting survey. Payment will be made once the final evaluation reports and other deliverables have been validated and provided withing the schedule and deadlines defined together, and the quality criteria referred to in the contract have been met.

16. Submission requirements

Proposals from interested consultant(s) should submit the following documents:

• Technical proposal with proposed evaluation design and methodology based on project needs outlined by this ToR;
• Financial proposal: all costs related to the evaluation without exceptions should be figured into the financial plan of the consultant, including consultancy fees, domestic and international travel, visa, accommodation, and per diem. The applicant must integrate all expenses related to the study which should include flights, logistics, organization of workshops etc. All the costs should be quoted in US Dollar. These should be integrated within the financial application.
• Proposed activities and detail timetable (considering that contextual limitations will later be communicated by project team);
• Up-to-date registration certificate (copy) with the Government of Afghanistan;
• Consulting Firm’s capacity statement describing how they meet required qualifications and competencies including summary of similar studies previously conducted
• Evaluation of the applications/proposals will be made through a selection committee only if a complete application is received. Criteria to select the best application will be based on quality of the methodology, human resources dedicated to the study, realistic work plan, previous experiences and sample of similar reports, demonstrated expertise of the applicant, and competitive financial proposition.
• Skills and experience of the consultant, including key members lists with core qualifications and their role in evaluation (attach the CVs of these key team members involved in the evaluation)
• CVs of each member of the Evaluation Survey Team (no more than 3 pages), detailing relevant skills, core qualifications, experience and their role in evaluation exercise.
• Three samples of evaluation reports from previous evaluations relevant to this study.

Please submit your technical, financial proposals and other required documents to (to be included). The deadline for submission of proposal is 13th July 2024 by the end of business hours (5:30pm) Afghanistan time. Consultant/firm should write “ECHO Final Evaluation” in the subject line of email when submit their application. Applications submitted after the deadline will not be accepted.

How to apply

Submission Address:

All quotations must be submitted sealed in Hard Copy on or before 13th July 2024 by 5:30 pm Kabul time at the Handicap International/ Humanity & Inclusion (HI) office.

Address: Handicap International (HI): House 431, street 8 Taimani, Kabul city Afghanistan

Tel.: +93(0) 745 153 485; +93(0) 700 680 030