End-line Survey and Final Evaluation of Community-Based Eco Disaster Risk Reduction Phase II: Climate Resilient Livelihoods Project

The Organization

Afghanaid is a British humanitarian and development organization operating in Afghanistan. For close to forty years, we have worked with millions of deprived, excluded, and vulnerable families in some of the poorest and most remote communities in Afghanistan. We build basic services, improve livelihoods, and strengthen the rights of women and children, help communities protect against natural disasters, adapt to climate change, and respond to humanitarian emergencies.

Our community-led approach gives ordinary Afghans a voice in their own development, making them active participants in shaping the future of their country. With our years of experience, the majority of the Afghan team, and our deep understanding of local, cultural, and ethnic issues, we have earned great trust and respect among the communities we work alongside. This has allowed us access to some of the most underserved areas of the country.

The Project

Afghanistan is especially susceptible to natural disasters and ranks at the top of the Ecological Threat Register as the country most affected by ecological threats including droughts, floods, increasing temperatures, and food insecurity. These hazards harm people, lands and infrastructure alike, wiping out family, community and national development gains, devastating agricultural productivity, and holding back the future development and job creation that would allow for families and communities to mitigate hazards, build resilience, and lift themselves out of poverty.

To address this complex set of challenges, Afghanaid and its partners (ActionAid, Concern Worldwide, and Save the Children) in the Afghanistan Resilience Consortium (ARC) have implemented the Community-Based Eco-DRR (CBED) project since 2018. The CBED project is implemented in Afghanistan’s 245 communities in 6 provinces (Takhar, Badakhshan, Samangan, Jawzjan, Sar-e-Pul and Bamyan). The first phase of the project (CBED-I), implemented between November 2018 and April 2021, focused on improving local ecosystems by undertaking Integrated Watershed Management (IWM) in 30 micro-watersheds and Community-Based Disaster Risk Reduction (CBDRR) in 245 communities. The project’s second phase (CBED-II) is being implemented from April 2022 to the end of December 2024 and focuses on improving the Livelihoods of vulnerable households in the 245 communities.

Summary of Project Outcomes and Outputs

The proposed impact of CBED-II is: Afghanistan’s vulnerable communities and households are more resilient to and able to mitigate the impacts of climatic and economic shocks

CBED-II has two inter-related outcomes, which are:

Outcome 1: Rural livelihoods are better adapted and more resilient to climate change and natural disasters.

Outcome 2: Rural communities benefit from improved stewardship and usage of natural resources.

The outcomes are achieved through the following outputs:

Output 1: Vulnerable households are supported to strengthen food security and recover from climatic and economic shocks.

Output 2: Men and women are supported to increase production, productivity, quality, and diversity of on- and off-farm products.

Output 3: Men and women are supported to improve value addition and access to markets and services for on- and off-farm products.

Output 4: Key stakeholders gain knowledge and skills in climate adaptation.

Purpose of the assignment

The assignment has the following objectives:

1. To collect and analyse end-line data to assess the project’s progress and in relation to the baseline indicators and evaluate the overall impact and effectiveness of the CBED-II project in achieving its intended outcomes related to climate change adaptation and disaster risk reduction at the community level (please refer to Section 2.1)
2. Evaluation and documentation of the CBED’s Climate Change and Adaptation Model (please refer to Section 2.2)
3. Provide exposure opportunities and tailored training for Afghanaid’s M&E staff (please refer to Section 2.3)
4. Submission of a comprehensive photo reports, documenting visual evidence of project outcomes and impacts (please refer to Section 2.4)

a) End-line data collection and evaluation

This consultancy assignment focuses on conducting an end-line data collection to gather comprehensive data, determining the final values of the project’s outputs, outcomes, and impacts as specified in the project’s logical framework. The end-line study will concentrate on collecting and analysing data to assess the project’s progress and in relation to the baseline indicators. This will involve a rigorous comparison of current data and outcomes against the initial baseline measures, providing a robust basis for measuring the project’s effectiveness and the extent of change that has occurred since the project’s inception.

This information will act as a primary data source for further project final evaluation of the project. Impact and outcome indicators of the project are listed below as an indication – the consultant will be provided with the full set of indicators and other relevant documentation after signing of the contract:

Impact Indicator: % households with 50% improvement household coping capacity according to the Coping Strategy Index in target communities against the baseline

Outcome Indicator 1.1: % households meeting vulnerability criteria having increased food security

Outcome Indicator 1.2: % increase in productivity and production of crops and livestock achieved in CBED communities

Outcome Indicator 1.3: % improvement in income generation from various livelihood interventions in CBED communities

Outcome Indicator 1.4: Reduced % of households whose livelihoods were disrupted were disrupted or destroyed due to disaster in target communities

Outcome Indicator 2.1: % increase in enhanced ecosystem service benefits in land under improved stewardship as a result of CBED-I&II interventions

Outcome Indicator 2.2: % increase in capacity of the community institutions (CBDMCs/WMCs/Producer Groups/VFUs/ASCs) measured by maturity index

Outcome Indicator 2.3: % Increase in frequency and quality of agricultural and veterinary advisory services received by the community members

This evaluation will identify and document best practices and capture valuable lessons learned throughout the project’s lifecycle highlighting areas where the project performed less effectively than anticipated. The findings of the evaluation will contribute to effective programming, refining the approaches, organizational learning, and accountability.

The evaluation aims to provide evidence-based insights into the outcomes and impacts of the project, which will inform future decision-making processes and contribute to the improvement of project design. The evaluation will also generate actionable recommendations to enhance the effectiveness and efficiency of similar initiatives in the future.

The evaluation will involve a participatory approach, engaging key stakeholders, including project staff, project participants, local de-facto authorities, and partner organisations, to ensure a comprehensive and inclusive assessment. The findings from the evaluation will be disseminated through detailed reports, presentations, and workshops, to ensure the knowledge gained is accessible and utilized for continuous improvement and capacity building within the organisation and among its partners.

Additionally, the final evaluation will utilise the data being gathered by the CBED-II end-line evaluation as well as Impact Study which is currently being conducted. The end-line evaluation will collect comprehensive data to determine the final values of the project’s outputs, outcomes, and impacts as specified in the project’s logical framework. This study is focused on collecting and analyzing data to assess the project’s progress in relation to the baseline indicators. The Impact Study will provide understanding of the livelihood impact of the community-level infrastructures implemented by the Community Based Eco-DRR (CBED) project.

b) Evaluation and Documentation of the CBED’s Climate Change and Adaptation Model

In addition to the current phase of the project, the evaluation will also encompass the Phase-I of the project (CBED-I), which was implemented from 2018 to 2021. This historical perspective will provide a comprehensive understanding of the project’s overall trajectory and long-term impacts as well as highlight the continuity, progress, and evolution of the project over time, offering deeper insights into its development and sustained benefits.

The primary objective of this evaluation is to conduct a thorough assessment and documentation of the CBED’s Climate Change and Adaptation model implemented through CBED-I and CBED-II. This will include evaluating its effectiveness, identifying key components that contributed to its success or limitations, and understanding how well it integrated with the local context and the needs of the target beneficiaries. Additionally, the evaluation will explore the innovative practices and approaches introduced by the model and their impact, as well as extract lessons learned to inform future climate change adaptation initiatives.

Key Guiding Evaluation Questions

1. How effective was the CBED’s Climate Change and Adaptation model in achieving its intended outcomes?

2. What were the key components of the model that contributed to its success or limitations?

3. How well did the model integrate with the local context and the needs of the target beneficiaries?

4. What innovative practices and approaches were introduced through the model, and what impact did they have?

5. What lessons can be learned from the implementation of the model for future climate change adaptation initiatives?

   c) Exposure opportunities and tailored training for Afghanaid’s M&E staff

In addition, the consultant will provide exposure opportunities and tailored training for Afghanaid’s M&E staff. The consultant will include Afghanaid M&E staff in all steps of the end-line process and final project evaluation; and Afghanaid M&E staff will participate in the consultant’s team meetings and decision-making processes for their exposure. In addition, the consultant will develop and facilitate training sessions for Afghanaid’s M&E team on every phase of the end-line/evaluation process, including, but not limited to, tool development, data collection, data cleaning and analysis, and reporting; the materials developed for these sessions (PowerPoint presentations as well as any other documents) will form part of the consultant’s deliverables.

d) Photo Reports

1. The end-line evaluation photo report

The end-line project evaluation necessitates the submission of a comprehensive photo report, documenting visual evidence of project outcomes and impacts. The parameters for the photo report are given below.

This report is tasked with visually documenting the tangible outcomes and profound impacts that have materialized as a result of the project’s implementation.

Key elements of the comprehensive photo report include:

1. The report should vividly showcase project successes and achievements. This includes after the intervention photographs illustrating how project interventions have transformed communities, landscapes, and livelihoods. For example, photographs can capture degraded landscapes into thriving ecosystems through watershed management initiatives, the evolution of kitchen gardens into sources of fresh produce, or the growth of bio-briquette production facilities.
2. Photographs should highlight the active engagement of project participants. This can encompass images of community members actively participating in project activities, such as tending to crops and livestock, or attending training sessions. These images help underscore the community-driven approach of the project.
3. The photo report should visually convey the impact of the project on vulnerable households, including their improved living conditions, increased access to food and resources, and enhanced resilience to climatic and economic shocks. This can involve images of families with healthier and more abundant harvests, households utilizing solar pumps for improved water management, and beneficiaries proudly showcasing their kitchen gardens or bio-briquette production.
4. Highlight the sense of community collaboration and collective efforts that have been fostered by the project. Capture moments of collaboration, dialogue, and cooperation among community members, as well as their active involvement in disaster risk reduction and climate resilience activities.
5. Emphasize gender inclusivity by capturing the active participation and leadership of women in project activities. This can include photographs of women farmers working in their kitchen gardens, women-led poultry initiatives, and women actively engaged in community-based disaster risk reduction efforts.
6. Document the construction, rehabilitation, or establishment of project infrastructure and facilities. This may include images of flood protection walls, irrigation canals, agro-service centers, veterinary units, and bio-briquette production facilities, highlighting their functionality and impact on communities.
7. Ensure clear, high-resolution images suitable for digital use.
8. Provide date and location stamps for each photograph. Add descriptive captions for context and identification.
9. Obtain consent for individuals appearing in the photographs
10. Represent diverse genders, ages, and ethnicities in the photos.

2. Photo report/photographs for project exhibition

Upon submission of the report, our team will select and curate a collection of photographs to display at a photo exhibition event planned for November 2024. This event aims to showcase community-led climate resilience efforts, sustainable practices, and innovative solutions from the CBED project. Therefore, the evaluation team must grasp the specific content that the team intends to communicate at the event.

The evaluation team will visit and photograph 30 watersheds in six target provinces to showcase the work done in managing the watersheds. The photo report will visually document the impact of the CBED project across three key themes: 1. Watershed Management Work, 2. Strengthened and Resilient Livelihoods, 3. Women in CBED.

The goal is to create a compelling visual narrative highlighting the project’s successes, challenges, and the human stories behind the data

1. Watershed Management Work

Focus Areas:

• Capture the transformation of landscapes through watershed management efforts.
• Document how these changes have mitigated climate change risks and improved livelihoods
• Examine how the impact of Watershed Management Work differs among various provinces, and underscore the similarities in their approaches.

Key Shots:

• Photographs of watersheds in different provinces, showing before-and-after comparisons where possible (these should be landscape shots mainly, avoiding vertical shots)
• Visuals of community members interacting with these improved environments. Photos of adults interacting with children or animals is highly appreciated.
• Images illustrating unexpected benefits, e.g. increased underground water levels which allow the farmers to extract water with solar pumps

1. Strengthened and Resilient Livelihoods

Focus Areas:

• Ensure that there is consistency between theme 1 and 2. The main objective is to highlight how the watershed work (theme no. 1) allowed farmers in the communities to develop new livelihoods. i.e tree plantations, gardens, solar pumps, etc., as a result of climate-induced hazards being mitigated. Showcase how communities have not only ensured their food security, but have also improved their diet through productivity, and resource management.

Key Shots:

• Farmers cultivating diverse cash crops, supported by CBED interventions. How CBED-conducted training improved their knowledge on agriculture and the impact on productivity
• Agro service centers in action, with farmers using tools and training and the impact of such equipment in their daily life
• Veterinary Field Units working with livestock and poultry, preventing disease and supporting livelihoods. Capture how a para-vet move around communities delivering services.
• Success stories of specific interventions with demonstrable outcomes.

1. Women in CBED

Focus Areas:

• Highlight the roles and achievements of women within the CBED project, emphasizing cultural appropriateness and community support. Highlight the benefits of women participating in economic life for their family and communities so this can be replicated in the future.
• How women’s inclusion in economic life boost their self-esteem and impact their life.

Key Shots:

• Women working in kitchen gardens that have/ or not expanded to commercial ventures.
  • Interactions between women and men within the community who support women participating in agricultural activities.
    • Portraits of women SHG leaders and members engaging in community activities, selling products, and empowering others.

Deliverables for the photographs:

• A series of high-quality, professional photographs (minimum 100 images) covering the above themes, with a strong focus on storytelling and impact. All photos must be shared with the team, preferably via USB; avoid transferring photos into Word documents.
• Each photograph must be accompanied by a brief caption (1-2 sentences) that includes the participant’s name, village and province name, and/or other relevant information to provide context (e.g., the name of crops being grown). Avoid watermarking or editing the photos in any way.
• Photographers must obtain explicit permission before capturing images, especially when photographing women or children.
• A final photo report compiled in a digital format, organised by the three main themes, with selected images and captions that best represent the project’s impact.

3. Scope of work

The end-line evaluation is to be conducted in 6 provinces (with target districts and implementing partners):

• ActionAid: Bamyan (Bamyan Centre and Yakawlang).
• Afghanaid: Badakhshan (Argo and Kishem), Samangan (Aybak and Hazrat-e-Sultan),
• Concern Worldwide: Takhar (Chahab),
• Save the Children: Jawzjan (Shebergan), Sar-e-pul (Sar-e-Pul Centre)

Responsibilities of the Consultant/Firm:

• Obtain permission documents from the de-facto authorities to conduct field work (if required)
• Develop a detailed evaluation plan, including methodology, sampling strategy, and data collection tools.
• Organise logistics and conduct the evaluation autonomously, engaging Afghanaid staff at every step of the process. Engage high-quality consultants and surveyors.
• Coordinate with the teams of Afghanaid and ARC partners (ActionAid, Concern Worldwide and Save the Children). Travel to the project’s field locations, complying with Afghanaid and its partners’ policies and security protocols
• Provide exposure opportunities to Afghanaid M&E staff throughout the various phases of evaluation process, including the development of data collection tools, data collection, data cleaning and analysis, and report writing
• Facilitate at least 1 training session per phase of the end-line evaluation, for a total of at least 4 sessions (tool development, data collection, data cleaning and analysis, and report writing), for Afghanaid M&E staff, explaining key approaches and practical advice for future surveys
• Collect both quantitative and qualitative data to assess the project’s outcomes and impacts.
• Analyze the data to compare end-line values with baseline values.
• Prepare a detailed report of the findings, including statistical analysis and qualitative insights.Ensure on-time delivery of the evaluation report by 31 October 2024, with at least 2 review rounds (1 week per review) by Afghanaid and its partners prior to the final deadline
• Provide actionable recommendations for future project phases or similar initiatives.

Responsibilities of Afghanaid:

• Manage the process of evaluation and provide support to the evaluation team:
• Deputy Consortium Manager with support of MEL Specialist will be responsible for overseeing the evaluation process through all phases.
• An internal Evaluation Committee (EC) will be formed as part of ensuring the independence and impartiality of the evaluation. The EC will include the following key internal stakeholders:
  • Deputy Director for Climate Action/Consortium Manager;
    • Deputy Director of Programme Development, Impact and Learning;
    • M&E Manager/team member
• Introduce the consultant team to our field teams, facilitating their introduction to the target communities,
• Ensure Monitoring & Evaluation staff are involved in the evaluation process for exposure and training purposes

Payment:

• 40% on approval of the Inception Report
• 40% on submission of the first draft report
• 20% on submission of the final report and other deliverables as listed below

4. Timeline

The study will take maximum 4 months, with a final deadline of 31 January 2025, and is expected to start as soon as possible

The activity period of the project will continue until 31 December 2024. Due to the seasonal nature of Agriculture in Afghanistan, the end-line evaluation will be conducted towards harvest time of 2024 agriculture crop, before the end of the project. This timeline is expected to capture majority of agriculture-related project information. This should be the basis for preparing the draft report.

5. Methodology

The end-line evaluation will encompass a mix of qualitative and quantitative data gathering and analysis techniques, with an emphasis on triangulating available data sources.

Methods for data gathering for this evaluation may include, but are not limited to, the following:

• Desk Review and Context Analysis: a careful analysis of existing data and information from secondary sources including policy documents, programme documents, monitoring reports, annual project reports, past reviews and evaluations;
• Quantitative primary data collection: from a representative number of households through a carefully designed survey (cluster sampling), bearing in mind that gender dimensions vary from one region to the other and there are key elements to be assessed;
• Qualitative primary data collection: through interviews, focus group discussions, key informative interviews and other participatory methods;
• Costs Analysis: to answer the questions related to efficiency will require detailed analysis of the cost drivers associated with the programme.

The consultants will further elaborate on the evaluation methodology in the Inception Report, which will determine the exact focus and approach for the evaluation.

The team of consultants will ensure the wide representation of the project stakeholders, considering the inclusion of those disadvantaged, minority groups, persons with disabilities, as appropriate.

The evaluation will adhere to the highest ethical standards, respecting informed consent, confidentiality, and the rights of all participants. It will also incorporate a human rights-based approach, gender equality considerations, and the inclusion of persons with disabilities.

6. Reporting Line

The consultant will work closely with the ARC Deputy Director, the ARC MEAL Specialist, the ARC Secretariat team, and the Afghanaid M&E Manager. The consultant will include the Afghanaid M&E team in all phases of the survey and reporting process.

7. Deliverables

The following deliverables are expected from the evaluation:

1. Inception Report, covering at minimum:

• Introduction
• Methodology and approaches: design, the sources, methods for data collection and data collection tools, details of how the evaluation will be inclusive and reaching women, people with disability, the elderly and others, work plan with the detailed timeframe and team structure, points of contacts including for field deployment and ground work.
• Quality Assurance
• Data Analysis
• Ethical considerations
• Any revisions needed in the budget or work plan for the evaluation job
• Any other reflections of the consultants for the project
• A details and practical plan for capacity building and engagement of Afghanaid’s M&E team

2. Draft Narrative Report

• A first draft report, followed by a week of review time by Afghanaid and its partners
• A second draft report, incorporating comments from the first review, followed by a week of review time by Afghanaid and its partners

3. Final Narrative Report, covering at minimum:

• Executive Summary
• Introduction: project background, evaluation objectives
• Methodology: the quantitative and qualitative methods used, sampling strategy, data analysis: the analytical techniques used to compare end-line data with baseline data and interpret the findings.
• Findings of the evaluation
• CBED’s Climate Change and Adaptation Model
• Challenges, Lessons Leant and Recommendations
• Conclusions
• Annexes

4. Full survey database, including all tools used and any supporting documentation produced over the course of the end-line evaluation

5. PowerPoint presentation of the evaluation covering the findings, to be scheduled by Afghanaid and the consultant after acceptance of the Final Narrative Report by the end of October 2024

6. PowerPoint presentations for at least 4 training sessions for Afghanaid M&E staff, each covering one phase of the end-line evaluation process (tool development, data collection, data cleaning and analysis, and reporting)

7. Photo Report (original format) taken during the study with the relevant descriptions (Section 2.4.1)

8. Photo Report/photographs for the project exhibition (Deliverables in Section 2.4.2)

How to apply

The Consultancy Assignment

The consultancy will collect and analyze end-line data to assess the project’s progress against baseline indicators and evaluate the overall impact and effectiveness of the CBED-II project in achieving its climate change adaptation and disaster risk reduction goals at the community level. It will also evaluate and document the CBED’s Climate Change and Adaptation Model, provide exposure opportunities and tailored training for Afghanaid’s M&E staff, and submit comprehensive photo reports documenting visual evidence of project outcomes and impacts.

Submission of proposals

The application should include:

• Technical Proposal
• Financial Proposal

Requirements for the Proposals

The proposal should encompass the following elements (maximum length, excluding CVs: 20 pages):

Technical Proposal

• A brief outline of the end-line evaluation approach and methodology (maximum 5 pages).
• A proposed work-plan and timeframe for the evaluation, delineating the phases, the number of consultants, staff and the duration of workdays (maximum 2 pages).
• Identification of potential challenges and dependencies likely to affect the evaluation’s implementation, along with proposed solutions (maximum 1 page).
• A description of the strategies to ensure data quality throughout the assignment (maximum 2 pages).
• An account of relevant previous assignments that showcase the required skills and experience outlined above (maximum 2 pages).
• A summary of the most relevant skills, experience and the roles of the proposed evaluation team, including the allocation of workdays for each team member.
• CVs of the lead consultant/s and other key members of the team, highlighting relevant previous assignments and clients (CVs to be provided separately; maximum 2 pages per CV).
• Contacts of at least three references
• Attach recent relevant report samples from at least two recent organisations where the services of the consultant/team or firm have been utilized

Financial Proposal

• The consultancy firm shall furnish a budget breakdown, delineating consultant fees and logistical expenses. The financial proposal should include provisions covering all costs associated with the assignment, including applicable taxes in USD (maximum 2 pages).

Expertise, experience and qualification

We anticipate that the consultancy firm/consultants will demonstrate the following qualifications and experience (or equivalent):

The lead consultant should possess a minimum of 10 years of experience in field research and the evaluation of community development, livelihoods projects, disaster risk reduction, or related initiatives. The lead consultant should hold a Master’s or Ph.D. degree in statistics, economics, data science, monitoring and evaluation, international development, agriculture, climate change and adaptation, disaster risk reduction or a related field.

The firm/consultants should provide evidence of the following:

• Previous relevant evaluation assignments conducted in rural areas of Afghanistan.
• A demonstrated history of successful assignments, supported by positive references from past clients. Robust experience in implementing end-line evaluations related to rural livelihoods, agriculture, value chain development, watershed management, disaster risk reduction, climate change adaptation, including climate resilient agriculture.
• Available qualified gender balanced technical and field staff across the country. A proven track record of enumerator recruitment, selection, training, and supervision, with a preference for firms that have an established roster of qualified local consultants in the project implementation and evaluation areas.
• Expertise in digital data capture and established procedures to ensure high-quality data collection.
• Experience in qualitative and quantitative data analysis, with the ability to present findings clearly in a concise, analytical report.
• Strong attention to detail, excellent communication skills, and the ability to discuss research concepts and data with diverse audiences.
• Familiarity with and experience in gender- and culturally sensitive evaluation approaches.

Assessment of the Proposal

The evaluation of the proposal will be based on the following criteria:

• Alignment with the scope of work outlined in the Terms of Reference.
• Demonstrated experience of the firm/consultants.
• Previous experience of key consultants involved in the project.
• Value for money.

How to Apply

National/International firms/individuals that meet the above requirements should submit their applications to [email protected] by 4:30 pm (Kabul time) on 10th of October 2024, including the following as the subject line: “RFP No. AAD-05-2024 Proposal for CBED-II Project End-line and Final Evaluation”. Applications without the proper subject line or submitted after the deadline will not be considered. Due to the high volume of applications we receive, we are unable to respond to every application. If you have not heard from us within 3 weeks of the deadline, then you have not been successful for shortlisting.

If you have any questions regarding this request for proposal, please email [email protected] no later than 4:30 pm (Kabul Time) on 7th of October 2024 so that we have time to respond to your request before the deadline for applications.

Electronic copies of the Proposals/Quotations in response to this request are to be submitted by 4:30 pm (Kabul Time) on 10th of October 2024.

Only qualified applications meeting the above criteria will be shortlisted. Due to the high volume of applications we receive, we are unable to respond to every application. If you have not heard from us within 2 weeks of the deadline, then you have not been successful for shortlisting.