Evaluation of the ICRC Economic Security Unit seed programmes in Africa

PLEASE SEE THE LINK PROVIDED TO ACCESS THE FULL RFP DOCUMENT AND ANNEXES

LINK

Introduction to topic/intervention

The ICRC’s Economic Security Unit (EcoSec) brings victims of armed conflict and other situations of violence rapid and flexible assistance to cover their essential needs and unavoidable expenditures in ways that are sustainable and respect their dignity. EcoSec programs are focused on five core components:

1. Addressing people’s critical food needs (including detainees)
2. Protecting and restoring people’s income to cover their essential expenditures
3. Ensuring dignified living conditions for affected people (including detainees)
4. Protecting and restoring critical food production
5. Strengthen local systems and institutions’ capacity to deliver essential services

Food production is one of the five core domains of EcoSec assistance, related to crop cultivation, animal health and breeding, and/or viable utilization of natural resources such as rain, land, forests, rivers, and the sea to produce or obtain food for household consumption or generation of income. Food production can be measured at the household and community levels using a variety of indicators which look at adequate production levels in terms of quantity and quality. EcoSec support the improvement of food production in both relief- and livelihood interventions through the provision of agricultural-, veterinary- or other inputs, among other types of support.[1]

Seeds distribution to vulnerable and conflict-affected households is part of EcoSec Emergency Food Production and has always been a core intervention during crises to assisting households in maintaining or rehabilitating their farming and food production systems. However, until recent years, the approach was not following the specific standards for procurement and distribution of certified seeds. Through the presence of a Seed Specialist working in LOG department and closely with EcoSec, since 2019-2020 the approach to procuring seeds and the quality of the seeds distributed to the farmers improved significantly. The current process involves scrutinizing seeds suppliers and producers against national seed standards and regulations, ensuring seeds have proper certification, developing seeds procurement guidelines to support ICRC delegations to better understand the seed sector. In addition, seed distributions in Africa region have focused on a system level approach by engaging governments and working towards strengthening capacities and formal policies on distributing certified seed.

However, until recent years the approach for procuring seeds did not follow the national regulations of the formal seed sector, meaning that the authorities working in the sector did not necessarily perform quality checks to the procured seeds. These checks, mainly in the form of germination tests at best, were left to the responsibilities of ICRC staff, whereas other quality features such as physical and varietal purity, and phytosanitary standards, were sometimes disregarded in ICRC procurement processes. This led on some occasions to issues of bad quality seeds being distributed to conflict-affected communities, leading to beneficiaries’ complaints along with high accountability concerns and reputational risks for ICRC. A decision was made in 2019 to hire a Seed Specialist working in LOG department to steer the seed procurement process and to work also quite closely with EcoSec. Therefore, in 2020 the approach for procuring seeds and the quality of the seeds distributed to the farmers improved significantly.

Overall, ICRC tries as much as possible to procure certified seeds within the formal sector/market. The current process involves understanding farmers preferences, scrutinizing seed suppliers and producers against national seed standards and regulations, and ensuring seeds have proper certification. Unfortunately, with the financial cuts, the position of the Seed Specialist was cut in 2023. Since then, EcoSec and LOG units have been trying to capitalize on the Seed Specialist’s work by developing seeds procurement guidelines that explain the formal seed system and summarize the seed procurement approach to develop common understanding and improve practices within ICRC delegations.

EcoSec has engaged in various types of support to the seed sector in several countries where ICRC operates, and in particular where the weaknesses of the system would have jeopardized ICRC’s ability to provide quality seeds to beneficiaries. The support ranged from supporting small-scale seed producers to reinforcing the certification and quality assurance services, to facilitating the access to foundation seed for multiplication. In Ethiopia, ICRC’s neutral role along with her engagement in the seed sector facilitated the movement of certified seeds from the south to Tigray when commercial access was blocked, thus providing invaluable support to farmers at a time of critical need.

The Post-Harvest Monitoring that EcoSec teams conduct on the ground show that farmers are very satisfied about the quality of the seed distributed by the ICRC, that it responds to their requirements and that their crop performance has improved. However, there is a need for more robust evidence generated through external evaluation to understand the outcomes and impact we produce for households and the whole seed sector in a given context.

[1] 4409_002 Agriculture– Economic Security; 07.2019; 500 (icrc.org)

Description of the theory of change or logic model

Seed represents the foundation of any farming system. In commercial/intensive agriculture systems, farmers may rely on hybrid seeds that are purchased every year or farming cycle. In developing countries in Africa, many farmers engage in subsistence and small-scale production where they cultivate a variety of crops for both household consumption and sale. Such farmers usually recycle a small portion of their harvest to be used as seed for next cycle of production, so they do not need to rely on the market to buy new seeds, and they are able to keep the crop varieties they appreciate the most (taste, pest tolerance, growth length, etc.).

However, there are three identified problems, farmers may keep the same seeds for decades, in some areas the period after which farmers rejuvenate their seed stock can go up to 90 years. Such seed has certainly diminished its genetic vigor and characteristics, especially seeds of cross-pollinating crops (e.g., maize). The second problem is that when farming communities are hit by conflict their seed stock may be destroyed or left behind, meaning losing the capacity to engage in agriculture if access to their fields is restored. Thirdly, often the seeds farmers purchase in the local market or provided by humanitarian agencies are of unknown variety and uncertain quality.

EcoSec has therefore been active in supporting the formal seed system and value chain through the following activities: provision of improved high quality and certified seeds to conflict-affected farmers, provision of input support and technical capacity building to seed producers/multipliers, financial and infrastructure support to service providers in order to strengthen the delivery of critical services such as field inspection and seed certification, among other interventions.

This has resulted in the increase in the availability of certified seeds of desirable crop varieties for farmers at local level, and the improvement of the capacities of relevant seed stakeholders to engage in and deliver quality services, including production and sale of foundation seeds, conduct sound testing for seed qualities and strengthen the overall access of certified seeds in hard-to-reach areas.

The expected impacts of the overall programme are improved crop productivity/production levels as well as improved food security conditions due to the increased harvests resulting from the use of high-quality seeds. In addition, supported seeds producers have increased their technical as well as financial capacities thank to the revived certified seed market and now represent relevant stakeholders in the value chain. Ultimately both public and private technical services have been strengthened and are able to deliver critical activities independently.

The context of implementing the interventions

The two country contexts selected for in depth primary data collection are Chad and Nigeria, both defined as protracted crisis contexts. In Chad, the socio-economic situation remains very fragile, also due to the political as well as the conflict scenarios. In the Lac Province, violent incidents are still occurring mainly in the insular areas, while the east of the country is affected by the spillover of the civil war in Sudan. Even though the EcoSec programme has seen a reduction as a whole, with a re-orientation of relief interventions towards new areas of conflict (east of the country), a Tier 2 (independent funding stream) EcoSec multi-country programme funded by AfD is being implemented around agro-pastoral interventions, including support to the seed sector.

The seed programme in Chad has been implemented since 2020 by partnering with public seed service providers on one side and supporting private actors on the other, mainly for multiplying foundation seed material. The EcoSec team there has enjoyed a fruitful collaboration with a regional federation of farmers and seeds producers, which facilitated distribution of large quantities of certified seeds to conflict affected communities.

Nigeria is plagued by armed conflict mainly in the North-East, while other situations of violence (OSV) continuing across the territory, with flashpoints in the South-South/South-East and North-West and North-Central regions. From a humanitarian perspective, Nigeria doesn’t receive major attention compared to the scale of needs of the population. In 2024 the delegation has prioritized geographic areas in which its added value and impact are clear, notably territory not formally controlled by the government forces in the northeast. It focuses on people affected by the armed conflict by consolidating its work and presence in Borno and strengthening its operation in South Borno (Biu), Northern Adamawa (Mubi) and Yobe (Damaturu) where it enjoys access.

The delegation continues its strategic shift from emergency to resilience strengthening, with one of the delegation’s Operational Priorities (OP3) for 2024 on food security, through a combination of nutrition-sensitive programming that include asset rehabilitation, support to food production (crop, cash, livestock), income generating activities with climate change considerations, along with more nutrition-specific activities.

Specifically on seed interventions, EcoSec works with 700 seed producers in seven communities receiving capacity building trainings on community seed multiplication programme for a range of crops: cowpea, rice, groundnut, and millet. The National/Regional Seed Council is engaged by following up with field inspections and seed certification processes. Targeted communities also receive some equipment to improve post-harvest practices and crop processing like threshing machine and storage facilities. In Nigeria the private seed sector is quite developed and such seed producers are also contacted to be out-growers for private seed producers in order to make seeds locally available. Ethiopia, Niger, and Somalia have also been engaging on seed programmes within their EcoSec interventions and will be part of a lighter desk-based review. In fact, Ethiopia has already benefitted recently from an evaluation of their agriculture programme, while Niger and Somalia`s access to the field is ever more limited due to security concerns.

About this evaluation

This is a multi-country, sector-specific impact evaluation and is intended to generate evidence on whether ICRC interventions promoting the use of certified seeds is more impactful for farmers than the seeds found on the informal market, based on maturity, pest tolerance, organoleptic characteristics, among others. In addition, the evaluation should generate evidence on secondary benefits, such as improved availability of quality seeds, improved capacities of public and private entities, and ICRC’s strategic positioning vis-à-vis other humanitarian organizations in the context. This evaluation will include in-depth primary data collection in Chad and Nigeria, and desk reviews focusing on Ethiopia, Niger, and Somalia. In addition, the ICRC delegation in Central African Republic (CAR) is planning a separate standalone evaluation of their seed program and these two evaluations will happen concurrently, allowing findings and observations from the CAR evaluation to feed into this regional evaluation, where possible.

Evaluation purpose

The purpose of this evaluation is to assess and document the results of promoting the procurement and distribution of certified seeds for the target households, the benefits for the broader national and international stakeholders and the seed sector in the selected countries.

Objectives

The specific objectives are the following:

1. Measure farmers satisfaction – disaggregated between women and men – on the seed varieties provided, in terms of quality and quantity, but also based on maturity/growth cycle, height and use of biomass for livestock, pest and disease tolerance, organoleptic characteristic like taste, texture, cooking time, among other criteria. These aspects will be compared to the varieties farmers typically plant, their preferred ones, or to what is found on the local informal seed market. Considering the critical role of women in local seed system, this study will seek to capture any meaningful differentiation between women and men’s crop utilizations and preferences.
2. Measure the outcomes of crop production/productivity of conflict-affected households receiving certified seeds, compared to those who used non-certified seeds i.e., from other organizations or local markets. As a result, seek to measure any emerging impact and added value on food and nutrition security of receiving a diversified package if any (cereal, leguminous, oil crops) of certified seed crops for targeted households in conflict affected areas. Where feasible, extrapolate the financial benefits of investing in strengthening certified seed systems, i.e., value of increased yield or quality, compared with the cost of the control model.
3. Assess the broader benefits of ICRC-EcoSec interventions in supporting the formal seed value chain for local and national stakeholders, the development of the seed sector and the larger humanitarian community, through presumed improved access to certified seeds within each country.
4. Assess the added-value of ICRC-EcoSec to leverage its role as NIIHA[1] and neutral intermediary to facilitate seed access in areas under siege or control of armed actors, in contexts where this is relevant.

[1] Neutral Impartial Independent Humanitarian Actor

Key evaluation questions

Effectiveness:

1. How satisfied are targeted farming households with the seed package provided based on criteria such as maturity/growth length, pest tolerance, biomass, organoleptic qualities, among others, and how these compare with the seed varieties found on the local market or provided by other agencies? Does any particular difference emerge in their feedbacks and preferences between men and women farmers?
2. How effective is distributing (a variety of) certified seeds in order to increase crop production and farm productivity at household level, and which financial benefits this might bring?
3. How do these achievements compare with other households using different seeds from local market or provided by similar programmes in the region?
4. What are the broader benefits of ICRC Eco-Sec interventions on certified seeds for local and national stakeholders, and for the development of the formal seed sector in general and larger humanitarian community engaged in seed distribution?

Sustainability:

1. What are the preliminary indicators that the intervention results are likely to be sustainable in the future? Which dimensions of perspective sustainability are observable i.e. technical, financial, institutional, and environmental sustainability.

Impact:

1. Is the ICRC approach to certified seeds procurement and distribution bringing about desired changes both expected and unexpected for food and nutrition security for targeted households in the selected countries?

Added value:

1. What is the added value of ICRC EcoSec interventions on certified seeds production and distribution in the region? What changes if any need to be made for future planning?
2. How has ICRC leveraged its role as NIIHA or neutral intermediary to facilitate access to quality seeds, and how can it best position itself to do so in areas under siege or otherwise inaccessible due to situations of conflict?

Scope

Technical scope: The technical focus is on the seed system at household level, mainly what farmers looks for in the seeds they choose, what is available in the local markets, and how ICRC seed interventions may have contributed to fill any identified gap. In addition, this study explores the formal and informal seed sector in the selected countries, by engaging with different actors and stakeholders involved, such as government, enterprises, local traders, and households using certified and non-certified seeds. This evaluation will strive to assess the wider range of benefits brought by ICRC seed interventions, such as households’ food security and capacities of the technical partners, among others. The results for households receiving certified seeds in comparison with households receiving different types of seeds, as well as the impact on the larger seed system, needs to be documented in a rigorous manner. In addition, the added value and prospects for sustainability will be explored for learning and accountability purposes.

Temporal scope: The evaluation may cover the period from 2019 onwards which is the start of the institution’s current approach to seed production and distribution, however the study will focus specifically on the latest cropping seasons in which farmers were involved, most probably 2023 and 2024.

Geographic scope: The evaluation will target some country delegations that have engaged in seeds programmes in Africa, namely Chad, Nigeria, Niger, Ethiopia and Somalia. Chad and Nigeria will be targeted for an in-depth evaluation including field visits while the remaining will be covered through a desk-based review to complement information.

Intended users

• Primary intended users: The primary audience for this evaluation is the Economic Security Unit, the PES Department, EODG, and the delegations selected for in depth data collection and secondary data review.
• Secondary intended users: Department of Operations, other ICRC delegations, the Movement partners, and for other humanitarian organizations involved in seed programmes.

Key stakeholders

The list of key internal and external stakeholders and their modes of engagement in the evaluation will be developed and finalized by the evaluation manager in discussion with the external evaluation consultants. Key stakeholders and participants will be engaged in the evaluation through different methods including as part of the validation meetings with the Evaluation Advisory Group and discussions of the findings when applicable, during inception phase consultations and key informant interviews as part of the data collection phase.

This list will be composed of the following types of stakeholders:

Internal ICRC – EcoSec staff, other relevant ICRC staff within the Protection and Essential Services Department (PES), relevant ICRC staff within Department of Operations (OPS), relevant ICRC staff in the Policy tea and in Logistics department, ICRC staff in the selected delegations, RC/RC Movement partners

External stakeholders – National and local level services providers, authorities, research institutes, private enterprises, and seeds producers, which are all engaged in different ways in seed programs. In addition, other relevant humanitarian organizations involved in similar work, especially relevant for the selected countries for in depth data collection and secondary data review partner academic institutions.

It is envisaged to engage with affected people among the targeted households in the selected countries to understand the outcomes and emerging impact produced by the use of certified seed.

Evaluation framework

The evaluation will draw on the OECD DAC criteria of effectiveness, sustainability, and impact. Effectiveness criteria will explore the extent to which distributing specific certified seeds satisfy farmers needs and expectations and produced anticipated outcomes at the household level in comparison to household using different types of seeds. Under this criterion other benefits of Eco-Sec interventions on certified seeds within national and sectoral context will be explored. Impact criteria will investigate the wider effect of the EcoSec interventions on certified seed at household levels covering both intended and unintended effects. Understanding the impact will include both scientific evidence enabled through the specific quasi experimental design of this evaluation and value judgments by households on the significance of changes. Sustainability will explore the extent to which the benefits of the certified seed intervention in the selected contexts is likely to last in the future. An additional criterion of added value will examine the unique expertise, resources, and approach that ICRC brings for the affected people and the seed sector in the selected contexts, what is different in ICRC approach and what changes if any needs to be made to future interventions.

Evaluation approach

Overall design, methodology

The evaluation is planned to be participatory and quasi-experimental by design involving control groups. The evaluation will draw on the data gathered from households that received certified seed through ICRC interventions in selected locations and control group comprised of households in the same location receiving other types of seeds. The primary data needs to be complemented by secondary data from other organizations with reference to the control group where applicable.

Specific methods

The evaluation will deploy both primary and secondary, qualitative, and quantitative methods for data collection. Quantitative data collection will include surveys of households among the two groups of households and secondary quantitative data from ICRC and other organizations working in the selected locations. Qualitative data will include information from focus group discussions among farming households of both groups, direct observations where applicable, key informant interviews with relevant ICRC and other humanitarian organizations in the selected contexts and review of ICRC internal documents as well as other literature and evidence on the topic externally, for instance relevant country reports on agriculture, food security and nutrition, other sector specific reviews and reports. Analysis methods will require triangulation between multiple data sources and methods with analysis matrix submitted as part of the evaluation report. This study will also draw from the FAO seed security assessment methodology and other similar and relevant guidelines whenever possible and relevant.

Primary data collection expectations, disaggregation

Primary data collection will include surveys involving households receiving certified seed and households receiving different types of seeds in selected locations in Chad and Nigeria. As random sampling is not possible in the context of this evaluation, using quasi-experimental design the consultants are asked to suggest approaches based on inception consultations and literature review for identifying comparison groups that are as similar as possible to the group receiving certified seeds in terms of baseline (pre-intervention) characteristics.

In addition to the survey, it is suggested to utilize focus group discussions (FGDs) with selected households in Chad and Nigeria. The FGDs can help integrate local and collective experience with the scientific rigor of surveys as part of the quasi-experimental approach.

In addition, direct observation is suggested as primary data collection that involves observing and documenting notable facets of changes at the household level in terms of crop production and farm productivity.

Depending on the methods and the sources, data should be collected in a way that it can be disaggregated by individual attributes (e.g., sex, age, disability etc.) as per the Sex, Age and Disability Data Disaggregation Framework[1] and/or contextual attributes including ethnic/tribal background for primary data collection in Nigeria and Chad.

Secondary data availability

Ethiopia, Somalia, and Niger are selected delegations for a lighter approach of evaluation, namely a programme review, mainly based on secondary data review. Secondary data will include EcoSec monitoring and analysis framework, including Post Distribution Monitoring and Post-Harvest Monitoring data, end-of-projects reports, internal agriculture strategy documents, among others. Relevant projects and country-level reports from other organizations working in the same contexts will be considered, along with additional literature in the sector, ICRC project reports, and relevant ICRC strategy documents. The final list of datasets and documentations for review will be determined during the inception phase.

Translation requirements (if applicable)

Members of the evaluation team must converse fluently in English and French. The final report and communication on the evaluation with the ICRC HQ and EcoSec team will be in English. The communication with ICRC delegation staff in Chad will be in French and in Nigeria in English. Surveys and focus group discussions with households is expected to be conducted in French and Arabic in Chad and English in Nigeria. The evaluation team needs to consider also other spoken indigenous languages especially among communities in both countries and allocate sufficient funds in the budget for any translation needs into local languages and into Arabic when engaged in data collection in Chad and Nigeria.

[1] 4698_002_Sex, Age and Disability. How a data disaggregation Framework Can Make Our Operations More relevant; 06.2023 (icrc.org)

Risks, limitations

The ICRC’s 2023-2024 restructuring led to higher workloads and additional responsibilities for staff members and managers throughout the organization, thereby reducing their availability and receptiveness for processes such as this evaluation. The evaluation will need to be accommodating regarding the availability of staff members, and in the case of non-availability contingency plans for data collection will be needed to ensure the validity of datasets.

The operational environments of ICRC delegations selected for case studies present inherent risks including but not limited to safety, security, limited communication, hygiene issues and transportation challenges. The Evaluation Office will share with the prospective consultants’ guidelines for security protocols during data collection that can be used in addition to other ICRC guidelines. The modalities for risk assessment and risk management during field visits by external consultants are case specific depending on the classification of context. In low-risk context the consultants are expected to assess and mitigate the risks independently through consultations with the specific ICRC delegation, available guidelines, and travel advisories. Therefore, consultants are expected to have prior experience operating in environments with inherent risk and have mitigation measures for various circumstances. Certain nationalities might be excluded from traveling to Chad and Nigeria for in depth data collection which is another limitation to consider by the consultants. As a mitigation measure, having partners in the countries among national researchers with in- depth knowledge of the country context and languages will be a benefit.

One of the methodological limitations is that ICRC will not be able to compare two sets of identical households that differ only by the type of seeds they plant, therefore the differences in crop productivity that can emerge might be due to other factors of production, thus affecting the purpose of this evaluation. In order to minimize this problem, the sets of households that will be selected for the evaluation will be as homogenous as possible, from a socio-economic and agro-ecology perspectives. In addition, ICRC will not have any prior (in advance of the evaluation) visibility of the type of seeds the control group has used, whether it comes from the informal market, whether formally certified or provided by other NGOs/INGOs because they will be selected after they have planted the seeds In addition, the sensitivities around documenting and reporting the different outcome of the seeds provided by the ICRC and other organizations needs to be considered when reporting the findings, including keeping the name of INGOs/NGOs confidential. Evaluation teams are required to expand on risks and include methodological risks, contextual risks and limitations, and mitigations in their proposal and during the inception phase.

Deliverables

1. A 1-page briefing outlining the purpose, timing, and key messages to explain the evaluation process to stakeholders. This acts as a communication tool internal within the evaluation particularly during the data collection phase. It includes the contact details of the evaluators, the evaluation commissioner, and the ICRC’s integrity weblink. In francophone countries this need to be prepared in French.
2. An inception report with PPT detailing a proposed methodology, evaluation matrix, list of stakeholders to be consulted, workplan and timeline, and the tools for data collection to be presented to the Evaluation Advisory Group (EAG) and the commissioning Unit/Delegation for validation. In francophone countries this need to be prepared/translated in French
3. A preliminary findings and recommendations presentation of the key findings, recommendations, lessons learned, and best practice addressed to the Advisory Group and the commissioning Delegations.
4. A draft evaluation report. This should be clear and simply written, free of unnecessary jargon. The main body of the report should not exceed 40 pages. The report outline will be agreed with the supplier as part of the inception report. The report will include the following sections:

• Executive Summary – a short overview of the report
• Context and background
• Methodology
• Main findings of the evaluation, and conclusions
• Concise recommendations, with consideration of the degree of prioritization and any necessary sequencing for actions and the responsible persons
• Annexes – ToR, evaluation matrix, list of documents consulted, anonymized list of stakeholders, tools used for the data collection and any other relevant documents to support the report.

1. A final report, with feedback integrated from the EAG and other internal stakeholders as relevant.
2. A visual communication product such as a poster, infographic or max 4 pages brief that presents the key messages and recommendations of the evaluation in an engaging manner. When relevant only and agreed upon at the inception phase with the evaluation manager.

Ethical considerations and safeguarding

Evaluators are required to adhere to international best practices and standards in evaluation. It explicitly requires evaluators to abide by the Professional Standards for Protection Work; the ICRC’s Code of Conduct; the ICRC’s Code of Ethics for Procurement; and the ICRC Rules on Personal Data Protection. The evaluation design and implementation must apply the ICRC’s guiding principles and approach on Accountability to Affected People. Evaluators will be expected to obtain informed consent from those interviewed and ensure that if an interviewee is quoted, the interviewee can’t be identified unless with consent. The evaluation inception report will be subject to formal Ethical Review Board approval (arranged by the ICRC) in the following cases:

• The primary data collection will include participants among affected people
• The line of questioning includes sensitive issues
• Individuals’ privacy or safety has the potential to be compromised
• Data generation involves non-universal distribution of resources
• Secondary data will be used for a purpose other than its original intent

For this evaluation, formal ethical approval of the inception report will be required, and it will be facilitated through the ICRC Evaluation Office.

The evaluators need to ensure the language and terminology used during data collection is contextually appropriate. Thus, a conflict sensitivity analysis needs to be performed by the evaluators during the inception phase to account for any potential challenges and propose mitigation strategies. As per the conflict sensitivity analysis, all data collection instruments need to be tested before application in the field to refine the language and use of contextually appropriate terminology, translation, and approach. Engagement with affected people will require the evaluators to develop a detail a plan on how protection of subjects and respect for confidentiality will be guaranteed. Questions should be asked sensitively, and interactions need to be flexible and participatory, to allow for the most authentic, spontaneous, and participant-led exchange in the language that is known to them.

Evaluation team members or consultants must not have any conflict of interest (COI). An evaluator cannot evaluate an intervention if they have previously participated in its design or implementation. Perception of COI is a broader consideration and bidders are required to explicitly identify any potential and any likely perceptions of COI and propose mitigation strategies.

The evaluation should not collect any personally identifying information and is therefore not considered a data processer on behalf of the ICRC for the purposes defined in the GDPR. However, data generated through the evaluation may be sensitive, and the confidentiality of stakeholders must be upheld.

The raw data generated by the external evaluators is owned by the ICRC but as an independent evaluation the ICRC does not request access to this material unless there is a serious issue, dispute, contractual failing on the part of the supplier, or for other reasons not yet determined. In the case of the ICRC requiring the supplier to handover primary data, it must be cleaned and anonymized to protect the privacy and confidentiality of participants.

Management of the evaluation

The EcoSec Unit is the commissioning team of this evaluation (“Evaluation Commissioner”), with the EcoSec Agriculture and Climate Change Lead serving as evaluation manager andhaving overall responsibility for managing the external evaluation team and convening the Evaluation Advisory Group. The Evaluation Manager is responsible for implementing the evaluation from conception to the management response and using the findings and recommendations.

At the scoping and preparatory phases, the Evaluation Manager develops the TOR, identifies, and convenes the EAG, organizes the procurement (with the Eval Office depending on the procurement route), and organizes the collation of relevant documentation to share with the external team.

At the inception phase, the Evaluation Manager sets the parameters for engagement with the external evaluation team (expectations, communications, document sharing) and identifies and introduces the evaluation team to key stakeholders to consult as part of their onboarding and planning for the evaluation. The EM manages the feedback process on the draft inception report among the EAG (and the Evaluation Office facilitates the formal quality assurance process).

At the data collection phase, the EM facilitates introductions to internal and external stakeholders as needed, supports the internal distribution of surveys if relevant, convenes the EAG for the presentation on findings, and facilitates the feedback process on the draft report.

At the dissemination phase, the EM proposes and facilitates sharing of the findings (e.g. the report, its executive summary, webinars) and manages the completion of the management response to the recommendations.

The team members of the EcoSec Unit will contribute to the processes of onboarding the external evaluation team, accessing documentation, and providing supporting information.

All evaluation deliverables will be submitted simultaneously directly to the Evaluation Manager and the ICRC Evaluation Office in due course for a first review. The Evaluation Manager will mobilize and share the inception report and final report with other key ICRC stakeholders for feedback, such as the Evaluation Advisory Group. The Evaluation Manager will provide consolidated feedback to the external consultants, including external quality assurance reviews overseen by the Evaluation Office. The key dates for deliverables and milestones will be determined during the inception phase in consultation with the Evaluation Manager.

The Evaluation Office at HQ will accompany the process providing technical advice and feedback to the Evaluation Manager at key stages, including scoping of the exercise, identifying and recruitment of the external evaluation team, providing feedback during the inception phase on the proposed methodology, methods, and data collection instruments, providing feedback on the inception and the final reports directly and through the established Evaluation Quality Assurance process and ensuring impartiality.

Evaluation Advisory Group

An Evaluation Advisory Group (EAG) will accompany this evaluation, providing their expert advice and feedback on key stages. The members of the EAG may be included in the initial onboarding process for the external evaluation team. The EAG will be consulted for the inception phase presentation/discussion on the plan for data collection and analysis, and for the presentation of the final report findings. The EAG for this evaluation consist of the following members: Cecilia BENDA (Agronomist, HQ), Arjun CLAIRE (Policy Adviser, HQ), Abdi Ismail ISSE (Operations Coordinator Africa region HQ), Steve Bonaventure Ako Tang (Protection Operations Partner, HQ), Louise Le Marie (Protection Operations Partner, HQ), Celestin Sikubwabo Bwenge (EcoSec Coordinator, Chad), Alhassana IDRISS (Agronomist, EcoSec, Chad), Michael Odong Hannon (Deputy EcoSec Coordinator Nigeria) (pending), Subhiya MASTONSHOEVA (observer status) (Evaluation Officer, HQ), Nelly Njeri Mwangi (observer status) (Regional Evaluation Adviser, Africa region)

Timeline (indicative)

The evaluation is anticipated to start in August 2024 with all deliverables expected by the end of QR1, in March 2025. The ICRC does not mandate that working days must be consecutive. The ICRC envisages that the in-country data collection will be required for Chad and Nigeria with the rest of the data collection with internal and external stakeholders happening remotely.

Evaluation Quality Assurance

The process will conform to the ICRC’s quality standards for evaluations, which set out what the minimum standards are for an independent, professional, and reliable evaluation. These sets of quality standards are in the form of checklists and the key products of the evaluation (Terms of Reference, Inception Report, and Evaluation Report), are all impartially assessed by an independent expert. This process is managed by the Evaluation Office. The quality standards are provided to the evaluators at the start of the contract. In addition, regarding sequencing, the Inception Report must be signed off by the internal evaluation manager and the Evaluation Office before any data collection activities begin.

Publication of final report

The ICRC’s Evaluation Office’s Policy is to publish the Executive Summary and or the full report on the ICRC’s website in line with the Access to Information Policy.[1] for transparency as per international evaluation best practices. The reports are fully anonymous and will be published unless there are strict limitations to its publication or if it includes sensitive information based on the categorization identified in the ICRC`s Access to Information Policy. This decision is discussed at the ToR preparation phase to ensure it is clear to all stakeholders.

Dissemination of findings

The ICRC’s Evaluation Office will publish the Executive Summary and or the full report based on the final agreement with the EcoSec team on the ICRC’s website. The full evaluation report with findings will be disseminated through different modalities internally, including through publication on Evaluation Office community site to the following stakeholders:

• PES
• OPS
• EODG
• Selected delegations

The evaluation consultants are expected to present the final report to selected ICRC internal audience in addition to the EAG members during one dedicated event organized by the Evaluation Office or the Commissioners of the evaluation.

Follow-up of recommendations

In order to strengthen the use of the evaluations at the ICRC, fostering ownership over the process of change and ensuring accountability for results, the intended users of this evaluation will initiate the management response process as a follow-up action facilitated by the Evaluation Office. The Evaluation Office will support the Evaluation Commissioner and responsible teams and stakeholders in developing and tracking the management response actions through dedicated systems and processes. An Evaluation Management Response template is provided at the end of the evaluation to the evaluation commissioners to keep all accepted recommendations in a centralized manner. The Evaluation Office organizes 6-month and 12-month check-ins to discuss the status of the recommendations’ implementation and collect feedback on the relevance of recommendations.

Budget range or anticipated working days

The budget ceiling for this evaluation is 90, 000 – 100, 000 CHF, however overall cost will be taken into consideration for the selection decision including cost effectiveness of the proposal. The consultants are expected to elaborate in the proposal on the number of anticipated working days for this assignment based on the budget range, proposed timeline and level of efforts required at each stage of the evaluation. The travel cost associated with the visit to Nigeria and Chad, and any translation costs should be included in the budget.

[1] https://www.icrc.org/sites/default/files/document_new/file_list/access-information-policy.pdf

Expertise required

The team for this evaluation can be combined team of international and national experts from the region.

Team Leader:

• Substantial experience (minimum 15 years) in leading teams and designing and delivering rigorous evaluations in the international and humanitarian sectors, meeting international evaluation quality standards.
• Proven experience in impact evaluation of agriculture and/or seed programme and humanitarian work in Africa region, understanding of emergency agriculture response
• Minimum academic qualification: post-graduate degree in evaluation, agriculture, humanitarian response, or other field relevant to this evaluation.
• Understanding of the ICRC’s mandate and work, previous experience with ICRC as a staff member or consultant would be a strong asset given the candidate does not have a conflict of interest and have not worked on the design or implementation of anything falling within the scope of this evaluation.
• Proven expertise in facilitating participatory workshops involving different groups and participants

Across the team:

• Proven experience of conducting mixed- methods humanitarian evaluations for livelihood and agriculture programmes
• Expertise in developing and adapting qualitative and quantitative evaluation tools for different stakeholders, geographic contexts, and thematic areas.
• Knowledge, understanding, and/or experience of the ICRC’s institutional and operating models.
• Expertise and technical knowledge on emergency agriculture response, assistance, and protection in the humanitarian sector.
• Understanding of the ICRC’s mandate and work, previous experience with ICRC as a staff member or consultant would be a strong asset given the candidate does not have a conflict of interest and have not worked on the design or implementation of anything falling within the scope of this evaluation.
• Ability to work in English and French, with additional languages such as Arabic being an added asset.

How to apply

PLEASE SEE THE LINK TO ACCESS THE FULL RFP AND ANNEXES

https://alnap.org/about/get-involved/vacancies/evaluation-of-the-icrc-economic-security-unit-seed-programmes-in-africa/

Firms are requested to submit the following their proposals to the following address:

To: evaluation_bids@icrc.org by July 17, 2024 (16:00 hours). Please indicate “Evaluation of EcoSec seed programmes in Africa” in the subject line. The details about the content of the proposal are outlined in the Request for Proposal document.