Terms of Reference for Project Final Evaluation Consultancy

The consultant reports to:

Name: Justina Agaji

Email: [email protected]

Description of consultancy

This is a final evaluation of the CBM Global supported Integrated Control of NTDs (Phase 3) project implemented in Nigeria.

1. Background and rationale

Recent statistics indicates that an estimated 100 million people in the Nigeria are at risk for at least one of the Neglected Tropical Diseases with several million cases of people being infected with more than one of them.[1] Lack of access to safe and clean water as well as good sanitary environment has made their situation even worse as some of these NTDs are related to water, sanitation and hygiene.

To address this problem, Health and Development Support Programme (HANDS) with funding support of CBM Global Disability Inclusion (CBM Global for short), implemented a third phase (Phase 3) of the Integrated Control of NTDs project in Yobe State, Jigawa State, and the Federal Capital Territory (FCT) aimed at contributing to the elimination of NTDs, focusing on onchocerciasis and lymphatic filariasis. The specific objective of the project is “Strengthened local capacity in the elimination of NTDs and care of people affected and at risk of NTDs through inclusive and accessible prevention and treatment services.” It was expected that this objective will be achieved through the achievement of the following Key Result Areas of the project, viz:

• Improved mass drug administration system leading to increased access and acceptability of target communities to preventive chemotherapy and transmission control for onchocerciasis and lymphatic filariasis.
• Improved access to safe water, sanitation and hygiene for all.
• Improved capacity of primary and secondary health facilities in the delivery of morbidity management care and disability prevention services for people affected by lymphatic filariasis.
• HANDS are demonstrating improved quality of implementation and operational compliance of the project at all levels.

The Phase 3 of the project built on the successes of Phase 2 and addressed areas for improvement that were identified by the end of (Phase 2) project evaluation such as:

1. Phase 3 of the project has contributed largely to the elimination drive for onchocerciasis and lymphatic filariasis (LF). So far, the two states targeted for Mass Administration of Medicines (Jigawa & Yobe) have passed epidemiological evaluation and are currently undergoing entomological evaluation (blackfly collection). Prior to the start of Phase 3 of the project, none of the endemic LGAs for LF had passed the Stop MDA TAS1. But eight (8) out of the 10 LGAs endemic for LF that are being supported by Phase 3 of the project have passed the first Treatment Assessment Survey (TAS-1) and will no longer require treatment for LF. Similarly, one (1) out of the four (4) LGAs endemic for LF in Yobe State has passed TAS-1, while the remaining three (3) LGAs are due for TAS-1 within this year (2024).
2. Whereas access to Water, Sanitation and Hygiene (WASH) facilities was Phase 2 not reflected in the previous funding, except for an additional fund that was made available to provide hand pump boreholes in nine communities in Jigawa, Yobe, and the FCT, WASH was captured as a distinct outcome (Result 2) of Phase 3. The WASH component of the project in Phase 3 included training of artisans for the repairs and rehabilitation of boreholes/water points, formation and training of WASH committees (WASHCOMS) in communities, WASH clubs in schools, and Community-Led Total Sanitation (CLTS), among others. This greatly improved the hygiene conditions of targeted communities. Also, the artisan training served as a means of livelihood for beneficiaries of the training.
3. Similarly, Phase 2 of the project did not capture the components of morbidity management and disability prevention (MMDP) and mental health, which are critical in the elimination of lymphatic filariasis (LF). MMDP and mental health were added as a separate outcome (Result 3) of the project in Phase 3. The inclusion of this component of the project enhanced its quality, as it demonstrated an effective intervention for addressing the needs of individuals suffering from NTD complications. People living with more advanced stages of lymphatic filariasis received treatment through delivery of limb care management and psychosocial support in selected primary health care facilities.
4. Furthermore, Phase 3 of the project adopted disabilities inclusive approaches to NTD care delivery, incorporating elements of training of community drug distributors, frontline healthcare workers, and NTD managers in disability inclusive service delivery, including review of NTD data capture tools to capture disability data elements, and production of disability-friendly Information, Education, and Communication (IEC) materials for community mobilization activities.
5. The working relationship of HANDS with key NTD stakeholders relevant to the elimination drive were strengthened. For example, HANDS collaborated effectively with the Rural Water and Sanitation Agency (RUWASA) in implementing elements of the national plans and strategies on integrating WASH and NTD programmes.

With the project now ended (on 30th June 2024), it is imperative for CBM Global with HANDS and their project stakeholders to assess the extent of achievement of the objectives of the project and the impact made over the implementation period. This will enable it to draw learnings for future programming.

2. Purpose and Deliverables

The purpose of the final evaluation of the Integrated Control of NTDs (phase 3) project is to assess the overall results and approaches of the project for accountability purposes, donor reporting, organizational learning, and planning of a new phase of the project.

Specifically, the objectives of the evaluation include:

1. To review the achievements or otherwise of the project objectives thus far and the contributing factors.
2. To examine the relevance, effectiveness, efficiency, impact, partnership and sustainability approach of the project and the degree to which the projects have set a foundation for community and government ownership and likely scale-up.
3. To document lessons learnt from project planning, implementation, monitoring and learning that would guide future project management.

The following documents are expected to be developed by timelines as final products (in English Language) of the evaluation:

1. An inception report (including methodology, data collection plan, and evaluation matrix before the field data collection) produced by 13th September 2024.
2. A validation workshop to give feedback to participants of the evaluation and key project stakeholders conducted by 4th October 2024.
3. A draft evaluation report produced by 10th October 2024.
4. A final evaluation report produced by 18th October 2024.
5. Data sets for all collected data, transcribed for CBM Global’s future use.
6. PowerPoint presentation summarizing the key findings from the evaluation submitted together with the final evaluation report by 18th October 2024.

The results of the evaluation will be shared with the project stakeholders (including government, partners, organizations of persons with disabilities, etc.) through integrated approaches, such as circulating soft and hard copies of the report via online and face-to-face methods, which will be agreed with the project stakeholders.

Both the Inception report and the Final Evaluation report should be presented in the standard evaluation report format to be shared by CBM Global.

This evaluation is being carried out as part of CBM Global’s programme development requirements. Its outputs will be used by CBM Global and HANDS to establish lessons that will be used for future programming, to plan for the sustainability of the project, to understand the impact of the project on the project participants, and to communicate results to donors as part of accountability practice.

3. Scope

The evaluation will be conducted in the FCT and Yobe and Jigawa States for a period of 30 working days. CBM Global Country Office will engage an external consultant to lead on the evaluation of the project. The evaluation should provide practical guidance to the project partner, and the key government line Ministries, Departments, and Agencies (MDAs) regarding possible replication of the project approaches as a model.

The evaluation will cover the period of project implementation, from 1st July 2021 to 30th June 2024 while also considering achievements on the previous phase and how these have been built on. The evaluation will include an assessment of how the project engaged with NTD stakeholders (federal, state, and local government health, education, security, water and sanitation authorities, health workers, organizations of persons with disabilities, civil society organizations, community-based organizations, persons affected by NTDs and their family members, opinion leaders, the media, etc.) at the project locations in the project cycle management. The consultant will ensure that the evaluation process takes a participatory approach with the involvement of a variety of the project stakeholders, especially the primary beneficiaries of the project to document lessons learned from participating in the project.

The evaluation consultant will contribute to reflection and learning from the implementation of the project throughout this process. Findings and recommendations will be disseminated and used to improve the overall quality and impact of CBM Global’s work. This will contribute to organizational learning for the project partner (HANDS) and CBM Global, create institutional knowledge, gain support among stakeholders, and promote understanding of the project evaluated. The staff of partner organization will play a strong role in supporting the evaluation process.

It is important that the evaluation does not duplicate work carried out as part of the NTD assessment surveys for the elimination of Onchocerciasis and Lymphatic Filariasis specifically for the project targeted local government areas led by the Federal Ministry of Health. However, the evaluation consultant will access the results of NTD assessment surveys for from the NTD Division at the FMoH for reference, analysis, and inference in line with this evaluation objectives.

The consultant will incorporate a feedback process with project stakeholders into the data collection and communication aspects of the evaluation to document key learnings and actions for follow-up in terms of the sustainability of the project.

To a best extent possible, disability-friendly venues and communication methods will be used to accommodate persons with various forms of disabilities. It is intended that the final evaluation products will be circulated in soft and printable versions that are in accessible formats to a wide range of project stakeholders.

The evaluation questions to be considered with regards to the Result Areas are discussed as follows.

Result 1- Improved delivery of mass drug administration in target communities leading to increased access and acceptability of preventive chemotherapy and transmission control for NTDs

• How was HANDs involved in advocacy with other stakeholders for effective inclusive mass drug administration? Were there any noted successes with regards to advocacy?
• What was achieved with regards to supporting mass drug administration through training of HANDs staff, and the expected cascading effect of training to LGA NTD team members, and the training of health workers and CDDs? What can be learned about this approach? Were the targets realistic?
• What approaches were taken in this training to increase consideration of the barriers that a) women and girls b) people with disabilities face in accessing MDA programmes? What can be learned about this approach? Did this have any impact on increasing access?
• What strategies were used by the project to manage turnover through retraining?
• What strategies were used by the project teams to increase the number of female CDDs. What can be learned from the approaches taken (successes/challenges)?
• What IEC materials were developed? How did they consider inclusion, and how were they distributed? What IEC materials could be adapted for other programmes?

Result two: Improved access to safe water sanitation and hygiene for all.

• How were other organisations engaged in these efforts for improved WASH? And how effective was their contribution to this area of work?
• What was the impact of the CLTS approach? Was this approach appropriate and effective? To what degree were gender considerations taken into account as part of the WASH Committees and the CLTS approach, and were they effective?
• To what degree was the work in schools effective, and how did it reinforce changes in community level CLTS approaches?
• What strategies were taken to ensure that people with more profound disabilities have better access to water, sanitation and hygiene? How does access to WASH improve for people with lymphedema and other disabilities?
• What can the learnings from this approach inform further replication?
• To what degree is new WASH infrastructure accessible and gender appropriate? How did the project team oversee the quality of any water points, latrines and hygiene facilities and assess them for accessibility and appropriateness?
• Were the IEC materials appropriate? What approaches were taken for effective distribution/ dissemination?
• What improvements were made or could assist to addressing the barriers to accessing WASH services for persons with NTD complications and the community at large?

Result 3: Improved capacity of primary and secondary health facilities in the delivery of morbidity management care and disability prevention services.

• To what extent was the planned training of health workers (primary and secondary level) been achieved? Were the plans realistic? To what degree will this have an ongoing impact/ sustainability? To what extent was the limb care training for health workers and beneficiaries and their carers sufficient to improve self-care practice? To what degree do their limbs show improvement?
• How have women and men affected by lymphoedema and hydrocele been impacted by this project?
• To what degree has the training for carers been effective and sustainable?
• What approaches have been taken to support people with lymphoedema and hydrocele and their carers to address their (a) mental health needs (b) access to water, sanitation and hygiene? How has the project tracked this?
• What can we learn about these approaches (successes/ challenges) that are useful for future projects?
• To what extent was the mental health training for health workers sufficient for them to feel confident to identify and refer persons with NTDs that need additional mental health support? To what degree do beneficiaries feel their health workers are listening to them and meeting their mental health needs now? To what extent were they made aware of and given access to mental health services? What improvements were made or could assist to addressing the barriers to accessing mental health services for persons with NTD complications and the community at large?
• Who or what helped to improve the physical and mental health of the people with NTDs? (in terms of the Most significant change)

Result 4: HANDS are demonstrating improved quality of implementation and operational compliance of the project at all levels

• How has the project funding strengthened the HANDS teams:
• Technical capacity in WASH, in gender, and in mental health.
• Capacity to do advocacy.
• Links to other organisations: E.g., co-ordination with other actors and their interventions, to ensure that the project is ‘value-adding’ or actively contributing to the elimination of NTDs in synergy with existing interventions rather than duplicating efforts.
• To what degree was the scope of the project achievable (budgets, targets & activities vs achievements)? How has the project team tracked the quality of outcomes (as well as tracking against targets)?
• Gender: The project put a specific focus on trying to improve the access of women to the project activities. To what degree were the strategies outlined in the project design (below) used, and were they effective? What can be learned about gender approaches for similar future programmes?
• To what degree did the project appropriately adapt to changing circumstances (e.g., COVID-19, political instability, insecurity, etc.)?
• How was the quality of support from CBM Global Country Team and wider CBM Global Technical Advisors in the management of the project?
• Were there any unintended outcomes (positive or negative) resulting from the project?

Assess progress to address the gender barriers as outlined in the project design.

Project Activities

MDAs

• Access to treatment

Project approaches

Most CDDs are male who are restricted to enter homes with female family members

Gender Barriers

Train female CDDs to enable access to target households.

Continue to engage FOMWAN and community women mobilisers in drugs distribution activities.

• Participation as CDDs

Project approaches

Literacy requirement and general community perception on women’s roles in community activities

Gender Barriers

Pair community women who have low literacy but are interested to become CDDs with other women who have high literacy skills or health workers for training and support.

Continue advocacy with community traditional and religious leaders to support women’s participation in MDAs.

• Decision making in MDA activities

Project approaches

Women are generally not among the council of elders who make decisions in the community including issues regarding health

Gender Barriers

Ensure participation of FOMWAN and the Community Women Mobilisers in MDA planning and training and take into account support needed for them to undertake MDAs

WASH

• Formation and training of WASH committees

Project approaches

Community acceptance/ perceptions on the role of women in community decision making.

Gender Barriers

Include FOMWAN in advocacy to community traditional and religious leaders for the inclusion of women as part of the trainees.

Ensure there are women WASH trainers to set an example of women’s leadership and encourage participation among community women.

Include the roles of women in family and community health as a topic in promotion of hygiene and sanitation training to increase gender awareness.

• Access to water points

Project approaches

Distance

Gender Barriers

Ensure to get women’s perspectives in the siting of water points. Inclusion in accessibility audit.

• Participation in CLTS training

Project approaches

Community acceptance/ perceptions on the role of women in community decision making

Gender Barriers

Include FOMWAN in advocacy activities to community traditional and religious leaders for the inclusion of women as part of the trainees.

Ensure there are women CLTS trainers to set an example of women’s leadership and encourage participation among community women.

Include the roles of women in family and community health as a topic in promotion of hygiene and sanitation training to increase gender awareness.

• Access to WASH information

Project approaches

Inadequate use of communication channels or content inaccessible by women.

Gender Barriers

Use of town announcers, content understood by women. Involvement of women in content creation for WASH.

MMDP

• Access to treatment and care

Project approaches

Distance, lack of transportation, lack of caregiver

Gender Barriers

Training of at least one health worker to provide services at nearest health centre.

Provide women living with lymphoedema/hydrocele with transportation and food support to enable access to treatment and care.

• Participation in mhGAP training; limb care management

Project approaches

Only few women as health workers and sometimes not prioritised for training or upskilling

Gender Barriers

Advocacy targeting the Director of Public Health to assign women health workers in the project.

• Case finding for MMDP

Project approaches

Male case finders do not have access to homes with female family members.

Gender Barriers

Engage female CDDs to train for as case finding. Ensure needed support to attend training is provided (e.g. transportation, food, communications).

4. Methodology

The consultant is expected to use a mixed-methods approach, including but not limited to:

• Desk review of project documents, reports, and relevant literature.
• Interviews with key stakeholders, including project staff, beneficiaries, and partners.
• Field visits to project sites to observe activities and gather qualitative data.

Please note that the consultant is not expected to conduct a large number of individual interviews. Instead, a limited sample will be enough. We encourage the use of focus group discussions (FGDs) to gather qualitative data and avoid individual interviews. Some meetings can be done online, especially with stakeholders such as NGO staff and CBM Global staff.

The consultant is expected to develop the project evaluation methodology in line with the framework of the available budget for this task in collaboration with CBM Global. S/he is expected to submit a brief inception report where an evaluation methodology should be proposed. The mid-term review must meet the principles of being inclusive, participatory, and interactive, involving both male and female beneficiaries and persons with disabilities.

The data that are being collected and stored by the project in various formats are stored in Excel, and others collected as descriptive and so, are stored as narrative reports. The data collected are disaggregated according to gender, age and disability or impairments.

Regarding confidentiality or data protection, the consultant must take all reasonable steps to ensure that the respondents are not adversely affected by taking part in the project evaluation. S/he must keep the responses from respondents confidential, unless their permission is granted, and must not do anything with their responses that they are not informed about at the time. Particular care must be taken with children and teenagers by ensuring that permission is granted by a parent or responsible adult, such as a teacher, for interviews with children aged under 18. The CBM Global Child and Adults-at-risk Safeguarding and Protection from Sexual Exploitation, Abuse and Harassment (PSEAH) policies should be applied in all circumstances.

6. Timeframe and duration

This project evaluation is expected to be conducted over four (4) weeks, anticipated to start on 9th September 2024. The deliverables and their expected time frame are highlighted as follows:

• Inception phase: The consultant will review relevant documentation as itemized in the preceding section above. Based on the review, s/he will produce a project evaluation inception report within the first week of commissioning the consultancy – by 13th September 2024. The report should include methodology, data collection tools, evaluation matrix to be used to guide the field data collection and analysis, and the consultancy work plan.
• Field data collection phase: This phase of the mid-term review will seek to collect primary data on the key review questions explained under the Evaluation Questions (section 3.1). The deliverables expected are the raw data collected from the field using appropriate tools, approaches, and techniques. A validation workshop to disseminate the initial findings of the research is expected during this phase.
• Data analysis and reporting phase: The deliverables include the final evaluation report, PowerPoint presentation, data sets, transcripts, and signed consents used in the research.

7. Place/ location of service delivered

The consultant’s regular place of work for this contract is in any of the project states – FCT, Yobe State, and Jigawa State, with travels to project communities.

8. Required Expert Profile

The evaluator will be a consultant. S/he will be responsible for the overall evaluation process and the production of the final evaluation report and related products.

The consultant must sign the CBM Global Safeguarding and PSEAH policies before they embark on field work. S/he should be familiar with disability-inclusive approaches in evaluations in line with international standards.

The consultant will be supported by a staff of HANDS during the field visits.

The consultant will be selected based on the following criteria:

• Proven experience in evaluating projects related to Public Health, Social Works, Development, or similar areas.
• Advanced degree/MSc degree in Public Health, Humanities, Social Sciences, International Development, or other relevant fields.
• A strong background in NTD programming and in disability inclusive development is desirable.
• Knowledge of disability-inclusive practices is an added advantage.
• Ability to work independently and meet deadlines.
• Ability to draw practical conclusions and to prepare well‐written reports promptly and available during the proposed period.
• Ability to provide strategic recommendations to key stakeholders.
• Excellent interpersonal and communication skills including ability to facilitate and work in a multidisciplinary team.

9. Coordination and Logistics

CBM Global has responsibility for*:*

• Overall coordination of the project evaluation process.
• Recruitment of evaluator.
• Gathering documents and data for the consultant in collaboration with HANDS.
• Liaising with HANDS.
• Organizing post-evaluation debriefing with the consultant, including HANDS in the process.
• Signing off on the deliverables for this project evaluation.

HANDShas responsibility for*:*

• Working with CBM Global Country Team to organize meeting schedule for the evaluation team.
• Identifying safe, secure, “neutral” and disability accessible locations for interviews/ meetings to take place (where people will feel free to speak as openly as possible).
• Organizing for interviews with project beneficiaries and community leaders according to the consultant’s requests/methodology.
• Organizing for interviews with the stakeholders in eye health in Kano and Jigawa States, according to the consultant’s requests/methodology.
• Organizing for interpreters for the evaluation exercise as required.
• Provision of guidance on security and safety at the field sites.

10. Payment schedule

The consultant will be paid upon presentation of invoices after each milestone under this consultancy has been completed. At the point of contracting, CBM Global will provide the Consultancy Invoice Information form, which gives the requirements for the formal content of such invoices. CBM Global will only settle invoices in the agreed currency.

The schedule of payment for this consultancy includes:

• Signing Contract: First advance – 25%.
• Submission and acceptance of inception report and data tools: Second advance – 25%.
• Submission and acceptance of final report, datasets, transcripts, and PowerPoint Summary by CBM Global and HANDS: Final payment – 50%.

How to apply

CBM Global welcomes applicants from diverse backgrounds and people with lived experience of disability.

Interested parties should submit the following in English to our email address [email protected]; with the following format: Last Name First Name: (JOB TITLE, Nigeria):

• A cover letter expressing their interest.
• Curriculum Vitae for individuals and corporate profile for firms. For corporate organizations applying for this evaluation, they must include the CVs of the members of their team and their defined roles to perform in the evaluation of the project.
• A short summary of their understanding of the Terms of Reference (ToR).
• Three previous project evaluation reports.
• A technical offer, which must include the technical requirements and the mid-term review approach/methods, and the work plan and the timeframe to address them. It would also be an opportunity for the consultant to challenge the ToR and offer options.
• A financial offer, including a budget for the evaluation. This should include the evaluation team’s daily rates for the assignment, detailing professional fees and per-diems. CBM Global will negotiate with them the final fees in line with the budget available for this evaluation and based on the experience of the chosen candidate.

More information about CBM Global Disability Inclusion can be found by visiting the CBM Global website: www.cbm-global.org

Closing date: 6th September 2024.

Any proposals received after the stated time and date will be rejected.