End of Project Evaluation for UK Govt-Funded Inclusive Education Project, Western Equatoria, South Sudan.

Introduction

The Scottish Catholic International Aid Fund (SCIAF) in partnership with the Sudan Evangelical Mission (SEM) is implementing a three-year project (2022 to 2024) with funding from the UK Government’s UK Aid Match (UKAM) Programme. This project is intended to increase equitable access to inclusive education for children with disability (CWD) in four counties of Western Equatoria State by making learning more inclusive and schools more accessible to children and students with disability. The project’s intended outcome is to ensure that 80% of 500 targeted CWD have access to mainstream education.

The project works with a total of 32 schools. Six of these schools have been selected for intensive support, and a further 26 schools have benefitted from the project to a more limited extent. The project focuses on three principle components:

1. Making learning inclusive and welcoming, and schools accessible to the children with disabilities. This includes the construction of ramps, accessible toilets, renovation of classroom windows and door, provision of school desks, provision of assistive devices such as wheel chairs, tricycles, braille machines, white canes, hearing aid and crutches to the children with disabilities. Teachers have trained in inclusive education and South Sudan Sign Language and Braille
2. Community outreach and the identification and support to individual children with disabilities and their families and caregivers using the Community-Based Rehabilitation (CBR) approach.
3. Awareness raising and advocacy to the community and key stakeholders on the rights of children with disabilities. This involves working with organisations of people with disabilities to engage with community members to discuss about rights children with disability to education and social welfare, and the challenges faced by the parents of these children taking their care.

It is expected that by the end of this project the following would have been achieved:

1. 80%, 400 of 500 CWD regularly attend (≥85%) a school in Mundri East & West, Maridi & Ibba County. (Access)
2. 80%, 400 of 500 CWD tell us that adapted activities have enabled them to learn. (Quality)
3. 80%, 400 of 500 CWD report reduced barriers to participate equitably in school lessons and activities. (Inclusion)
4. 95%, 475 of 500 CWD carers are using ≥80% of the recommended care practices in their CWD’s individualized care package. (Access)

Purpose of the evaluation

The purpose of final project evaluation focuses on both accountability (i.e. assessing the design, implementation and achievements of the project over the entire implementation period) and learning from the project that can be used in the design and implementation of future projects and partnerships, and how SCIAF addresses both gender and disability inclusion, and their intersection.

The evaluation will aim to be of use to the implementing agencies (SEM and SCIAF), the funding agency (the UK Government’s Foreign, Commonwealth and Development Office (FCDO) and the UKAM Programme fund manager Mannion Daniels), project participants, local officials, as well as other stakeholders.

The evaluation will make use of the revised OECD DAC Evaluation Criteria and focus on three of the six Criteria: Relevance, Effectiveness, and Impact.

The evaluation will assess the project design and the underlying Theory of Change, and results chain as set out in the logical framework, as well as the way that it responded and adapted to learning. It will assess achievements against the indicators and targets set out in the project logical framework. It should capture significant learning for future interventions, their design and implementation.

The evaluation should pay particular attention to disability inclusion, gender equality and girls and women’s empowerment, and how they intersect in the project, evaluating achievements and any barriers to promoting effective inclusion for both female and male children and people with disabilities, and lessons to be learned for the future. The role of menstrual health interventions in increasing access should be assessed, as well as students’ perceptions of safety at school.

The evaluation should capture the self-reported experiences of project participants (students, carers, CBRWs, school staff, community members and local opinion leaders) as well as relevant stakeholders to assess achievements and impact. Results should be triangulated against other self-reported data and objectively-verifiable data where possible.

The evaluation should include and report on SCIAF’s Standard Indicator. This is a self-reported assessment by project participants of the extent to which they feel their life has changed over the project period, in what domains and to what extent they credit the project as causing the change. It should be incorporated into any survey of participants.

Scope and focus of the evaluation

The evaluation should focus on both Accountability and Learning. The geographic location for this final project evaluation is the four counties (Mundri East, Mundri West, Maridi and Ibba) where the project has been implemented. It will cover a project implementation period from 1.1.22 to 31.12.24, and a closure period up to 31.3.25.

The evaluation should be completed no later than 31.3.25. Inception work is planned for December/January, field work is expected to take place in January and February 2025, with a draft report produced late Feb/early March and a final report completed and signed off by SEM and SCIAF in mid-March. The evaluation should be completed no later than 31.3.25.

The evaluation will include review of the project design and assumptions made at the beginning of the project development process. The evaluation should assess the relevance of the design; the effectiveness of the implementation strategy and the results (for example, of the Community-Based Rehabilitation (CBR) approach); the extent to which project objectives have been achieved and the impact of the project. It should assess what local partnerships have been established and how these contributed to both achievements and potential sustainability of activities and impacts; what capacities have been built and how. The evaluation should assess the extent to which disability and gender have been addressed both as individual issues and as intersecting issues, and how communities and marginalized groups have been involved and given a voice.

Methodology for evaluation

The final project evaluation will be carried out in accordance with SCIAF and SEM evaluation norms and standards and the OECD DAC evaluation criteria (focusing on relevance, effectiveness and impact) taking into consideration normal evaluation ethics, data protection and informed consent of all informants. The methodology must be appropriate for children, and children and people with disabilities.

SEM and SCIAF encourage a mixed-methods approach, using both primary and secondary data, and effective consultation with all significant stakeholders (including project participants (students, carers, CBRWs, school staff, community members and local opinion leaders) as well as relevant stakeholders). Evaluators are expected to design an effective evaluation strategy, method and tools, which must ensure effective consultation with people with disabilities and women and men, girls and boys. We encourage the use of participatory methodologies. The evaluation should capture the self-reported experiences of project participants to assess achievements and impact. Results should be triangulated against other self-reported data and objectively-verifiable data where possible.

Where relevant data should be captured so that it can be disaggregated and analysed by gender, age, disability (using the Washington Group categories) and by school.

Where possible we encourage feedback by either the evaluation team or the project team to local stakeholders, and this should be considered in the design. A validation workshop with SEM and SCIAF will be held online to review the draft report and agree any changes to be made.

Implementation timeline (indicative)

• Call for proposals – deadline Wednesday 20.11.24, 23.59 UK Time.
• Contracting date – COB Wednesday 6.12.24
• Inception Report submission – COB Friday 10.1.25
• Inception Report sign-off – COB Friday 24.1.25
• Draft Report Submission – COB Monday 28.2.25
• Validation Workshop (online) – Wednesday 12.3.25
• Final Report sign-off – COB Friday 28.3.25

Expected Deliverables

1. Inception report

The Evaluator will prepare an inception report which details the evaluators understanding of the evaluation and how the evaluation questions will be addressed. The inception report will include the evaluation matrix summarizing the evaluation design, methodology, evaluation questions, data sources and collection analysis tool for each data source and the measure by which each question will be evaluated. The report will include the scope of work, work plan, and time frame. The inception report should include a proposed schedule of tasks, activities, and deliverables, with clear responsibilities for each task or product. The inception report will be discussed and agreed upon with SEM and SCIAF.

This is a critical stage and sufficient time will be needed for review by SCIAF and SEM. No data collection will be possible until the Inception Report is signed off (including amendment of any elements of the strategy, method or tools in line with SCIAF’s requests) – therefore sufficient space should be built in to the schedule to allow for this.

1. Draft Evaluation report

The Evaluator will prepare end of Project Evaluation Report, meeting all the criteria in the TOR. It should be accompanied by a completed SCIAF Evaluation Summary. (Template to be provided.)

It will be submitted to SEM and SCIAF for review and comments. Comments from the SEM and SCIAF will be provided. These will be reviewed at a Validation Workshop.

1. Final Evaluation Report and Evaluation Summary:

These should address all agreed comments from the review, and should fully meet the requirements and objectives set out in the TOR.

Budget and Schedule for Payment

The budget for this evaluation is a maximum of £18,000, with some space for additional local logistics costs. The budget should include all professional fees and costs, administration and support costs, report production costs, and any international flights, ground travel, visas etc. The budget should provide include a breakdown of costs for all consultants, and detail of fees, costs and support costs. We prefer more detail than less.

The consultation will be paid the consultation fee upon completion of the following milestones:

• Signing the contract: Payment equivalent to around 30% of contract value
• Approval of the final report: the balance of the contract value.

Required expertise and qualification for consultant / team

• A Master’s Degree in a related subject and/or comparable experience.
• Demonstrated experience and ability in research methods, both quantitative and qualitative, and statistical analysis.
• Extensive expertise and experience in the field of evaluation of development programmes or similar.
• Demonstrated experience of designing, leading and conducting similar evaluations. At least one example of previous work should be provided with the proposal.
• Significant demonstrated experience and knowledge of the fields of gender equality analysis, disability inclusion and inclusive education, the context for education in Sub-Saharan Africa or South Sudan in particular, and application of these to monitoring, evaluation and learning.
• Demonstrated ability to communicate in English (written and verbal).
• Knowledge and experience of working in context of South Sudan is desirable.
• Working knowledge of local languages in Western Equatoria is desirable.