Introduction

Generally, African societies value boys and men more than girls and women, respectively. Close attention is paid to the behaviour and attitude of boys and girls from as young as 3 years, where observed signs of deviant behaviours and attitudes are punished. Social exposure prepares boys as men and girls as women and introduces a wide range of deeply rooted inequalities between boys and girls, and in later stages, between men and women. For instance, girls are expected to undergo harmful practices in preparation for their status in society; higher attention is given to the views and opinion of boys and men than girls and women; and better access to socio-economic opportunities are accorded to men and boys than women and girls.

Somalia is one of the African countries with some of the most severe norms and practices, and as such the country is listed in the top five on the gender inequality index[1] (0.777 out of 1). The index looks at three dimensions: reproductive health, empowerment, and labour market participation, with Somalia scoring poorly on all three dimensions. In Somalia, gender inequality exists in economic, social, and political domains with a wide access gap between men and women in economic and social fields. For instance, male leaders are preferred and seen as better decision-makers than female leaders. This widespread preference for male political leaders over female political leaders has hindered the realization of the 30% reserve quota. According to the CHANGES 2.0 project annual reports[2], despite the increased number of women aspirants in the 2022 parliamentary elections, women managed to secure just 20.6% of the 315 seats in both houses.

Discriminatory practices are still common in Somalia and disproportionately affect girls and women more than boys and men. Existing norms expect women and girls to do most of the household chores such as cooking, washing utensils and clothes, or making beds, and give boys and men more free time to interact and socialize with their peers or concentrate on their personal goals and aspirations. According to CHOICES 2.0 project pre-post survey, girls spend about 4.8 hours on household activities compared to about 3 hours for boys.

Gender disparity in education and gender stereotypes are also common, negatively impacting girls and women. Most Somali girls, especially those from rural communities, hardly finish their primary or secondary education due to limited educational opportunities, household poverty and gender stereotypes that discourage girls from pursuing their dreams or limit their chances of achieving their goals. Adolescent boys also believe that they deserve more educational opportunities than girls or that they can outperform girls.

Female genital mutilation (FGM), classified into Pharonic (most severe) and Sunnah types, is the most common and widely supported discriminatory practice in Somalia. FGM has been challenging interventions due to its superior adaptive capacity. Whereas support for and prevalence of Pharonic FGM has declined over the years, support for and prevalence of Sunnah FGM has increased. Social and behaviour change experts have linked this unfortunate outcome to flaws in advocacy and campaign design which treated FGM and Pharonic FGM as the same, and focused on the negative consequences of Pharonic FGM to discourage its practice. This has indirectly validated Sunnah FGM and has motivated the medicalization of FGM.

Child Early and Forced Marriage (CEFM) is another common discriminatory practice in Somalia. However, unlike FGM, the support for CEFM has been declining in past decades. An evaluation by the previous CHANGES 1.0 project in 2021 observed a significant reduction in CEFM prevalence. In addition, community perceptions on the ideal age of marriage for girls has also improved from 18 to 20 years in 2017 to 20-25 years in 2021[3].

Lack of, or poor decision-making skills among adolescent girls is a major challenge in the fight against discriminatory practices. According to CHANGES 2.0 baseline, 20-22% of adolescent girls reported decision-making power regarding marriage, but very few adolescent girls reported to have full influence on whether, who and when they will marry,^[4] with men and women believing that girls should not be given strong influence over marriage decisions. Limited decision-making power in sexual and reproductive health was also observed among adolescents, with four out of ten adolescents reporting feeling confident that they would be able to visit a health facility to obtain information on ASRH, while close to three out of ten indicated that they did not feel confident, and another three out of ten indicating they were unsure. Insufficient knowledge on ASRH rights and life skills among adolescent boys and girls is another major limitation. According to CHANGES 2.0 baseline, only 14-16% of adolescent girls and boys demonstrated sufficient knowledge in critical life skills.

Discriminatory norms and practices in the labour market also affect Somali women and adolescent girls. According to 2020 Human Development Report, Somali men are three times more likely to be engaged in income generating activities than Somali women. Such inadequate access to economic opportunities and poverty are subjecting Somali women to a lower position in society than men, thus giving men grounds to override or overlook women’s contributions during decision-making.

CHANGES 2.0 Overview

The project is carried out in 16 targeted districts and 76 communities (34 SCI, 24 CARE, 16 IRC) from Somaliland, Puntland, Galmudug, HirShabelle, Banadir and Jubaland.

• Save the Children (SCI) implements the project in Somaliland (Hargeysa, Burao) in collaboration with NAGAAD Network; in Puntland (Galkacyo North, Galdogob) in collaboration with Asha Gelle Foundation; in Galmudug (Adado, Abudwak), HirShabelle (Beledweyn) and Southwest State (Diinsor) with Somali Peace Line (SPL)
• CARE International implements the project in Somaliland (Badhan, Erigabo, Buhodle) in collaboration with NAGAAD Network and Tadamun Social Society (TASS); in Jubaland (Kismayo) in collaboration with Somali Women Study Center (SWSC)
• IRC implements the project in Galmudug (Galkacyo South, Hobyo) in collaboration with Daryel Bulsho Guud (DBG); in Banadir (Karan, Howl-wadag) with Save Somali Women and Children (SSWC)

The CHANGES 2.0 project aims at:

• Challenging harmful social norms including FGM and CEFM, and
• Increasing women’s social and economic empowerment.

The CHANGES 2.0 project is composed of four outcome chains designed to bring about the required behaviour, attitude, and perception changes, and improve skills and uplift the economic status of women. The project aims at decreasing discriminatory social norms and practices that perpetuate and validate sexual and gender-based violence against women and girls (Outcome 1100). To achieve this, the project trained adolescent boys and girls (10-19 years) and their caregivers to improve their attitude towards positive social norms. Specifically, adolescents were trained on CHOICES (+) curricula, while caregivers were socialized on gender equality and positive social norms using the SC VOICES videos. The project sensitized religious and traditional leaders on key messages and awareness raising activities to improve their commitment towards positive social norms. Community influencers developed and implemented action plans to prevent violence against women and girls in their communities.

Secondly, the project trained adolescent boys and girls on adolescent sexual and reproductive health and rights (ASRHR) and life skills to improve their voice and agency to make and act on life decisions for themselves, including with regards to ASRH and SGBV (Outcome 1200). Under this outcome, the project used the Girl Shine curriculum with adolescent girls (10-19 years) and the Boys Wise curriculum (10-19 years). The project also engaged Women Rights Organizations (WRO) to support adolescent girls and boys to lead advocacy activities on gender equality and rights in their respective communities.

Thirdly, the project aimed to enhance the enabling environment in support of gender equality and women’s and girls’ rights, specifically working with WROs, media associations, and duty bearers at the Federal and regional government levels (Outcome 1300). Under this outcome, the consortium strengthened the capacity of WROs to influence and hold government officials accountable to their mandate and promises on gender equality, in addition to sensitizing law enforcement officers and gender desk staff officers on existing and upcoming gender equality laws and policies. Furthermore, the project trained media associations on existing and upcoming gender equality laws to increase their capacity and willingness to inform the public and hold government officials accountable on their mandate to protect the rights of women and girls through the adoption and implementation of gender equality laws at the federal and regional levels.

Finally, the project aimed to enhance the social and economic empowerment of women to improve their participation and involvement in household and community decision-making (Outcome 1400). The project trained women in setting up and managing Village and Savings Loan Association (VSLA) groups, and on business skills, savings practices, and accessing credit services. The project distributed tools and equipment to women to help them start and run their own businesses. The project also facilitated gender discussion sessions for VSLA women and their male household members to improve male attitude towards women’s decision-making power in the household.

Endline Evaluation Purpose and Objectives

The overarching objective of the endline evaluation is to evaluate the performance of the ultimate, intermediate and immediate outcome indicators against the milestone targets and the baseline as specified in the PMF.

Performance Measurement Review Indicators:

The Endline Evaluation will cover the following project indicators:

Ultimate Outcome Indicators:

• % of girls who have ever been married before their 18^th birthday
• % of girls who have undergone any form of female genital mutilation or cutting (FGM)
• % of VYA and OA girls (and boys) who report they are empowered according to the Power Index

Intermediate Outcome Indicators:

• % of VYA and OA girls (and boys) who report existence of discriminatory social norms and practices
• % of VYA and OA girls (and boys) who consider a husband justified in hitting or beating his wife
• % of caregivers who have taken at least one action against child marriage in the past 6 months
• % of VYA and OA girls (and boys) who report decision-making power with regard to marriage
• % of VYA and OA girls (and boys) who report decision-making power with regard to ASRH rights
• % of men and women who are not in favour of Pharonic / Sunnah FGM/C
• No. of actions – e.g. national/state action plans, policy statements, public declarations, enforcement practices, budget allocations – taken by the government at the federal and state levels to implement relevant GE laws related to FGM/C and CEFM
• Level of commitment by federal and regional governments to adopt and/or implement relevant GE laws
• % of women and men, girls and boys who feel that the government has made progress in adopting and/or implementing relevant GE laws that support women and girls’ rights
• % of women who have started or sustained an income generating activity after receiving project-related training and inputs
• % of women who decide how their income earned from IGA  will be used
• % of women who report decision-making power with regard to household and community decisions

Immediate Outcome Indicators:

• % of VYA and OA girls and boys who hold positive attitudes towards gender equality and equal rights for girls and boys (gender construct, access to education, FGM and bodily autonomy, age of marriage)
• % of VYA and OA girls and boys who hold positive attitudes towards equal roles and responsibilities of women and men (household decision-making power, division of household chores, income generating work, women roles, leadership positions)
• % of caregivers and community members who hold positive attitudes towards gender equality and equal rights for girls and boys (gender construct, access to education, FGM and bodily autonomy, age of marriage)
• % of caregivers who hold positive attitudes towards equal roles and responsibilities of women and men (household decision-making power, division of household chores, income generating work, women roles, leadership positions)
• % of VYA and OA girls and boys who demonstrate knowledge of ASRH rights, SGBV and life skills
• % of men and women who hold positive attitudes towards women’s decision-making power in the household

Evaluation Criteria

The following key questions will guide the endline evaluation’s assessment of the project against the DAC Criteria for Evaluating Development Assistance. The consultant will be asked to review and update the existing data collection tools (from the baseline) to incorporate the questions needed to answer the following DAC criteria questions:

Effectiveness: The extent to which the project attained its outcomes.

1. To what extent were the outcomes achieved? To what extent were the indicator targets achieved?
2. What were the major factors influencing the achievement or non-achievement of the outcomes?
   • o   How effective were the consortium structures and approaches in the systematic coordination and engagement with external stakeholders and actors.

Efficiency: The extent to which the project used the least costly resources possible in order to achieve desired results, considering inputs in relation to outputs.

1. Assess and establish the efficiency and specific and combined contribution of intervention models (CHOICES (+), Girl Shine/Boys Wise, SC VOICES videos, and EASE) in changing the behaviour, attitude, and knowledge of the respective target groups.
2. What, if any, challenges in project implementation were encountered, including managerial, organizational and any other unforeseen factors?
3. To what extent did the project collaborate with national and sub-national partners and stakeholders (technical, advocacy, funding, etc.) to achieve results in an efficient manner?

Impact: The positive and negative changes produced by the project, directly or indirectly, intended or unintended.

1. What has happened as a result of the project – either intended or unintended, positive or negative?

Relevance: The extent to which the project was suited to the priorities of the target beneficiary group(s), stakeholders, and to the donor.

1. Was the project relevant to the needs of the beneficiaries, as identified at the project inception/design stage?

 Sustainability: The extent to which the benefits of the project (outputs, outcomes) are likely to continue after project completion.

1. What is the likelihood of the continuation and sustainability of the project outcomes and benefits after completion of the project?
2. What commitments (financial, human resources, etc.) have been made by stakeholders to maintain or improve results?
3. How will improvements in stakeholder knowledge, attitudes, capacities, etc. contribute to maintaining results?
4. o   To what extent is support available from the external environment to maintain or improve results?

Study Design and Sampling Framework

The endline evaluation will adopt a mixed methods approach. The quantitative method will use a sample of 5,300 individuals divided equally between 50 intervention communities and their 50 comparison/control communities. The sample covers four strata: 1600 Very Young Adolescents (800 girls and 800 boys), 1600 Older Adolescents (800 girls, 800 boys) and 1600 caregivers (800 women, 800 men) and 500 community leaders (250 women, 250 men) stratified across the intervention and comparison groups. A caregiver and community leader survey will used to collect data from adult men and women, while an adolescent survey will be used to gather relevant data from the adolescent boys and girls.

A tracking list containing relevant contact information of the caregivers and community leaders will be used to track respondents at the endline. Adolescent boys and girls will be traced through their caregivers and unavailable respondents will be replaced. For the comparison communities, replacement matching the characteristics of the respondents (age, sex, education level and economic status) will be randomly selected from the community members. In the intervention communities, replacements will be randomly selected from the project beneficiaries e.g. a replacement for an OA boy will be selected only from the OA boy beneficiaries.

Table 1: Quantitative Sample Size

For the qualitative method, the endline evaluation will utilize key informant interviews (KII) with government officials, members of parliament, and women rights activists; and focus group discussions (FGD) with beneficiary groups to collect additional information to complement the quantitative methods and assess presence and extent of community level conversations between direct and indirect beneficiaries.

Scope of Work

1. The qualitative consultant will be required to collect and analyze the qualitative data (KIIs and FGDs) and produce a qualitative report which will be shared with the evaluation consultant. A separate TOR will be posted for this consultant.
2. The evaluation consultant (the current TOR) will be responsible for analyzing the quantitative data, and utilizing the findings from the qualitative report to produce the Final Endline Evaluation Report for the CHANGES 2.0 project.

The implementing partners (SCI, CARE, and IRC) will be responsible for the quantitative data collection (adolescent, caregiver, community leader surveys) in their respective locations with the Consortium MEAL manager overseeing the overall data collection exercise. The MEAL focal points from the implementing partners will be responsible for training the enumerators, supervising data collection and conducting data quality checks.  Quantitative data will be collected using Kobo and the data collection account will be under the custody of the Consortium MEAL manager, with access limited to the MEAL team and the evaluation consultant.

At the consortium level, the evaluation will kick off with a three-day Training of Trainers (ToT) workshop for the 10 MEAL staff, consisting of the Consortium MEAL manager, the MEAL focal persons of the implementing partners and the local partners. In the workshop, the team will,

• Go through the evaluation study design and methodology in addition to reviewing quantitative and qualitative collection tools (.docx and. xlsx versions) and data quality controls,
• Develop field plans encompassing enumerator training, field supervision and safety and security risk mapping of the field team,
• Review, appropriate and finalize the quantitative data collection budget
• Review and update the caregiver and community leader, and adolescent survey respondent tracing lists, and
• Update the intervention and comparison communities as per the intervention.

After the workshop, each implementing partner will hire and train their enumerators in their locations and will take lead in managing logistics and ensuring data quality of their respective working districts under the project. The implementing partners will cover the field costs in their locations. The implementing partners will hire an equal representation of male AND female enumerators, to ensure that women and adolescent girls are interviewed by female enumerators, and men and boys are interviewed by male enumerators. SCI will be responsible for the overall supervision of the field data collection.

Distribution of Roles and Responsibilities

• provide the evaluation consultant with relevant and requested documents including project proposal, PIP, Annual Reports, Work Plans, PMF, LM, Baseline Study, Midterm Review
• review and approve inception report and survey tools,
• translate survey tools into Somali
• provide any other support needed by evaluation consultant
• provide the evaluation consultant with
  1. clean data sets in Excel,
  2. an indicator analysis plan documenting how each indicator is to be measured;  and
  3. the qualitative report in English, containing an analysis of the FGDs and KIIs
• review draft report and provide constructive feedback to the evaluation consultant
• approve and sign-off on the Final Endline Evaluation

The evaluation consultant will be required to:

• review project documents, including but not limited to: Project Implementation Plan, Logic Model, Performance Measurement Framework, Baseline Study, Annual and bi-annual Reports, Work Plans, annual budgets and expenditures, data collection tools, etc.
• review and update the existing data collection tools to incorporate the questions needed to answer the DAC evaluation questions.
• produce an inception report demonstrating his/her technical capacity to handle the evaluation by explicitly explaining his/her evaluation methodology and design in line with the details on these already provided above. The inception report should exhaustively cover the proposed design and methodology, analysis plan, suggested final report format among others.
• prepare and submit final draft report following the CHANGES 2.0 Baseline report and Midterm report formats. The consultant should work closely with the Consortium MEAL Manager and respond to the feedback in good time.
• produce four thematic factsheets (max 2 pages) aligned with the project impact indicators (FGM, CEFM and Women Economic and Social Empowerment and Adolescent Sexual Reproductive Health and Rights) summarizing the major approaches and findings of the project.

Qualification and Experience

• Masters in Gender Studies, Development Studies, Public Health, or Social Science with demonstrable experience in GBV and CP Humanitarian projects or gender equality. Alternatively, master’s degree in development studies, social sciences with statistics and economics background will also be accepted.
• Demonstrable experience in working with children preferably in conflict prone countries such as Somalia.
• Advanced data analysis skills in regression and econometric analysis using STATA, R or SPSS,
• Considerable record of accomplishment and proven experience in quantitative and qualitative methods of data analysis and data management.
• Profound experience and technical knowledge in gender equality, adolescent sexual reproductive health and rights, and social and economic and social empowerment of women
• Previous experience in Somalia is an added advantage,

Duration of Consultancy

Tentative work dates:

• July: Posting of TORs for the Evaluation Consultant
• August: Contracting of Evaluation Consultant; contract signed by end of August
• August: Consortium MEAL Manager and SCC MEAL Advisor finalize data collection tools, with input from Evaluation Consultant (quantitative and qualitative)
• July-August: SCI, CARE, IRC prepare logistics for data collection – hiring enumerators, developing logistics plans, identifying beneficiaries
• August: Posting of TORS for the Qualitative Consultant
• September: Contracting of Qualitative Consultant
• September: Quantitative data collection takes place in September for 3 weeks
• September: Data cleaning 1- 2 weeks
• September-October: Qualitative data collection takes place
• October: Quantitative data ready for the endline consultant to start work
• October: Qualitative analysis takes place
• November: Qualitative report shared with Evaluation consultant
• October