Consultancy for Endline Evaluation of the Project OVOC: Women and Youth for Democratic Civic Space

Terms of Reference

Consultancy for Endline Evaluation of the Project

Our Voices Our Choices: Women and Youth for Democratic Civic Space

1. About Plan International in Asia Pacific

Plan International is an independent development and humanitarian organization that advances children’s rights and equality for girls. We believe in the power and potential of every child and young people, but this is often suppressed by poverty, violence, exclusion and discrimination. And it is girls who are most affected. Working together with children, young people, our supporters and partners, we strive for a just world, tackling the root causes of the challenges facing girls and all vulnerable children. We support children’s rights from birth until they reach adulthood. And we enable children to prepare for – and respond to – crises and adversity. We drive changes in practice and policy at local, national and global levels using our reach, experience and knowledge. We have been building powerful partnerships for children for over 75 years and are now active in more than 70 countries. Plan International has been working in the Asian region since 1948 to enable marginalized children as well as their families and communities, to access their rights to health, education, economic security and protection. Currently, our programs are running in 15 countries in Asia and the Pacific together with our development partners.

1. About the project

Plan International is implementing a project titled “Our Voices Our Choices: Women and Youth for Democratic Civic Space”. The project, hereafter referred to as Action, supported by the European Commission (EC), is being implemented in 3 countries (Bangladesh, Nepal, Thailand, together with regional outreach with Plan’s Asia Pacific regional team (APAC)) The overall coordination is through a consortium of Plan and its civil society partners coordinated through Consortium Lead hosted by Plan International Thailand. Plan International Netherlands (NLNO) is the affiliated National Office serving as contract holder and direct contact with the European Commission.

The overall objective of the Action is: to support and empower civil society in protecting and promoting democracy, human rights and fundamental freedoms worldwide. The overarching goal is to contribute to the EIDHR[1] Multiannual Indicative Program (MIP) 2018-2020[2], in particular Objective 1. ii, 2. ii10, and 3.i&iii by ensuring that young people and especially women and girls in target communities in Bangladesh, Nepal and Thailand (also the broader Asian region) enjoy fundamental human rights and are actively engaged in improving their lives and communities through inclusive, effective, pluralistic and representative civic participation with a focus on inclusive civic space. Inclusive civic space plays a central role in our Action. Our Voices Our Choices: Women and Youth for Democratic Space (OVOC) aims to address the needs and constraints of youth and women in all their diversity including Dalits, LGBTIQ+, persons with disabilities and other excluded groups, as they are often overlooked, suppressed, underestimated, and excluded on decision making, especially national policies that affect them.

Throughout the design, implementation and evaluation, this project will continue to assess the ability of rights holders to claim their rights and of duty bearers to meet their obligations in advancing and safeguarding those rights. This project and its outputs will compliment and inform the AICHR[3] 2021-2025 Workplan, the drafting of the ACWC[4] Workplan, and mechanisms in SAARC to support equitable, inclusive and open civic space throughout the Asian region.

2.1 Target groups

The project has directly targeted both rights holders and duty bearers in each of the countries of intervention (Bangladesh, Nepal and Thailand). The key right holders of the action are 13,498 youth aged 15-29, 605 journalists, 14 national media platforms, 220 human rights defenders (HRDs), and 461 youth/women-led CSOs, 1 National Youth Commission, 1 Dalit Commission and 1 National Women Commissions residing in Bangladesh, Nepal and Thailand. Also included are 44 duty bearers consisting of 8 ministries, 16 ward offices, 21 municipalities/local authorities and 3 National Human Rights Commissions across the three countries and 1 parliamentary caucus.

At the regional level, Plan International Asia Pacific Regional Hub (APAC) will engage regional civil society organizations and bodies focusing on human rights, children and young people’s rights, and gender equality will also be engaged for policy changes and intergenerational dialogues, among others. This includes but not limited to the ASEAN Intergovernmental Commission on Human Rights, the ASEAN Committee on the Promotion and Protection of the Rights of Women and Children, Southeast National Human Rights Institutions Forum (SEANF), and South Asian Association on Regional Cooperation.

1. Purpose of the Evaluation

The purpose of the evaluation is to establish endline value against each indicator given in 3.1

(Indicators) The endline evaluation has specific objectives as below:

1. To measure the progress towards achieving the specific objectives, and impact of the project, focusing on key indicators of the project.

2. To compare the baseline findings with the endline and measure the changes/improvement in comparing with its indicators that the project contributed.

3. To identify and document lessons learned, success stories/case studies and make appropriate recommendations for any adjustment and improvement for future project/s.

4. To recommend best practices-which of the interventions, approaches, and modalities/strategies that have been most effectively implemented by the project.

5. To assess the sustainability of project results, impact, and approach at different levels.

   Scope of work

   The scope of work includes designing and implementing a qualitative method endline evaluation to achieve evaluation purpose and objectives. The end line evaluation will be built on the results of the OVOC Project baseline study. This includes primary and secondary data collection with relevant stakeholders. The evaluation should cover the 3 project countries including Bangladesh, Nepal, Thailand and the Asia region. This should include the 6 mentioned target groups.

   The consultant will work closely with the OVOC Consortium Lead and Plan International Netherlands PMEL and will coordinate the endline evaluation with Plan International’s teams in Bangladesh, Nepal, Thailand and APAC. The consultant will have overall responsibility for organizing, managing and leading the endline evaluation and producing a compiled quality report. Plan International Bangladesh, Nepal, Thailand and APAC’s teams will be involved in finalizing the methodology and tools.

   The scope of work will preferably include, but not limited to:

   Prepare an inception report that will include the final methodology aligned with objectives, and timeframe and submit it to Plan International for review and approval. A section on the assessment of anticipated risks during the study period and mitigation strategy should be included in the inception report;

   Review key project documents, including the Project’s Logical Framework, baseline study report, and various reports prepared and submitted by the project;

   Review, adapt, translate, format and pilot a qualitative data collection tool;

   Recruit qualified enumerators and organize training for them in data collection (including ethics and child protection) and quality control.

   Administer data collection in the field and ensure a mechanism of quality control during data collection.

   Produce a draft report and share it with Plan International and its stakeholders for feedback. Finalization of the report by addressing the feedback.

   Present key findings, impacts and changes in different levels to Plan International staff members and stakeholders, especially during the project’s validation and sharing workshop of the project.

   Prepare success stories/case stories (2 cases/ countries) in English, Bengali, Nepali and Thai in consultation with OVOC Consortium Lead and Country Offices PMERL Staff.

   Maintain regular communication with the key contact person(s) of Plan International throughout the assignment period.

   Methodology

   The study is expected to use a qualitative method with relevant data collection methods like; Focus Group Discussions (FGD), Key Informant Interviews (KII) and In-depth Interviews (IDI) and should be in consultation with the OVOC Consortium Lead and Plan International PMEL. Two case studies will be required for each country.

   The methods and approaches that have been used to assess the indicators during endline must be considered when developing the overall evaluation methodology. To assess evaluation criteria, Plan International also welcomes new methodologies and tools and baseline tool modifications. The consultant/consultancy should work closely with the OVOC Consortium Lead and Plan International PMEL to ensure that the methodology meets their expectations and requirements.

   The data collection methods can include but not be limited to the following:
   Desk review: Review of relevant literature that provides a picture of the current status quo on democratic civil space. The study will also review relevant documents of the projects being implemented.

   Focus Group Discussion (FGD): FGDs are recommended with youths, HRDs, civil society members and other relevant government stakeholders.

   Key Informant Interview (KII)/In Depth Interview (IDI): KII/IDI is recommended with Youth, Youth/Women led CSOs, HRDs, Journalists, Ministry representatives, local authority leaders, Mayor of Rural/Municipality, Ward chairpersons, relevant focal persons in respective administration areas, and other relevant government and non-government stakeholders. Conducting KII/IDI could be done online/ on-site as agreed during the inception meeting.

   Survey: Survey with youth and women led CSOs (if applicable)

   Validation and Sharing Workshop: the endline findings should be validated through consultations with key staff of partner organization, youths, youth/women-led CSOs, staff of local government agencies. The preliminary findings of Nepal shall be present to Nepal team by end of May 2024 while the multi-country workshop (online) should be conducted for sharing of the findings and refining recommendations before the finalization of the report in later months.

   The consultant is expected to provide an inception report with a detailed study methodology in consultation with the PMEL colleagues/local consultants of the 3 involved countries and the Regional Hub (APAC). The methodology will contain study design, study area, study population, sample size, data collection methods and tools, and ethical considerations. The inception report should also include a detailed timeframe mentioning all the milestones of the study.

   The consultant is encouraged to propose other suitable participatory methods not limiting to those provided in this ToR.

   Users
   The intended users of the final report are Plan International, Plan partner organisations, local government agencies, Plan National Offices, and EU Commission.

   Methods for data collection and analysis

   The consultant should describe in their proposal regarding the data collection process in detail. Similarly, the consultant is responsible for data cleaning, data quality assurance, data transcription and data analysis, which they must clearly mention in the proposal. Further, the consultant should clearly describe data analysis methods including software for quantitative data and qualitative data analysis.

   Key informant selection

   The consultant should explain key informant selection and recruitment process in the inception report which represent target groups from 2.1 which are youths, youth/women-led CSOs, journalists, human rights defenders, representatives of local and regional governmental agencies, and representatives of partner organizations. To ensure the consultant seeks these views in the study process, outline key stakeholders and target groups expect to be involved.

   The consultant could also identify and suggest additional or excluded stakeholders that they feel are important to involve in ensuring they get the information they need to answer the Study Questions.

   Quality Assurance, Risk Mitigation and other considerations

   It is expected that the study team will use gender-inclusive and participatory approaches to seek the views of beneficiaries and, where appropriate, non-beneficiaries. Inclusive techniques will be expected from the consultant, to seek active participation in the study by beneficiaries and stakeholders at different levels.

   Upon signing of the contract, the consultant and Plan International (Consortium Lead and NLNO) will have an inception meeting to ensure that both parties have the same understanding of the study objectives, how these objectives are to be achieved, and when.

   Consultant must also explicitly provide details on data quality assurance at the field and assurance of the quality of final report in the proposal. Also, potential risks and limitations of the study and consultants’ plan to mitigate them should also be specified in the proposal. Further, the consultant should mention data analysis, data management and data security in their proposal.

   Outputs/Deliverables

   The following deliverables are expected from the consultant in the due process of the assignment:

   a) Inception Report: the consultant should submit an inception report within 7 working days after contract signing detailing description of methodology tools and detailed work plan. The following structure of the inception report is recommended:

   Background

   Purpose of the study/evaluation

   Key study/evaluation questions

   Methodology

   1. Study design, study area, study population, sample size, data collection tools and data collection procedures
   2. Study matrix table (that links the study variables/questions as well as methods to the study objective)
   3. Analysis technique (explain both qualitative and quantitative, and disclose use of software such as SPSS, Nvivo etc), data management and data security mechanism.
   4. Ethical considerations

   Questionnaires or interview forms.

   FGD and KII guides will also be submitted for review at this stage.

   b) Data Analysis and Reporting: A draft report should be shared for feedback. Upon submission of a draft report, the consultant should be responsible for presenting the most significant findings to Plan International and key stakeholders through a meeting and getting their feedback on critical areas. The meeting will be arranged by the consultant.

   The Final Evaluation Report (page limit 60 or agreed during inception meeting) should consist of the following sections (a) executive summary not more than two pages (b) the context/introduction (c) description of the project/program (d) objectives (e) methodology (f) study analysis (g) findings and conclusions structured based on qualitative and quantitative findings (h) conclusion (i) major gaps (j) lesson learnt and recommendation (k) appendices.

   Furthermore, the consultant should submit the report brief 2-3 pages (English, Bengali, Nepali, Thai version) in separate word file along with final report.

   The consultant should submit a final report along with all data (csv, excel, SPSS or STATA compatible formats; raw and cleaned), Completed consent forms, English translated transcripts of the FGDs and KIIs and IDIs, and Final Data Collection Tools (English, Bengali, Nepali, Thai version).

   Appendices should include study terms of reference, sample framework, end notes (where appropriate), bibliography and quotes from the stakeholders including relevant photos.

   c) Case stories (2 cases per country)

   d) Presentation slides (including presentation of findings during learning sharing workshop)

   Timeframe and estimated working days

   The evaluation should start after signing the contract. The timeframe of the evaluation is 1 May – 31 August 2024.

   Within this timeframe, all activities should take place listed in the table below. Importantly, in Nepal data collection and preliminary findings need to be finished by May (according to Nepal’s project specific timeline) while data collection in Bangladesh shall be conducted in June and Thailand/APAC by July.

   Endline duration: Tentatively May– August 2024

   Qualifications and experience of the consultant

   The consultant(s) should have the following academic qualifications and experience:

   Advance degree in Education, Sociology, Development Studies or related fields relevant to assignment. Experience of conducting similar evaluation will be preferred.

   Strong qualitative and quantitative data and analysis skills and experience.

   Experience participatory data collection methodologies and good facilitation skills.

   Strong communication skills, both written and oral (knowledge of communities’ language is an advantage)

   Gender balance should be maintained in the study team.

   Language skills: Proven proficiency in English. Professional level of written English.

   Experience in coordinating with local consultants from the Asian region.

   Similar work experience on human rights initiatives or a similar work experience in a project funded by EU.

   Living or staying in the Asian region is an advantage (preferable). rt

   Evaluation/ Selection Criteria:

   The following selection criteria will be used for technical evaluation of the proposals.

   Does the consulting firm/team leader have experience conducting relevant study? 25%

   Does the team leader have relevant academic qualifications? 10%

   Is the composition of the proposed team balanced in terms of gender and expertise? 10%

   Are the review methods/methodology proposed realistic and relevant to the objectives of assignment? 35%

   Are data analysis methods, data management and security mechanisms well explained? 15%

   Is the proposed field plan (timeline) realistic? 5%