Terms of Reference: Quantitative perception survey in Northwest Syria

Summary
1.1 Ground Truth Solutions (hereinafter: GTS) seeks the services of a research firm/organisation for the implementation of a perception survey in Northwest Syria. This quantitative data collection is part of a broader process to co-create a community-led system for dialogue and feedback. This process aims to engage diverse community members in discussions of what a trusted, community-led, effective, and inclusive feedback process could look like. Ideas that emerge during the discussions will feed into the creation of a prototype which will be piloted and discussed with community members in an iterative process.

1.2 The qualitative process described above will be complemented by perception surveys across Northwest Syria. The combined findings from the quantitative survey and qualitative study will be used to design and pilot a sustainable, locally-led system for dialogue and feedback in Northwest Syria, as well as to amplify communities voices and share their insight with those who take the decisions that affect the aid and services they receive, including UNICEF and its implementing partners, other UN agencies, local civil society organisations and other stakeholders.

1.3 The requested services include the finalisation of the survey instrument together with GTS, contribution to GTS’ sample design, inclusion of enhanced questionnaire testing (including cognitive testing + piloting), data collection (face-to-face surveys), and translation of any open-ended responses into English. The services will include one round of face-to-dace data collection in 50 locations within the following 36 sub-districts in Northwest Syria with an overall sample of 1,500 surveys: (Bulbul, Raju, Ma’btali, Sheikh El-Hadid, Jandairis, Sharan, Afrin, Ar-Ra’ee, Al Bab, Tadaf, Aghtrin, Suran, A’zaz, Mare’, Jarablus, Ghandorah, Daret Azza, Atareb, Ariha, Ehsem, Mhambal, Harim, Dana, Salqin, Kafr Takharim, Qourqeena, Armanaz, Bennsh, Teftnaz, Maaret Tamsrin, Sarmin, Idleb, Jisr-Ash-Shugur, Badama, Darkosh, Janudiyeh.)

1.4 The sample will include 50 locations, 11 of which will be major cities within the above sub-districts (subject to amendments based on discussions with data collection firm): Idleb, Atma, Afrin, Deir Hassan, Darhashan, Qah (Co), Sarmada, Al Bab, Azaz, Kelly, Bab El Hawa, Salqin. The sample size for these locations will be allocated proportional to their size, ranging from 25-75 per location.

1.5 In addition, 39 more locations will be randomly selected by GTS within the 36 sub-districts using probability proportional to size, with a sample size of 25 per location.

1.6 The sample will be stratified by displacement status (IDP on site/off-site, general population). Depending on location size, sampling at site level will be done using random walk or using randomly generated GPS points, where interviews will be conducted. For each location, one scoping day should be budgeted, during which IDP sites, locations of IDP off site will be mapped, to ensure that IDPs can be found and random walk approach and sampling using randomly generates GPS points can be implemented. The service provider will be asked to provide the results of the scoping to GTS before data collection ensues.

2. Purpose and scope
2.1. The purpose is to capture the perceptions and views of the population in Northwest Syria about the accountability of the response, existing engagement and feedback processes, and what a community-led two-way feedback mechanism could look like. The survey tool will be supplied by GTS and designed based on the findings of the qualitative phase.

2.2. In carrying out this assignment, the appointed data collection company will be expected to:
a.Work closely with GTS in the finalisation of survey tools (including translation of survey questions from English into Arabic).

b. Appoint, train (with GTS guidance), and manage a team of capable enumerators (ideally half of which should be women) with appropriate education levels and experience in collecting both qualitative and quantitative data from people affected by humanitarian crisis, ideally also with experience in the use of cognitive interviewing. Staff conducting the training will be expected to participate in a ‘Training of Trainers’ session with GTS prior to training the enumerator team. The data collection company needs to ensure adequate behaviour of enumerators and appropriate language skills or translators to accompany them to work with ethnically diverse groups of affected people.

d. Ensure that enumerators are provided with materials and training they need to enable them to complete the assignment within the timeframe, including training on collecting data with SurveyCTO and the survey provided by GTS.

e. Demonstrate that it has a methodology for testing the survey. The testing phase must include:
i. A Cognitive Testing Phase: To test understanding of the questionnaire; to assess the ease/difficulty with which respondents understood the question and to provide suggestions on how to improve the questionnaire. Interviews should take place within one day. Cognitive test scores and recommendations for changes to the survey should
be shared and discussed with GTS.
ii. A pilot phase: The pilot phase will be carried out using the finalised survey tool and should involve a sample of 50 to 100 participants. The questionnaire to be used with the SurveyCTO software will be supplied by GTS.

f. Facilitate face-to-face surveys with recipients and non-recipients of humanitarian aid, based on the random walk approach and sampling using randomly generates GPS points shared by GTS.
i. The independent service provider must explain how it will be able to work with the GPS coordinates to identify the locations provided of where the surveys will take place (including mapping applications, which are operational independently of the telephone connection). to ensure random sampling in locations pre-identified by GTS.

g. Ensure enumerators submit data via SurveyCTO directly after each interview and share with GTS brief daily updates where necessary on progress and/or challenges experienced in the field.

h. Submit a final report after data collection covering the number of successfully completed interviews and the number of unsuccessfully completed interviews, adherence to agreed survey and sampling strategy, and any relevant observations made and issues experienced by the data collection company and its enumerators.

3. Quality assurance
The following measures will be taken in order ensure that the data quality is of a high standard and accurately captures the opinions of affected people while respecting the sampling strategy and other guidance shared by GTS. The independent contractor must also describe in detail the data quality checks it carries out on a daily basis (how it has done so for previous data collections) and how it identifies poor quality data and fraudulent interviewers.

a) Collaborate with GTS to develop enumerator training materials that reflect the enhanced questionnaire testing required.

b) Ensure that all interviews are an appropriate length determined upon finalisation of the questionnaire. GTS reserves the right to reject any interviews under a pre-determined appropriate length which does not adhere to our quality assurance standards.

c) Interviews should be submitted immediately following the conclusion of the interview before beginning the next interview. GTS reserves the right to reject any interviews over what GTS deems an excessive length of time which does not adhere to our quality assurance standards.

d) Submit 50-100 initial data points including translations for quality assurance after the test day or first day of data collection according to the GTS format guidelines.

e) Respond to feedback by GTS based on daily quality assurance checks which GTS will conduct on the data, and provide feedback to the enumerators, addressing any potential concerns.

f) Ensure sufficient supervision of enumerators conducting surveys to make sure that the project explanation is accurate, questions are being asked in a clear way, and that responses are being accurately recorded. Documentation of the daily quality assurance analysis and/or meeting notes from debriefs with enumerators.

4. Code of conduct
Ensure that all enumerators adhere to the GTS Code of Conduct for Enumerators (Annex A). If any aspects of the Code of Conduct have not been adhered to, GTS reserves the right to withhold payment completely or until data has been collected in line with the above code of conduct.
In cases where the enumerator’s behaviour constitutes a fundamental breach of the Code of Conduct, leading to the safety/wellbeing of participants being compromised or resulting in reputational damage of the organisation, GTS reserves the right to immediately terminate the contract of the enumerator/s and the data collection company.

In cases where it is brought to the attention of GTS that the actions of enumerators have constituted a clear breach of safeguarding practices, (e.g. sexual exploitation and abuse) GTS will also report this to relevant authorities.

The independent service provider must clearly indicate how the data will be collected in a responsible manner which ensures that no harm is caused to any person directly or indirectly involved.

The contractor is required to follow the protection incident referral protocol. Investigators will be trained to identify and report cases safely.

The independent contractor must have a protocol and risk assessment protocols in place for the protection of its interviewers and how GTS will be informed and involved where appropriate.

5. Survey methodology and sample design
5.1. There will be around 20-25 questions, including some demographic questions. Questions will be either multiple choice or binary/Likert scale questions and, will include some open-ended questions.

5.2. The field researchers will identify themselves as working for an independent organisation and as completely independent from the aid organisations working on the ground in Northwest Syria.

5.3. No respondent is required to identify himself or herself by name, and no reader of the survey data (or subsequent reports we will write) will be able to identify what any individual respondent has said.

5.4. The field researchers will use cognitive interviewing in an initial testing round to ensure respondents fully understand the questions asked of them.

5.5. Surveys will be conducted in the regions listed under 1.2. Summary: (tbd)

5.6. Sample size: approximately 1500 surveys with men and women, aged 18 and above will be conducted in total across specified areas in Northwest Syria using a random sampling approach. Gender and status (IDP on site, IDP off site, general population) will be representative of the target population.

6. Deliverables
6.1. The expected deliverables for the data collection are:
a) verification and translation of survey tool;
b) completion of face-face data collection;
c) submission of translated open-ended questions;
d) a short report after the data collection covering the number of successfully and unsuccessfully completed interviews, adherence to agreed survey and sampling strategy, and any relevant observations and issues;
e) completion of enumerator survey.

7. Timeframe
The data collection is scheduled to commence beginning of February 2024, with submission of the data no later than 1 March 2024 (including translations of open-ended questions).

8. Contractual arrangements
8.1. The agreement will be between the appointed data collection company and GTS, and it will specify quality and assurance checks on the data. The agreed overall budget will be paid in full upon satisfactory completion of the subtasks specified within the agreement. Failure to satisfactorily complete the tasks may result in the agreement becoming void.

8.2. The data collection company and its staff will be expected to conduct themselves in accordance with the requests and guidelines of GTS, when moving around and engaging with affected people. Before surveying can begin, all necessary permission and consent need to be sought from the relevant authorities and/or organizations operating in the country.

8.3. All data and work product produced during this engagement will be the sole property of GTS, and the data collection company will have no authority to use or disseminate any of it without prior approval by GTS.