Terms of Reference for the Mid-term Evaluation of the New Narratives 2022-2024: Excellence in Independent Journalism in Liberia
Date: 30th August 2024
1. General Information
1.1 Introduction and Context
Trusted, compelling and rigorous investigative journalism is a foundational pillar of a thriving democracy. Without it leaders are not scrutinized, free to make corrupt and incompetent decisions without consequence. Transparency advocates have no vehicle to spotlight corruption. Populations have no information with which to hold leaders to account.
From the richest settings to the poorest, news organizations are struggling to raise the funds that investigative journalism requires. Low resource settings such as Liberia are hampered by political, economic and infrastructure challenges, and aid interventions that create professional workshop attendees but give little financial support to journalism. Once journalists build skills, confidence and sources they often leave journalism for more stable jobs in government, aid and worryingly, extractives companies that hope to buy off troublesome critics.
In 2022 Liberian independent journalism had reached its lowest point in a decade. In the years after the civil war many media organizations were built on government advertising, “brown envelope” payments for coverage from development organizations and interested parties. A knock-on effect of that business model was that there were few incentives to do original journalism. The 2020 economic downturn, the withdrawal of the UN mission, and the reduced presence of the development community have reduced the income available to Liberian media from traditional sources. Many media shut down. The only way to protect independent, original journalism in Liberia now is to introduce new sources of independent revenue, including donor funding.
A growing movement of news innovators – from the New York Times to Malaysiakini to Premium Times in Nigeria[1] – is rebuilding news businesses using new tools to build revenue and trust with audiences. Bringing these ideas to Liberian newsrooms will be key to building the long-term journalistic independence required if journalists are to utilize capacity building that must be done in tandem. Journalists also need a supportive ecosystem of legal, psychosocial and security support and close relationships with CSOs. An impactful project must be Liberia-led, with a deep understanding of the challenges facing journalists and buy-in from newsrooms.
New Narratives will target 6-8 newsrooms which have demonstrated a commitment to building independent businesses to underpin independent journalism. They will demonstrate best practice in independent journalism and independent revenue generation for peers and share skills and inspire new innovative approaches.
1.2 New Narratives
New Narratives’ (NN) mission is to increase quality independent journalism in low resource settings to arm citizens with the factual information they need to play their role in a resilient democracy. We recognize the fundamental challenge to quality journalism is a broken news business model. For this reason NN supports the journalism ecosystem – lifting journalism skills while driving revenue growth to create a sustainable marketplace for quality journalism.
NN is a U.S. based organization, working in Liberia for 14 years in consortium with Liberian registered NGO New Narratives Liberia (NNL) to build investigative and explanatory journalism. NN’s holistic approach understands that building sustainable media ecosystems cannot be done in a workshop alone. NN uses peer-to-peer mentoring, pairing editors and experts with journalists to build skills while doing investigations. We build the ecosystem players and strengthen relationships with newsmedia – CSOs, academics, newsmedia businesses, technologists, media lawyers, psychosocial and security trainers and fundraisers – to ensure sustainability.
2. Evaluation object and scope
The evaluation is intended to cover the first two years of the New Narratives (NN) project “Excellence in Independent Journalism in Liberia” funded by the Embassy of Sweden in Monrovia. NN calls for this evaluation as part of its agreement with the Embassy of Sweden (EoS) in Liberia.
The NN project uses a combination of capacity building of individual journalists and of media houses, their collaboration with civil society and through providing investigative journalism grants and mentorship to journalists. A total of SEK 6.5 million has been disbursed towards the project which ends in May 2025. SIDA has opened up for NN to apply for a one year extension of the grant to end in May 2026.
The project has two main outcomes that are expected to be achieved by the intervention:
- That Liberian journalists are equipped with resources, know-how and support to pursue independent and investigative journalism during and after the project, and
- That Liberian news organizations and New Narratives have the resources and know-how to enhance financial viability in a sustainable way to support investigative journalism
NN will help build a sustainable investigative journalism ecosystem by:
- Building strong investigative and independent journalism capacity
- Skills and confidence to follow leads, chase key players, gather facts, context and data.
- Psychological resilience and peer support.
- A culture of safety so journalists and newsrooms can make smart decisions to protect journalists and sources.
- Knowledge of international forces shaping stories they cover to put stories in proper context and hold the right actors to account.
- Strong storytelling to make compelling stories that clearly illustrate impact on people.
- Diverse and inclusive reporting teams that foster trust with communities and empower them to use journalism as a tool for justice, equality and change.
- Legal advice pre-publication.
- Building trusted news organizations with large reach and sustainable, independent revenue
- Businesses support the costs of independent and investigative journalism.
- Journalism finds audiences in all forums where they receive news including social media.
- Journalism reaches influencial actors who set the national agenda.
- Collaborations with regional and international newsrooms to share resources, expertise and ensure the maximum audience reach.
- Strengthening civil society collaborations
- Build strong relationships between journalists and civil society actors to work together to hold wrongdoers to account in every jurisdiction where they operate.
Celebrate people and organizations that have found solutions and foster a sense of community action.
NN has been tracking results during the first two years of the three-year project, through its monitoring framework. It has made some changes to adapt to the needs of the beneficiaries.
The scope of the evaluation and the intervention logic or theory of change of the project shall be further elaborated by the evaluator in the inception report.
2. Evaluation rationale
This mid-term evaluation presents an important opportunity for assessing progress in view of the project’s desired outcomes, documenting the results that have been achieved so far – intended as well as unintended. The evaluation will also provide an understanding of the environment in which the project unfolds and recommend any desirable adjustments in project plans for the second half, in order to reach the intended outcomes as effectively as possible.
3. Evaluation purpose: Intended use and intended users
The purpose or intended use of the evaluation is to help assess the success of the project and provide the EoS and NN input to upcoming discussions concerning the way forward.
The primary intended user of the mid-term evaluation is NN. The UoS in Monrovia will accompany the process to ensure full transparency and agreement of feasible adjustments and ways forward.
The evaluation is to be designed, conducted and reported to meet the needs of the intended users and the evaluator shall elaborate in the proposal how this will be ensured during the evaluation process.
4. Evaluation criteria and questions
The objective of this mid-term evaluation is to evaluate the relevance, efficiency, effectiveness, impact and sustainability of the project and formulate recommendations as an input to upcoming discussions concerning the second half of the project.
The evaluation questions are:
Relevance
- To which extent has the project conformed to the needs and priorities of the beneficiaries and donor policies?
Coherence
- How compatible has the intervention been with other interventions in the country, sector or organisation where it is being implemented?
Effectiveness:
- To what extent has the intervention achieved its objectives, and its results, including any differential results across partners?
Efficiency:
- How efficient are the management and implementation modalities employed by NN.
Impact
- What is the overall impact of the project in terms of direct or indirect, negative and positive results?
- To what extent were marginalised communities and people reached by the project?
- Could gender mainstreaming have been improved in planning, implementation or follow up?
Sustainability
- Is it likely that the benefits (outcomes) of the project are sustainable?
- Did the project build capacity of different structures and systems to carry on the activities beyond the project cycle?
Questions are expected to be further developed in the proposal by the evaluator and further developed during the inception phase of the evaluation.
5. Evaluation approach and methods for data collection and analysis
It is expected that the evaluator describes and justifies an appropriate evaluation approach/methodology and methods for data collection in the tender. The evaluation design, methodology and methods for data collection and analysis are expected to be fully developed and presented in the inception report. A clear distinction is to be made between evaluation approach/methodology and methods.
NN’s approach to evaluation is utilization-focused, which means the evaluator should facilitate the entire evaluation process with careful consideration of how everything that is done will affect the use of the evaluation. It is therefore expected that the evaluators, in their proposal, present i) how intended users are to participate in and contribute to the evaluation process and ii) methodology and methods for data collection that create space for reflection, discussion and learning between the intended users of the evaluation.
Evaluators should take into consideration appropriate measures for evaluation design and for groups included or excluded as well as for collecting data in cases where sensitive or confidential issues are addressed, and avoid presenting information that may be harmful to some stakeholder groups.
6. Organisation of evaluation management
This evaluation is commissioned by NN. The intended users are the EoS in Monrovia and NN. The EoS and NN will form a steering group, which has contributed to and agreed on the ToR for this evaluation. The steering group is a decision making body. It will evaluate the inception report and the final report of the evaluation. The steering group will participate in the start-up meeting of the evaluation, as well as in the debriefing/validation workshop where preliminary findings and conclusions are discussed.
NN will be responsible for the management of the contract including all administrative issues related to the evaluation.
To safeguard independence, the steering group will play an ongoing advisory role and at a minimum reviewing the choice of the stakeholders to interview. The Embassy will reserve the right to contact the evaluation team independently for a progress report at any point during the evaluation period.
In line with the EoS’s standard approach, this evaluation will be carried out in a spirit of partnership and participation. NN and the EoS will be given the opportunity to comment on the inception and draft reports before final reports are submitted; ensuring reports are accurate, relevant and useful for purpose. The Embassy will provide a management response for the evalutation, as per Sida’s standard evaluation protocol.
7. Evaluation quality
The evaluation evaluation shall conform to OECD/DAC’s Quality Standards for Development Evaluation[2]. The evaluators shall use the Sida OECD/DAC Glossary of Key Terms in Evaluation[3]. The evaluators shall specify how quality assurance will be handled by them during the evaluation process.
8. Time schedule and deliverables
It is expected that a time and work plan is presented in the tender and further detailed in the inception report. The evaluation shall be carried out between 1st October and 1st December 2024. The timing of any field visits, surveys and interviews need to be settled by the evaluator in dialogue with the main stakeholders during the inception phase.
The table below lists key deliverables for the evaluation process..
Deliverables
Participants
Deadlines
- Start-up meeting in Monrovia/remote
Embassy of Sweden, NN and Evaluator(s)
November 1st, 2024
- Draft inception report
Evaluator
Tentatively Nov 15th, 2024
- Comments from intended users to evaluators
Embassy of Sweden and New Narratives
Tentatively Nov 21st, 2024
- Final inception report
Evaluator
Tentatively Nov 31 2024
- Field collection of Primary Data
Evaluator
December 15th 2024
- Debriefing and initial feedback meeting
Embassy of Sweden, NN and Evaluator
January 15 2025
- Draft evaluation report
Evaluator
January 21 2025
- Comments from intended users to evaluators
Embassy of Sweden, NN
January 30th 2025
- Final evaluation report
Evaluator
February 15th 2025
The inception report will form the basis for the evaluation process and shall be approved by New Narratives and the EoS before starting to implement the evaluation. The inception report should be written in English. The report should cover evaluability issues and interpretations of evaluation questions, present the evaluation approach/methodology, methods for data collection and analysis as well as the full evaluation design. A clear distinction between the evaluation approach/methodology and methods for data collection shall be made. A specific time and work plan, including number of hours/working days for each team member, for the remainder of the evaluation should be presented. The time plan shall allow space for reflection and learning between the intended users of the evaluation.
The final report shall not exceed 35 pages and be written in English. The executive summary shall not exceed 5 pages and be written in English. The report, taking into account feedback from NN and the EoS, will include:
- An executive summary including the main conclusions and recommendations resulting from the evaluation (listed by order of priority) .
- A main report describing the context, objectives and methodology of the evaluation, the limitations, the detailed findings and results of the evaluation in relation to the objectives and evaluation questions, and the evaluator’s conclusions and recommendations (listed by order of priority).
- Appendix (list of documents, list of people interviewed, etc.)
The restitution in English should be based on a PowerPoint presentation to present the key findings and recommendations to New Narratives and the EoS. The restitution should last 1 hour, with a 30-minute presentation to allow time for questions and answers.
9. Evaluation team qualifications
Preference will be given to consultants with relevant expertise and experience who propose a participatory evaluation methodology.
The following skills will be sought:
- Specific expertise and experience related to the project :
- Expertise in journalism and/or media ecosystem development
- Knowledge of the West African and/or Liberian context
- Skills and significant experience in project evaluation (methodology, interviewing, analysis, report writing, etc.)
- Experience of evaluation of European-funded projects ;
- Fluent spoken and written English
It is important that the skills of the individual evaluation team members are complementary. It is highly recommended that a local evaluation consultant is included in the team, as they often have contextual knowledge that is of great value to the evaluation.
The evaluators must be independent from the evaluated project and activities, and have no stake in the outcome of the evaluation.
10. Resources
The maximum budget amount available for the evaluation is $US34,000. This includes all fees and reimbursables. The Consultant should submit a detailed budget showing the appropriate costs.
The contact person at New Narratives is Executive Director, Prue Clarke.
The evaluator will be required to arrange the logistics such as:
– Preparation on interview guides and other relevant tools
– Arranging for interviews in consultation with New Narratives and the EoS
– Plan field visits in consultation with New Narratives and the EoS
[1] While America Looks Away, Autocrats Crack Down on Digital News Sites, Smith, B. New York Times, July 2020
[2] DAC Quality Standards for development Evaluation, OECD, 2010.
[3] Glossary of Key Terms in Evaluation and Results Based Management, Sida in cooperation with OECD/DAC, 2014.
How to apply
4/ Submission of the offer and selection
Consultants interested in the evaluation assignment should include the following documents in their application:
- A technical proposal detailing the understanding of the evaluation stakes (with a critical analysis of the terms of reference), the proposed evaluation methodology, as well as the implementation schedule considered.
- A CV describing education and experience.
- A list describing previous evaluations or consultancies. Please give details of similar evaluation/consultancy contracts: donor and organisation that implemented the project, budget and duration of the project concerned, budget and duration of the evaluation/consultancy, main results, etc.
- A detailed financial proposal (estimate) with the total budget all taxes included.
Proposals must be submitted in English. Incomplete applications will not be considered.
Full applications should be sent by email to the following addresses by midnight 27 September 2024 (US east coast time) at the latest:
- prue@newnarratives.org
Proposals will be evaluated according to the following criteria:
- Technical and financial proposal
- Understanding of the terms of reference
- Evaluation questions
- Proposed methodology
- Consultants’ expertise and experience
- Lead Consultant
- Associate consultant(s)
- Complementarity of the team as a whole
- Budget and timetable
- Proposed budget
- Proposed duration, timeline and work plan
- Added value: specific tools, expertise, etc.
The interviews with pre-selected applicants shall be organised from 30th of September.