Caritas Internationalis Emergency Appeal Final Evaluation

Introduction

Trócaire is an Irish development agency that was established in 1973 by the Catholic Bishops of Ireland. Trócaire’s work is inspired by Catholic Social teachings, with an overarching commitment to social justice. Trócaire works in 17 countries in Africa, Asia, Latin America, and the Middle East, working through local partner organizations and communities to deliver programmes relating to: Accountable Governance and Human Rights; Natural Resource Use and Resource Rights; Women’s Empowerment and Humanitarian Preparedness and Response

Trócaire has been working in Somalia for 30 years and established offices in the districts in Gedo in 1991. In Somalia, we implement projects directly and collaborate with local communities to deliver Humanitarian programs in the following sectors.

• Health
• Nutrition
• Water, Sanitation and Hygiene Promotion
• Education in Emergencies
• Protection
• Food security and Livelihood

Caritas Somalia is a national catholic organization with a mission to accompany, serve and defend the poor that was established in 1980 as a response to the amount of refugees in need from the “Ogaden war” in Somalia. From then, it has been working all over Somalia directly and indirectly, mainly to respond to catastrophes such as drought, floods, famine and armed conflict.

Caritas Somalia was recognized and affiliated to the Caritas Internationalis in 1983. It was recognized by the Somali government as a Somali benevolent association on 17th August 1988.

Its current global strategic Framework from 2021-2025 has six priorities, namely:

• Migrants and IDPs empowerment
• Development Projects
• Emergency Response
• Education and Health
• Women Empowerment
• Advocacy &Peace Building

Caritas Somalia has a lean organizational structure. The office is located in Djibouti, where the president and executive director are also located.

As far as the activities are concerned, it relies on local partners, but always retains the financial management of the projects.

Caritas Somalia has signed the Code of ethics for Caritas Internationalis member organizations in 2014. In December 2020 Caritas has also signed and integrated in their projects, the Children and Vulnerable Adults Safeguarding Policy and the Anti-Harassment Policy.

Context

Somalia is currently experiencing one of the worst droughts in recent history as a result of poor performing rainfall seasons. Whereas sustained humanitarian assistance is viewed as having averted famine in Somalia during the first quarter of 2023[1], levels of food insecurity remain high across the country, with over 6.6 million people expected to face Crisis (IPC Phase 3) or worse acute food insecurity outcomes, and a projected acute malnutrition burden of 1.8 million children (including 477,700 who are projected to be severely malnourished) from January to December 2023[2].

An IPC multi-partner technical released last year indicated that the nutrition situation has deteriorated across most of the country. Acute malnutrition case admissions among children under age five continued to rise sharply. Based on the results of 29 integrated food security, nutrition and mortality surveys conducted by the Food Security and Nutrition Analysis Unit (FSNAU) and partners and subsequent IPC acute malnutrition analysis conducted, the total estimated acute malnutrition burden for Somalia from August 2022 to July 2023 was found to be approximately 1.8 million children.

The Somali health system is fragmented and under-resourced. An estimated 19% of health facilities are fully functional, limiting access by a large percentage of the population, especially children and PLW’s and those in famine-risk districts (HNO 2023). UNICEF reports that some IDP settlements have no water or drugs and inadequate staffing to meet the overwhelming need for care. Vulnerable populations are most at risk due to inadequate essential primary health care, MNCH care and services for people with disabilities. Cholera and AWD are endemic in Somalia , and has been experiencing uninterrupted transmission of Cholera since 2017. The current outbreak is concentrated among drought-affected, mainly displaced people who have no access to safe water or sanitation and poor hygiene habits. The prolonged drought has caused surface water sources to dry up. Furthermore, aquifer recharge has seriously been affected, leading to dropdowns of ground water levels in shallow wells and boreholes; some of these wells and boreholes have also dried up. The combination of limited water availability and rising fuel prices to run water pumps has caused price hikes for water, leaving communities with few options other than unimproved water sources. Open defecation remains a common practice.

The drought in Somalia has worsened the protection situation and amplified the needs, especially for women and children Harmful practices (including Female Genital Mutilation (FGM) and CEFM (Child Early Forced Marriage) persist in Somalia. Crisis disproportionately affects women and girls, exposing them to GBV ranging from sexual violence and exploitation in the form of communal violence, intimate partner violence (IPV), emotional and psychological violence, early and forced marriages, to female genital mutilation. The risk of GBV is heightened with a protection cluster survey indicating that women and girls felt less safe since the onset of the drought with 80% of the respondents noting an increase in reports of violence against girls and women since the crisis/emergency occurred, and majority reporting an increase of IPV affecting women and girls.

Trocaire and Caritas Somalia sought an emergency appeal from Caritas Internationalis to respond to the current crisis in Somalia.

For this response, Trocaire prioritized the districts of Belet Hawa, Dollow, Garbaharey, Luuq, and Burdhubo while Caritas Somalia implemented in Beletweyne, Bulo-Barte, and Huduur through Volunteers for Agricultural Development – VAD (a local humanitarian and development organization established in 2019).

Programme Interventions

The project’s specific and intended results of the project included:

Impact: To address priority and immediate needs of drought-affected and vulnerable communities through continued provision and extension of quality integrated health, nutrition, and WASH services

Outcome 1: Contribute to reduction of morbidity, mortality, and malnutrition rates of vulnerable and marginalized communities in Luuq, Dollow, Belet Hawa, Garbaharey and Burdhubo Districts in Gedo

Output 1.1: Health and nutrition services provided in all districts

Output 1.2: Integrated Community Case Management (iCCM) services provided at community level

Output 1.3: Disease & SAM surveillance conducted with severity and trends tracked

Output 1.4: Emergency medical and non-medical supplies prepositioned in all districts

Outcome 2: Targeted communities provided with improved water, sanitation & hygiene services

Output 2.1: Fuel subsidies provided for IDP and riverine communities

Output 2.2: Water trucking for targeted communities done

Output 2.3: Shallow wells rehabilitated

Output 2.4: Hygiene promotion conducted through CHWs and hygiene promoters

Output 2.5: Hygiene kits provided for targeted communities

Output 2.6: Gender-sensitive latrines constructed in IDP camps

Output 2.7: Garbage collection and clean -up campaigns conducted

Outcome 3: Communities in targeted locations have access to safe, timely and lifesaving GBV services

Output 3.1: CMR and case management services provided in all districts

Output 3.2: Conduct Clinical Management of Rape (CMR) training for health staff and case workers conducted

Output 3.3: Dignity kits for vulnerable women and girls provided

Output 3.4: Awareness on service availability, prevention, and mitigation of SGBV/FGM raised

Output 3.5: Solar and overhead lamps installed within IDP camps to enhance safety of women, girls and other community members

Key Stakeholders

The evaluation will target key stakeholders of the project as respondents to the process. These will include community members, government officials, religious leaders, clan elders, Trocaire staff and sector actors e.g. from the clusters.

Rationale of the evaluation

As the project comes to a close, a final evaluation is being commissioned to assess the project progress toward achieving its objectives.

The evaluation will be required to track and ascertain progress against the outcomes of the project towards achievement of the overall goal of the project, identify lessons learnt, good practices and provide concrete recommendations for future similar interventions, in order to enhance adaptive programming.

OBJECTIVES

The objective of the evaluation will be to:

1. Assess the relevance, appropriateness, effectiveness, accountability and impact/sustainability of the programme
2. Assess the extent to which the project has achieved its purpose based on intended outputs and outcomes
3. Identify lessons learned, best practices and recommendations to inform future programme design

KEY QUESTIONS

Relevance/appropriateness

• Was programme design based on an impartial assessment of needs? Are needs assessments disaggregated by age, sex and disability? Do they include people’s needs, vulnerabilities and capacities?
• Did the assistance provided meet the needs of the affected population? Were the persons most in need identified, selected, and supported by the programme?
• Which parts of the assistance were the most appropriate and why? Which were least appropriate and why? Were activities aligned with the affected population’s needs and priorities?
• Were recommendations and learning from past reviews and evaluations applied to the response?

Effectiveness

• Was the response timely?
• What internal and external factors affected the speed of the response?
• Was the internal organizational and managerial structure of the project effective?
• Were there appropriate systems in place to monitor activities, outputs and outcomes of the programme? Did monitoring outcomes inform programme adjustments/revisions?
• Did the project activities lead towards the achievement of the expected results/indicators as set in the Results Framework?

Accountability

• To what extent has the affected population been involved in the design or implementation of the programme?
• Were appropriate systems of downwards accountability (participation, information sharing and feedback/complaints), put in place and used by project participants? Were project participants aware of the feedback/complaints mechanism?
• Were project participants and communities aware of selection criteria?
• Were project participants and communities aware of the assistance they should receive?

Coordination

• How effective were Trocaire and Caritas Somalia in coordinating internally?
• How effective was Trocaire in coordinating with Caritas Somalia?
• How effective were Trocaire and Caritas Somalia in coordinating with external stakeholders such as other agencies, organisations, the local and national government?
• What aspects of coordination could be improved in the future and how?

Impact/Sustainability

• Has the response strengthened local capacities?
• What are the intended and unintended, positive and negative effects of the project?
• What, if any, aspects of the programme will have a longer-term impact?

METHODOLOGY

The final evaluation will rely on two main evaluation stages (1) design phase (2) field phase

Design Phase

• The evaluator(s) will undertake a desk review of programme documentation including planning documents, project proposals, situation reports and quarterly reports
• The evaluator(s) will review other monitoring and reporting documents from secondary sources e.g. FSNAU

Field Phase

• After the design phase, the evaluator(s) will conduct fieldwork, by visit project sites in-person, to collect and analyse data in order to answer the evaluation questions
• Data collection methods should be inclusive and utilise a range of methods, including focus group discussions and key informant interviews and with key project stakeholders. The use of surveys and other remote data collection tools should also be explored by the evaluators to maximise data collection
• The evaluator(s) should ensure a systematic triangulation of data sources and data collection methods and tools, and seek to validate data through regular exchanges with programme staff where appropriate

EVALUATION SCOPE

The consultant will be responsible for:

• Development of a brief inception report including a comprehensive evaluation protocol.
• Desk review of existing program documents (including but not limited to proposal, monthly activity progress reports, needs assessments, secondary sources e.g. FSNAU).
• Development of evaluation tools and upload to CommCare
• Data collection, analysis and management.
• Development of draft and final report.
• General oversight of the assessment process.
• Presentation on the evaluation including feedback of findings, to Trocaire management and programme team.
• Overall quality management.

EXPECTED OUTPUTS

The evaluator(s) should produce the following key deliverables:

1. Inception meeting with project staff, review of key relevant documents
2. Develop and submit inception report/ protocol which will include an interpretation of the tasks and study design & methodologies, sampling procedures and detailed work plan
3. Develop appropriate tools and instruments for gathering information and present to Trocaire team for approval
4. Conduct survey interviews with households in the field
5. Undertake KIIs and FGDs with key stakeholders in the field
6. Provide raw and cleaned datasets and/or transcripts
7. General oversight and coordination of the evaluation process including logistical arrangements for data collection in collaboration with the Project Coordinator
8. Draft Evaluation Report to be submitted to Trocaire and Caritas Somalia
9. Final Evaluation Report inclusive of:

• Executive Summary
• Background
• Introduction
• Context
• Description of Methodology
• Main findings
• Conclusions inclusive of best practices and lessons learned
• Recommendations.

1. PowerPoint presentation on the evaluation to Trocaire management and programme team

TRÓCAIRE RESPONSIBILITIES

• Provision of Trócaire specific organisational and programme documents – including policies, strategies, project plans, needs analyses, and evaluations. Note: this is not exhaustive; it will need to be supplemented with information from credible external sources.

USE OF THE EVALUATION RESULTS

The intended audience for the evaluation is Trocaire and Caritas Somalia key staff, including senior management, who have supported the programme, the Caritas Internationalis Humanitarian Department and the Caritas Confederation.

Evaluation findings will be shared with programme participants as appropriate.

REQUIRED COMPETENCIES

The competencies required from the External Evaluator are:

• Advanced degree in social sciences, political sciences, economics, development or related fields;
• Experience in leading evaluations, especially in the field of humanitarian response;
• Ability to use participatory approaches to evaluation;
• Experience of operational management of humanitarian/development programmes;
• Good knowledge of the local context;
• Good analytical skills;
• Excellent writing skills in English language
• Somali language skills required for fieldwork;

TIME FRAME

The evaluation will take 21 days, to be completed by end of September 2023.

BUDGET AND PAYMENT

The Consultant shall specify their proposed fee for conducting the assignment, including professional fee, transport, accommodation, data collection, statutory taxes etc.; as appropriate to be able to deliver on the assignment.

Invoicing and Payment Procedures

A first payment of 40% of the total costs will be settled as soon as Trócaire has approved the workplan and inception report. The final 60% will only be paid once fieldwork is completed, a draft report is shared and Trócaire has approved the final report, after comments are incorporated. A final invoice will be submitted upon the completion of the assignment and submission of all outputs/ deliverables.