FINAL PROJECT EVALUATION – Action for Mental Health Assistance in Libya (AMAL) at Handicap International – Humanity & Inclusion


Un(e) Area Manager Bambari – RCA at Handicap International - Humanity & Inclusion

1. General information

1.1 . About Handicap International – Humanity & Inclusion

Handicap International (HI) is an independent and impartial aid and development organization working in situations of poverty and exclusion, conflict and disaster. We work alongside people with disabilities and vulnerable people to help meet their essential needs, improve their living conditions and promote respect for their dignity and fundamental rights. Since the organization was first founded in 1982, HI has set up emergency and development programs in more than 60 countries. Today, HI has a budget of around 230 million euros, with about 4,700 employees worldwide.

Our vision: Outraged at the injustice faced by people with disabilities and vulnerable populations, we aspire to a world of solidarity and inclusion, enriched by our differences, where everyone can live in dignity.

Our mission: HI is an independent and impartial aid organization working in situations of poverty and exclusion, conflict and disaster. We work alongside people with disabilities and vulnerable populations, taking action and bearing witness in order to respond to their essential needs, improve their living conditions and promote respect for their dignity and fundamental rights.

On its 35th anniversary, the Handicap International network changed its name to Humanity & Inclusion.

HI is engaged in an employment policy in favor of workers living with a disability.

1.2 About Handicap International in the country/region

HI is currently implementing several programs in Tripoli and Benghazi, supporting vulnerable persons and their communities. Activities include provision of psychosocial support, capacity building of local actors, technical support and donations of materials and equipment to health and mental health partners, as well as explosive ordnance risk education (EO RE) and inclusive humanitarian action awareness-raising and training provision to other humanitarian actors.

Under the funding of European Union (EU DG NEAR), since end of 2018, HI has been implementing three years project title ‘Action for Mental Health Assistance in Libya (AMAL)” with an objective to improve the access and quality of health care services in Libya. Implemented in partnership with the Libyan Ministry of Health and a Tunisian NGO, and two other local implementing partners based in Libya, the overall objective of the project is to allow the most vulnerable people in Libya from host communities, those internally displaced and returnees and migrants, suffering from mental health disorders/psychosocial disabilities to regain or to preserve good mental health by accessing quality interventions at community level, primary health care level, and secondary and tertiary level.

From July 2020, HI also started a one-year project in partnership with UNDP, which aims to enhance the development of a culture of peace, cooperation and responsibility – with a focus on youth (male and female) – through improved provision of MHPSS services at community level. This project strongly relied on the Libyan Civil Society Organizations.

2. Context of the evaluation

2.1 Presentation of the project to be evaluated

Project titleAction for Mental Health Assistance in Libya (AMAL)

**

Implementation date: December 2018 – June 2022

**

Location/Areas of intervention :Tripoli, Benghazi, and Misrata**

Operating Partners

Project was implemented through an implementing partner based in Tunis (NEBRAS) and is currently implemented by two implementing partners based in Libya (Women and Youth Empowerment Forum and Bila Houdoud) and HI directly.

Target Groups

The final beneficiaries of this action are children, adolescents and adults with mental health disorders/psychosocial disabilities living in the three major cities of Libya – Tripoli, Benghazi and Misrata – and their caregivers, including all status of persons: host community members, IDPs, returnees, refugees and migrants. Due to social stigma, difficulty of access and lack of MHPSS services in Libya at all levels of service delivery (community, primary, secondary and tertiary level) the needs of these people are largely uncovered. Target groups of the action;

• People with mental health disorders/psychosocial disabilities (final beneficiaries) through the implementation of actions at the community level: awareness campaigns and sessions on the most common mental health disorders and access to mental health care to tackle the issue of stigma; direct provision of services at community level such as home visits, individual and group sessions on psychoeducation, low-intensity psychological interventions (such as Problem Management + (PM+); developed by WHO), parenting skills trainings, peer to peer groups. Based on HI current projects, special attention will be paid to provide access to MHPSS to people with functional limitations and to integrate physical and psychosocial rehabilitation. Victims of explosive hazards are also a target group for HI in Libya, linking its current Mine Action activities in the provision of victim assistance services.

• Health professionals (junior and senior, non-specialized and specialized), through a capacity building component: psychosocial workers, nurses, family doctors/GPs, psychologists and psychiatrists. All these professionals, either newly graduated or already active within public health facilities (Primary Health Care Units, Primary Health Care Centres, MHPSS departments of General Hospitals, MHPSS Specialized Hospitals), benefit from training provided by NEBRAS and HI on clinical and community intervention in mental health.

• The Ministry of Health (the Directorate of Primary Health Care Services, local departments of the Ministry of Health in Greater Benghazi and Tripoli, selected services (PHC Units, PHC Centers), MH departments of General Hospitals, MH Specialized Hospitals). Selected public health facilities benefit from increased technical capacities of their staff (see target group above), but also from better working procedures, referral pathways, and guidelines regarding care of people with mental health disorders/psychosocial disabilities.

• 2 pre-identified CSOs technical support to implement the MHPSS activities.

Project Budget

Main objectives and expected results of the project

Program Goal: The ultime goals of the inclusive MHPSS intervention such as the one implemented are:

  • To ensure the emergence of a community-based development process, i.e. the development of local networks, structures for promoting health, prevention actions, local government policy;
  • To build organizations’ capacities (of professionals and of people with disabilities);
  • To implement and/or strengthen social inclusion and equal opportunities policy.

The overall objective of the project is therefore to allow the most vulnerable people in Libya (adults, adolescents and children) from host communities, those internally displaced, returnees and migrants, suffering from mental health disorders/psychosocial disabilities to regain or to preserve good mental health by accessing quality interventions at community level, primary health care level, and secondary and tertiary level. The project’s objective is to be attained through three major outcomes focusing on: de-stigmatisation of mental health disorders, provision of community based psychosocial interventions, access to mental health care services, capacity building of health professionals (specialized, and non-specialized) and building the capacities of two local Libyan NGOs for MHPSS service delivery.

With the following results to achieve the above:

Output 1.1: Access to MH services for people suffering from mental health disorders/psychosocial disabilities is improved

Output 1.2: People suffering from mental health disorders/psychosocial disabilities preserve their mental health and increase their coping mechanisms thanks to interventions involving MHPSS professionals.

Output 1.3: Awareness and prevention regarding mental health disorders in children, adolescents and adults are increased and stigma is reduced in order to better prevent mental health disorders/psychosocial disabilities, detect people suffering from mental health disorders/psychosocial disabilities, and facilitate their access to care and their inclusion and participation in society

Output 2.1: Libyan health professionals are trained and upskilled to improve the delivery of MHPSS services

Output 2.2: The quality of MHPSS services delivered at community, primary, secondary care level in selected project locations and facilities is improved

Output 3.1: 2 CSOs are supported by HI to improve their technical capacity to respond to mental health

Main activities implemented

Activity 1.1.1. Assessment and mapping of existing services at community level (carried by HI)

Activity 1.1.2. Identification of population groups suffering from mental health disorders (carried by HI)

Activity 1.1.3 Facilitated the establishment and implementation of referral mechanisms between targeted health facilities for more efficient services in Libya (carried by HI and NEBRAS for Tunisia)

Activity 1.2.1. Peer to Peer group activities for people with mental health disorders/psychosocial disabilities to develop their coping mechanisms (carried by HI)

Activity 1.2.2. Provision of individual support to people with mental health disorders/psychosocial disabilities and/or their caregivers to develop their coping mechanisms (carried by HI)

Activity 1.3.1. KAP survey on perception of mental health disorders and access to mental health services (carried by HI)

Activity 1.3.2. Awareness sessions in the community on the most common mental health disorders, how to cope, and where to refer (carried by HI)

Activity 1.3.3. Media awareness campaign on the most common mental health disorders amongst children, adolescents and adults (carried by HI)

Activity 2.1.1 Delivery of training for senior family doctors/GPs from the Board of Specialties and senior nurse (9 family doctors/GPs from the Board of Specialties and 9 senior nurses) on delivering training and coaching sessions (ToT) on mental health intervention to nurses and family doctors/GPs inside Libya (carried by NEBRAS with support from HI)

Activity 2.1.2 Delivery of training and internship in Tunisia for 45 family doctors/GPs on mental health intervention to be able to identify and provide basic assistance to common mental health disorders and follow up stabilized severe cases referred from the Specialized Mental Health Hospital (carried by NEBRAS with support from HI)

Activity 2.1.3 Delivery of training for 30 nurses by Libyan trainers on mental health intervention and drug management who could identify symptoms and refer to focal point mental health family doctors/GPs in the primary health care (carried by NEBRAS with support from HI)

Activity 2.1.4 Delivery of training in Tunisia for 10 junior psychiatrists on specialized mental health topics (trauma, addiction, child psychiatry) (carried by NEBRAS with support from HI)

Activity 2.1.5 Delivery of training for 30 junior psychologists on mental health topics (academic curriculum) and internship in Tunisia (carried by NEBRAS with support from HI)

Activity 2.1.6 Delivery of training for 50 psychiatrists and clinical psychologists on psychotherapeutic interventions (EMDR and NET) an/or emergency intervention (EMDR Group protocol) (carried by NEBRAS with support from HI)

Activity 2.1.7 Delivery of training for 36 mental health professionals (among previous trainees) on ToT and supervision of provision of psychosocial support (carried by HI)

Activity 2.1.8. Delivery of training for 20 health professionals on psychosocial support to people with mental health disorders (carried by HI)

Activity 2.1.9 Delivery of clinical supervision sessions to mental health professionals (carried by NEBRAS with support from HI)

Activity 2.2.1. Assessment of the satisfaction of the people with mental health disorders/psychosocial disabilities on services provided into selected health care facilities (carried by HI)

Activity 2.2.2. Monitoring of continuity of care provided to people with mental health disorders/psychosocial disabilities in selected health care facilities (carried by HI)

Activity 3.1.1. Provide technical guidance and monitoring to CSOs sub-granted

Activity 3.1.2. Technical training and support to 12 staff of 2 CSOs on awareness sessions on psycho-social distress and coping mechanisms and on preparedness to face events causing psychosocial distress through Psychological First Aid Intervention

Activity 3.1.3. Delivery and supervision of MHPSS interventions conducted by the CSOs

Activity 3.1.4. Provision of Individual Support to people with mental health disorder/psychosocial disabilities and/or their caregivers to develop their coping mechanisms (carried by CSOs)

2.2 Justification of the evaluation

The main purpose of this end-of-project evaluation is to assess the quality of design, inception and implementation of the project with achievement of its objectives through related activities. This exercise will also inform, assess and advise on the implementation modalities (partnerships and collaborations with other health structures or humanitarian organizations). This study will also shed light on the impact of interventions, through partners and to what extent their capacities have been built during this project. It shall also highlight the accountability of all project stakeholders, and participation of local communities. Furthermore, this exercise will provide a set of best practices, lessons learned and recommendations for future interventions for continuous improvement of HI programming as per HI Project Quality Framework.

3. Objectives of the evaluation

3.1 Overall objectives and expectations of the evaluation

The objective of the evaluation is to provide HI with an overall independent, qualitative assessment of the AMAL-funded project achievements compared with its objectives at the end of the intervention in Libya. The second objective is the identification, formulation and presentation of the lessons learned that could be of interest and use to HI and other stakeholders to make the positive impacts of the project more sustainable.

3.2 Specific objectives

The specific objectives are as follow:

· To provide a comprehensive understanding of project outcomes and effects of the action on the direct beneficiaries, indirect beneficiaries, 2 CSO partners and other organizations HI collaborates namely PHCI, Nebras and other collaborators within the target area within and beyond the frame of planned indicators

· Assess the activities and project implementation through the scope of the selected criteria of HI Project Quality Framework

· To formulate the recommendations for all project stakeholders that will shed light on how to ensure sustainability of future actions and their positive impacts

3.3 Evaluation criteria and evaluative questions

By analyzing the different evaluation questions, the evaluation will go through 4 out of the 12 HI Project Quality Framework criteria, namely “Relevance”, “Capacities”, “Effectiveness” and “Synergy”. These criteria aim at covering all the main areas of quality defined by HI’s Quality Framework, i.e. “Benefits”, “Management”, and “Stakeholders”. Based on these criteria, the evaluation questions are suggested as follows and will be finalized following the selection of the consultant:

Criteria and definition

Evaluative questions

RELEVANCE – The project responds to demonstrated priorities and adapts to the context of intervention **

Needs

The project meets the demands and needs of beneficiaries and contributes to achieving priorities of other stakeholders (authorities, partners, donors …).

To what extent did the project meet the needs of the direct beneficiaries?

CAPACITIES – The project helps strengthen internal and external capacities

Autonomy

The project helps to build the capacities of local stakeholders and to ensure their autonomy at the end of the project

How has the project increased the capacity of local stakeholders so they become more autonomous?

Competencies

The project team and partners develop the skills necessary for implementing the project

Have the necessary skills been developed among HI and partners teams to carry out the project and contribute to expected results?

EFFECTIVNESS – The project successfully achieves its objectives Feasibility

**

The project has the necessary resources (human, financial, logistical, technical…) to achieve its objectives

To what extent have the resources (human, logistical, financial, technical) available enabled the project objectives to be achieved?

Product/Service

**

The project’s outputs are of the required quality, in compliance with HI and/or international technical standards

Is the technical quality of the project achievements in line with HI’s technical standards?

SYNERGY – The project is consistent with its environment and interacts positively with other stakeholders.

Complementarity

The project is in coherence with other interventions to ensure a comprehensive response to the multiple and changing needs of the target groups.

Has the project been properly integrated with other interventions in the intervention area?

In addition to these criteria, the evaluation is expected to pay particular attention to lessons learned and recommendations to continue some of the activities beyond the implementation period.

Other evaluation criteria such as coherence, coverage of target groups, gender and alignment could be addressed as per relevance and feasibility.**

It is finally to be noted that these criteria and evaluative question have been defined at project proposal designing stage. Should the evaluative needs of HI change by the end of the project, as will be determined by the Evaluation Steering Committee composed of selected HI team members and depending of feasibility/relevance of associated collection and analysis methods (taking into account, in particular, of the context, security and sanitary situation), the above list could be subject to some changes. **

4. Evaluation methodology
**

The evaluator will propose an approach based on participatory evaluation techniques and tools. A strong participatory dimension is expected to guarantee the ownership of the results by the different project stakeholders.

The evaluation will be conducted by an external evaluator with the support of the Project Manager and Operations Manager to facilitate the evaluation exercise to be conducted, as well as the MHPSS Technical Specialist based on a set of defined criteria in order to guarantee that the evaluative exercise can be conducted in a qualitative and independent manner.

4.1 Collection methodology

The evaluator should use a combination of methodologies relying on internal and external sources, based on semi-qualitative data collection. **

Internal sources:

  • Project documents and tools review ;
  • Review of reports produced within the project ;
  • Key Informant interviews at head office and program level (face-to-face or remotely);
  • Review of HI internal documents related to the project (regular reports, project monitoring documents, training and workshop reports, beneficiary data, indicators follow up tool, etc.)
  • In depth interviews with HI project staff

External sources:

  • Interviews with beneficiaries (direct and indirect) and other project’s contributors in the target areas ;
  • In depth interviews with the staff of the partners ;
  • Key Informant interviews with other stakeholders of the project (e.g. key community members, local authorities’ representatives, humanitarian workers, MoH/PHCI, Civil Society Organizations – CSOs’ staff, etc.);
  • Review of the literature (Inter-Agency documents produced, needs assessments, government reports, studies etc.).

Baseline information

A complete list with the names and personal details of the persons concerned will be given to evaluator(s) at field level. The arrangement of meetings with the persons concerned will be facilitated by HI Libya teams according to need.

Other documents and tools to be made available either before or during the evaluation timeframe while on the ground would be:

  • Project Proposals and modification requests with relevant annexes ;
  • All training, meeting and workshop reports;
  • Pre and post tests, evaluation reports;
  • Beneficiary patient registers, when accessible ;
  • Monthly, quarterly and annual reports donors.

Other documents may be provided at the request of the evaluator.

To the highest extent possible, preference will be given throughout this final evaluation exercise to face-to-face exchanges, physical presence of the evaluator in key project locations (e.g. intervention areas targeted by the project, HI field and HQ offices, etc.) and direct observation. However, depending on access to the mission sites and entry into Libya the evaluation exercise may have to be carried out partly or entirely in a remote fashion.

In this regard, and based on the evolution of the situation at the time of the realization of the exercise, the evaluator should propose an adapted evaluation methodology (including collection methodology – e.g. recruitment on field data collectors, face to face and phone interviews, etc.) depending on the possibility to travel and access key project’s sites.

5. Principles and values

5.1. Protection and Anti-Corruption Policy

  • Code of Conduct
  • Protection of beneficiaries from sexual exploitation, abuse and harasment
  • Child protection Policy
  • Anti-fraud and Anti-corruption policy

5.2. Ethical measures*

As part of each evaluation, HI is committed to upholding certain ethical measures. It is imperative that these measures are taken into account in the technical offer:

Guarantee the safety of participants, partners and teams: the technical offer must specify the risk mitigation measures.

Ensuring a person/community-centered approach: the technical offer must propose methods adapted to the needs of the target population (e.g. tools adapted for illiterate audiences / sign language / child-friendly materials, etc.).

Obtain the free and informed consent of the participants: the technical proposal must explain how the evaluator will obtain the free and informed consent and/or assent of the participants.

Ensure the security of personal and sensitive data throughout the activity: the technical offer must propose measures for the protection of personal data. **

5.3. Participation of stakeholders and beneficiaries

As part of each evaluation, HI is committed to ensure the participation of all relevant stakeholders ( institutional and implementing partners, national/local authorities, HI team members, external consultants/evaluators of the project) and its beneficiaries in the evaluation process of the project.

6. Expected deliverables and proposed schedule

6.1. Deliverables

  • An inception report refining / specifying the proposed methodology for answering the evaluation questions and an action plan. This inception report will have to be validated by the Steering Committee composed of selected HI team members.
  • A presentation document presenting the first results, conclusions and recommendations, to be presented to the Steering Committee composed of selected HI team members.
  • A final report of approximately 20-30 pages maximum by June 15, 2022.
  • A summary of 5 pages ;

The final report should be integrated into the following template:

The quality of the final report will be reviewed by the Steering Committee of the evaluation using this checklist:

6.2. Evaluation schedule

It is expected that the overall duration of the evaluation exercise will be for 30 days up to the validation of the final evaluation report. Depending on access restriction due to the security and sanitary situation and on subsequent adaptations of the methodologies, this expected duration might have to be revised.

Key steps of the evaluation schedule will include:

  1. Initial briefings with HI management at Libya coordination level (and in the field, if access is granted);
  2. Development of data collection tools in close collaboration with the Libya Operations Team ;
  3. Agreement on schedule with specific dates with the Libya Operations Team
  4. Submission and approval of an inception report including data collection tools ;
  5. Data Collection;
  6. Submission of preliminary findings with the Libya Operations Teams and the Steering Committee composed of selected HI team members
  7. Submission of draft report ;
  8. Incorporating stakeholders’ feedback in draft report.
  9. Final draft submission and validation by the Operations Teams.
  10. A summary report to be submitted to the Steering Committee composed of selected HI team members
  11. The preliminary findings will be shared with all relevant stakeholders, in lessons learned workshop organized by HI

7. Means

7.1. Expertise sought from the evaluator

The selection of the evaluator will be based on the consideration of a set of criteria, which will be determined by the Steering Committee composed of selected HI team members, which could include:

§ A solid and diversified experience in humanitarian programming; (15%)

§ Previous experience in end of project evaluation for INGOs (previous experience of evaluating EU-funded projects would be an asset); (15%)

§ A Masters Degree required and technical background in MHPSS is considered a strong asset; 15%

§ Full working knowledge of English and Arabic and excellent report writing; (10%)

§ Fully conversant with the principles and working methods of project cycle management. (15%)

§ Available for work from mid-May to mid-June, 2022 (30 days from the signing of contract)

§ Financial Proposal (30%)

The complete application files must be sent by e-mail to appel-offres@tunisie.hi.org before 10 May 2022, with the mention in the subject line: “AMAL FINAL PROJECT EVALUATION “.

How to apply

The complete application files must be sent by e-mail to appel-offres@tunisie.hi.org before 10 May 2022, with the mention in the subject line: “AMAL FINAL PROJECT EVALUATION “

More Information

  • Job City Libya, Tunisia
ArabicChinese (Simplified)DutchEnglishFrenchGermanItalianPortugueseRussianSpanish