Call for Proposal: Mid-term evaluation of MSF project at Médecins Sans Frontières


Doctors Without Borders/Médecins Sans Frontières (MSF) is an international medical humanitarian organization determined to bring quality medical care to people in crises around the world, when and where they need regardless of religion, ethnical background, or political view. Our fundamental principles are neutrality, impartiality, independence, medical ethics, bearing witness and accountability.

The Stockholm Evaluation Unit (SEU), based in Sweden, is one of three MSF units tasked to manage and guide evaluations of MSF’s operational projects. For more information see: evaluation.msf.org.

Projet/Mission: Mid-term evaluation, Bangassou, Central African Republic

Start date: November 2022

End date: Final report latest March 24th , 2023

Terms of Reference (full ToRs in PDF) : https://evaluation.msf.org/call-proposal-mid-term-evaluation-bangassou-central-african-republic

Specific considerations:

Field visits if any should be discussed in close collaboration with the project, Consultation Group for the evaluation and SEU. Due to the volatile context, there is a caping regarding the number of staff based in/visiting the project location.

These TOR are written in English, but the evaluation requires fluency in French (contact with stakeholders, documentation review, report writing)

BACKGROUND[1]

Overview of MSF presence in Bangassou

MSF OCB has been present in Central African Republic since 2013. Following an assessment in October 2013, during the conflict in the country, MSF started its support to Bangassou hospital (Hôpital Régional Universitaire de Bangassou, HRUB) in February 2014. The rationale to intervene was the dire condition of the hospital because of the conflict, hampering its ability to provide care according to needs, as well as the difficulties of the population to access and afford healthcare. MSF supported the HRUB through rehabilitation work in key services such as operation theatre, emergency room, in patient department, as well as by ensuring functioning water and electricity supply. MSF also implemented free healthcare to decrease barriers to care for the general population. The implementation of free healthcare resulted in a lack of income for the health facilities which motivated the payment of incentives to non-MSF staff, as part of the MSF support, to compensate part of the salaries.

From 2016, MSF was fully in charge of the HRUB, providing direct medical support to care provision in a wide range of services including the emergency room, maternity, pediatrics, surgery, laboratory, consultations for HIV/TB patients. A Memorandum of Understanding was signed between MSF and the Ministry of Health (MoH, in French Ministère de la Santé et de la Population, MSP), defining the modalities of the collaboration. According to the MoU, the CONGES ou Coges[2] plays a role of intermediary between the hospital, the project in collaboration between MSF and the MoH, and the community. The CONGES is a structure set up by the MoH to increase the autonomy of the health facilities as well as ensure greater involvement of communities. The CONGES can, if deemed necessary, recruit staff for these facilities and pay them via cost recovery.

In May 2017, many organizations left the area due to increased conflict and violence. MSF stayed but focused on life-saving activities only. CONGES staff already working in the hospital were recruited by MSF as MSF staff to ensure continuity of care in the event of MSF having to withdraw from Bangassou, as MSF staff were not from the area (relocated from Bangui or international staff). Indeed, security and safety of MSF teams was threatened to such an extent that MSF had to leave Bangassou from November 2017 until April 2018. However, MSF continued to provide support through the payment of incentives, as well as supplies of medicines and fuel.

The reopening in 2018 was done with an increased engagement with the community, in order to increase acceptance and diminish security risks. In the hospital, more responsibility was given to the MoH, through the CONGES, versus previous full management by MSF. This was done in order to mitigate the impact of any potential withdrawal of MSF on healthcare provision. In 2020 staff contracts were shifted from MSF to MoH/CONGES, with MSF incentives (approx. 200 staff). A new Memorandum of Understanding was signed end of 2021.

Description of the project

In 2022, the project has activities in the Mbomou prefecture (part of the sixth health region in the country), specifically in the two health districts of Bangassou and Ouango. The Bangassou health district includes the sous préfectures of Bangassou, Rafai et Bakouma, while the Ouango Health district includes the sous préfectrues of Ouango and Gambo. The health district of Bangassou includes 37 Formations sanitaires (FOSA), one regional hospital, 25 Health centers and 11 Health posts.

Support is provided to the HRUB and to health centers and health posts along three geographical axes (North, East and West of Bangassou). The project also includes emergency preparedness and response, as well as an advocacy component.

The overall objective of the project is to reduce morbidity and mortality in the Mbomou prefecture, with activities focusing on secondary healthcare at HRUB, primary healthcare in outreach, as well as surveillance and outbreak response, and advocacy[3]. A MSF “round table” meeting is planned for Q 2 2023, during which a multiyear strategy for the Bangassou project will be presented.

The incentives scheme

To implement the project medical activities with CONGES and MoH staff and compensate the health structures for the lack of income they face due to free healthcare implementation by MSF, incentives are being paid by MSF to:

  • The structures identified for the outreach activities
  • The CONGES staff (local staff managed and paid by the CONGES, according to the MoU)
  • The MoH staff (a limited number of staff paid by the Ministry)
  • Medical and paramedical students in nursing and midwifery who carry their internship at the HRUB.

The Memorandum of Understanding signed between MSF and the MoH (at national level with the Minister) in January 2021 details the modalities of the current incentives scheme.

As stated in the MoU, the MoH policy is to have incentives schemes based on performance and therefore it is such a system that has been implemented in Bangassou. MSF pays incentives, both a fixed part (60%) and a variable one (40%) linked to performance of the staff. Detailed list of positions granted for incentives as well as amounts are annexed to the MoU. The incentives are individualized according to positions and paid by MSF to the MoH and CONGES who then pay their staff. The incentives, both fixed and variable parts, form part of the staff’s salary but do not necessarily correspond to its full amount. The MoH and CONGES remain responsible for payment of salary, on top of incentives if there is a discrepancy. The variable part of the incentive is linked to individual performance appraisals conducted by MoH and CONGES with their staff, and then shared with MSF for approval and subsequent payment.

There have been some delays in implementing the system fully, especially for the variable part. It is now in place and the project conducted in September 2022 the second round of staff performance appraisals in order to prepare the payment of the incentives for Q2 2022 (the first round of appraisals was done for Q1 2022).

PURPOSE and intended use of the evaluation

The payment of incentives based on performance, implemented in Bangassou, is new to OCB and seen as a potential system to replicate in other projects. It is a major component of the collaboration with the MOH in Bangassou and CAR. An evaluation is wanted by the project and support teams to describe and assess the value of the incentives scheme based on performance. MSF OCB wants to know how the system works, what is working or not, the reasons why this specific system is in place, as well as understand which improvements could contribute to a better system and ultimately implementation of the project.

Findings from this evaluation may inform the renewal of the current MoU, which expires end of 2022. It will be renegotiated with the MoH, in line with the definition of the strategy for the next years, which will be decided during the round table meeting in OCB, Q 2 2023.

The primary users of this evaluation will be the project team, and by extension the country and regional support teams. Beyond MSF, the evaluation will be highly interesting for the MoH in CAR and specifically in Bangassou, and the evaluation findings will be shared with them (see deliverables). Beyond CAR, many OCB operations are interested in knowing more about this system and so is the Human Resources Department.

Evaluation questions

These are the main questions the evaluation should answer. It is understood that sub questions can be introduced by evaluator(s). The evaluator(s) should suggest relevant framework(s) to answer the evaluation questions. The evaluation needs to make sure to involve the MoH and CONGES and to take into consideration different perspectives during the process.

Overarching evaluation questions:

  1. Description of the incentives scheme based on performance, as implemented in Bangassou since 2019. What are the modalities of this system, how is it implemented in practice?
  2. Assessment of the merit, worth and significance of this incentives scheme: what is working or not, why? What are the objectives of this incentives scheme and to what extent are they being achieved?
  3. What are the lessons learned and which adjustments could be made to the system?

deliverables

  1. Inception Report in French

As per SEU standards, after conducting initial document review and preliminary interviews. It will present how the evaluation will be conducted.

  1. Draft Evaluation Report in French

As per SEU standards. It will answer to the evaluation questions and will include conclusions, evaluator’s judgement, lessons learned and recommendations if any.

  1. Working Session

With the attendance of commissioner and consultation group members. As part of the report writing process, the evaluator(s) will present preliminary findings, and facilitate discussion around those. Ideally, the working session will also include a co-creation of the recommendations.

  1. Final Evaluation Report in French

After addressing feedbacks received during the working session and written inputs on the draft.

  1. Short version of the report in French alt. different format to increase accessibility of evaluation findings

This format could for example be used by decision-makers or stakeholders potentially interested in overarching evaluation findings.

  1. Webinar presenting and discussing evaluation findings. Other dissemination deliverables to be suggested and defined during evaluation process. See Dissemination and intended use below for suggestions.
  2. Presentation of the evaluation findings to the MoH, exact audience to be discussed with Consultation group of the evaluation

tools and methodology

In addition to the initial evaluation proposal submitted as a part of the application (see requirement chapter), a detailed evaluation protocol should be prepared by the evaluators during the inception phase. It will include a detailed explanation of proposed methods and its justification based on validated theory/ies. It will be reviewed and validated as a part of the inception phase in coordination with the SEU.

RECOMMENDED DOCUMENTATION and sources

  • Project documents (eg narratives, logframe, monitoring reports, presentations, visit reports and end of mission reports, MoU current and previous ones, incl annexes)
  • HR documents eg organigrams, budgets, MoU, evaluation tools, strategy, policies
  • External litterature, similar experiences, other actors, humanitarian standards etc.

This list is non exhaustive and the evaluator(s) are expected to ask for documentation according to what they believe is needed, as well as to refer to non-MSF documentation as they see fit.

PRACTICAL IMPLEMENTATION OF THE EVALUATION

Number of evaluators : One or several evaluators – a team can be assembled by the SEU of individual applicants where necessary.

Timing of the evaluation: November 2022 to March/April 2023

requirements

  • Requirements:
    • Proven competencies in evaluation
    • HR background or competencies
    • Expertise in humanitarian programming
    • Experience of running projects working in close collaboration and within MoH structures.
    • Experience in evaluating or implementation Health staff incentives programs
    • Fluent in English and French
  • Assets:
    • Academic background in public health, evaluation, medical background
    • Knowledge of the CAR context
    • MSF experience and/or understanding

DISSEMINATION AND USE

Once the report is finalized, how can evaluation findings be shared and used? Some of the activities below are standard for all SEU evaluations, while others are suggested for this specific evaluation. The evaluator(s) are welcome to suggest and contribute to these activities, with the overall objective to increase accessibility and use of the evaluation, for learning and accountability.

[1] The information in this section is based on Bangassou internal project document, 2022 as well as Memorandum of Understanding (Protocole d’accord) signed in January 2021 and valid until 31 December 2022.

[2] COGES stands for Comité de Gestion and is the terminology used for smaller health facilities (health posts and health centers) while CONGES stands for Conseil de Gestion and concerns larger facilities such as hospitals. For facility of reading, CONGES will be used throughout this ToR.

[3] According to Bangassou project document, 2022

How to apply

The application should consist of a technical proposal, a budget proposal (separate), CV, and a previous work sample. The proposal should suggest relevant evaluation theory and methodology to answer the evaluation questions. The SEU encourages applicants to challenge those Terms of Reference if they see fit. The proposal should include a reflection on how adherence to ethical standards for evaluations will be considered throughout the evaluation. Offers should include a separate quotation for the complete services, stated in Euros (EUR). The budget should present consultancy fee according to the number of expected working days over the entire period, both in totality and as a daily fee. Travel costs, if any, do not need to be included as the SEU will arrange and cover these. Do note that MSF does not pay any per diem.

Applications will be evaluated on the basis of whether the submitted proposal captures an understanding of the main deliverables as per this ToR, a methodology relevant to achieving the results foreseen, and the overall capacity of the evaluator(s) to carry out the work (i.e. inclusion of proposed evaluators’ CVs, reference to previous work, certification et cetera).

Interested teams or individuals should apply to evaluations@stockholm.msf.org referencing “BANGA” no later than 23:59 CET on Monday 7th November 2022. We would appreciate the necessary documents being submitted as separate attachments (proposal, budget, CV, work sample and such). Please include your contact details in your CV.

Please indicate in your email application on which platform you saw this vacancy.

 

Related NGO Jobs in Africa

ArabicChinese (Simplified)DutchEnglishFrenchGermanItalianPortugueseRussianSpanish