Terms of Reference Mobile Leprosy (MLep) End Evaluation External Evaluation - mPoD Project

1. Introduction

NLR is a professional organization working in the field of leprosy and disability in 16 leprosy endemic countries. Each year, around 219.000 people globally are diagnosed with leprosy. Indonesia ranks number 3, with relatively high disability rates. The main partner of NLR Indonesia is the Ministry of Health (MoH) and activities are aligned to MoH policies, priorities and roadmaps. At provincial level, the Leprosy Control Program Technical Advisors (LCTAs) are strengthening the capacity of the provincial and districts government’s health staff to improve the quality of leprosy services. Next to supporting leprosy control, NLR Indonesia’s Inclusion and Disability program aims to support and advance holistic, community-based rehabilitation and sustainable inclusive development for persons with disabilities, including children with disabilities and people affected by leprosy.

The project “mPOD: Reducing Disability, Enabling Futures” started in December 2014 and end in 2017; it using mHealth tools (sms gateway, mweb and Facebook discussion group/forum) for morbidity management and disability prevention in the Indonesian leprosy control program. An important objective of this project is to identify obstacles, find solutions and thereby develop methods that can be scaled up to other disabling diseases and to other regions. The project was piloted in Serang District of Province Banten and Gorontalo District in province Gorontalo.

In 2018 after the final year of implementation an evaluation NLR is planning to conduct an external evaluation in order to assess the effectiveness of the project and find out if there is still a necessity for this project. The evaluation will evaluate three years of implementation in pilot project areas, Banten and Gorontalo. The evaluation will focus on the OECD DAC criteria effectiveness and relevance.

2. Objective of the Evaluation

• To study the lessons learned and information on impact achievements toward mPoD Project
• To provide information on what worked, what did not work and why.
• To determine effectiveness, and relevance of project related to the prevention of disability due to leprosy.
• To provide recommendations for sustainability: specific, actionable, and practical recommendations in terms of project transition and sustaining to other related counterparts, especially Government actors.
• To provide recommendation for scale-up: the model and system, possibility for other diseases (or combined approaches), cost, human resources and other resources.

3. Scope of the Evaluation

The evaluation will be conducted in mid November 2018 with an estimated duration of maximum 15 days. The evaluation will cover both regions Gorontalo and Banten where the MleP was implemented. It will assess how the MleP project which is using mobile technology has improved the capacities of the health care providers by providing a mobile application, and regular supervision. It will assess how the People affected by leprosy have participated in the project cycle and benefited from it.

4. Evaluation Questions

Evaluation questions relate to the objective and scope of the evaluation and intend to measure standard OEC-DAC evaluation criteria: relevance, effectiveness, sustainability and impact of the Mlep Projects.

Relevance

• To what extent does the MleP project contribute to the country program, strategies, and approaches of Indonesia’s national priorities? Is there a relevance of a continued intervention along the lines of this project.
• After 3 years of implementation, to what extent is the design of the project still relevant? What are the gaps identified in the overall design (outcomes and outputs), approach, and strategies of the MLep project? How should these gaps be addressed? To what extent do these gaps call for further project action?
• Are the activities and outputs of the project consistent with the overall intended impact and objectives of the project?
• The extent to which MleP aligns with the priorities and policies of the government in its leprosy roadmap and of the beneficiaries needs?

Effectiveness of the Project

• To what extent has the country project achieved intended results (outcomes and outputs)?
• To what extent were results achieved? Do the outputs produced meet the required high quality? Are there any unintended benefits?
• Have the beneficiaries been better assisted through the MleP in all their needs? Have more people been able to benefit from the project?
• What are the three good practices of the MlEP project
• What are three important weaknesses of the project? (process, results)

Sustainability

• Is the MLEP project supported by national/local institutions? Do these institutions demonstrate commitment and technical capacity to continue to work with the project or replicate it?
• What good practices do you think would continue after the project phase out? Why?

Impact

• What are the intended and unintended, positive and negative, long-term effects of the MLEP project?
• To what extent are the benefits of the project likely to continue?
• To what extent can the identified changes be attributed to the MLEP project?
• What is the evidence that the MLEP project prevented disability and supported morbidity management?

5. Tangible Outputs

• Inception Report – Analysis of existing situation and plan of work for the assignment; no later than one week after the commencement of the consultancy. The inception report will include a draft outline of the final report.
• PowerPoint presentation of preliminary findings to NLR and key stakeholders. It will be presented at the end of the field visits. The comments made by key stakeholders should inform the draft report.
• Interview Matrix and other raw material used for the data collection
• Draft Final Report – Description of achievements including problems encountered and recommendations; no later than 1 months after the commencement of the assignment. The reports shall not be more than 20 pages and written in the English Language.
• Final Report - Short description of achievements including problems encountered and recommendations; The final report is submitted within 2 weeks of receiving comments on the draft final report from the Project Manager identified in the contract.

6. Methodology

The methodology and techniques to be used in the evaluation should be described in detail in the inception report and in the final evaluation report, and should contain rationale and information on the instruments used for data collection and analysis, whether these be documents, interviews, field visit, and questionnaires of participatory techniques. 
Methods will have to include:

• Identification of a wide range of information source for data collection (documents, filed information, beneficiaries, staff, donor, experts, government officials and community groups etc)
• Proposal of the methodological framework
• Determine the instruments and methods for collecting the needed information (interview, observations, 
focus groups, literature review, survey, rating, knowledge test, site visits) and specify the sample procedure/s with each method
• Ensure that the main evaluation questions are addressed by multiple methods and data
• Project a schedule for information collection

7. Required Expertise of the Consultans Team / Specifications

The following qualifications are sought:

• Experienced evaluator(s), with professional track record in evaluations (incl. project)
• Understanding the Public Health situation in Indonesia
• Preferably good knowledge about health and IT solutions
• Language skills in English
• Ability to analyse situations, draw conclusions and recommendations
• Excellent reporting skills

8. Remunerations and Terms of Payment

The Consultant will receive a consultancy fee to be paid in the following installments:

• The first installment in the amount of 25% of the consultancy fee shall be paid upon submission and approval by NLR of inception report one week after commencement of the assignment
• The second installment in the amount of 20% of the total fee shall be paid upon finalization of the field research and the presentation of the initial findings
• The third installment in the amount of 20% of the total fee shall be paid upon submission and approval by NLR of DRAFT Final Report
• The final installment in the amount of 35% of total fee shall be paid upon submission and approval by NLR of the final report

9. Code of Conduct

The evaluation of the project is to be carried out according to ethical principles and standards established by NLR in its Code of Conduct to be understood and signed by the consultant(s).

Anonymity and confidentiality. The evaluation must respect the rights of individuals who provide information, ensuring their anonymity and confidentiality.

Consent of interviewees must be obtained prior to interviews, surveys and FGD’s

10. How to Apply

Interested applicants are invited to send:

• A short expression of interest (max 5 pages) clearly specifying understanding of the assignment, steps in the research, limitations, innovative suggestions as well as suitability and availability.
• Interested consultants, or organizations can submit their Expression of Interest (EOI) and CV)s)/Company profile to recruitment@nlrindonesia.or.id  indicating the title of this assignment, with the following contents:
• Introduction
• The projects and the Objectives
• Understanding of the assignment
• Scope of the evaluation
• Methodology (draft)
• Deliverables of the assignment
• Proposed team
• Draft Timeframe and Work Plan
• Budget Plan
• Portfolio indicating previous evaluation experiences with qualitative and quantitative methods on social issues.
• Samples of previous consultancies or evaluations, or assessments.

Application deadline will close at 5.00 pm (Jakarta time), 09 November 2018. 

Only shortlisted applicants will be contacted.

The assignment will be awarded to candidates whose proposal shows the soundest methodology and efficiency in using resources.

We promote equal opportunities for men and women and welcome applications from all suitably qualified persons regardless of their race, gender, disability, religion/belief, sexual orientation or age.