Endline Survey and Final Evaluation Consultant for Integrated Health-DRR Project in Wonogiri

TERM OF REFERENCE

Endline Survey and Final Evaluation Consultant for Integrated Health-DRR Project in Wonogiri

I.          Project Information

The Indonesian Red Cross (Palang Merah Indonesia/PMI) Wonogiri branch has started implementing community-based health and first aid (CBHFA) activities in 10 villages after the 2006 Yogyakarta earthquake. The CBHFA toolkit was then first piloted in Wonogiri CBHFA I Project starting in 2009 by PMI under the support of the American Red Cross (Amcross).

During March 2012 to March 2015, supported by Amcross and the Spanish Red Cross (SRC), PMI started implementing an integrated health and disaster risk reduction (DRR) project in Wonogiri District. The project aims to strengthen community resilience to natural disasters and health vulnerabilities by improving the capacity of targeted communities and PMI institution in Wonogiri District. The main outcomes intended are to increase PMI’s capacity in implementing Health and DRR community-based program and to strengthen PMI’s partnership with local authorities in order to develop, deliver and sustain such project. The project also aims to improve the community’s knowledge/practice to address and prevent priority health issues as well as to prepare for and respond to future disaster risk and impact

This project serves as a pilot in four fronts: 1) integration between health and DRR sectors; 2) utilization of RAMP survey in data collection; 3) roll out of healthy lifestyle module to address issues of non-communicable diseases (NCDs); and 4) establishment of a three-way partnership between PMI, Amcross, and SRC. The total fund of the project is USD 300,000, targeting 30 villages and 30 schools within 15 sub-districts (out of 26 existing sub-districts in Wonogiri District), which thus will benefit up to 116,850 direct and 726,370 indirect beneficiaries (such as community members, village volunteers, village committees, PMI personnel in Wonogiri District, Central Java Province, as well as NHQ). Among these 30 villages, 10 villages were identified from the previous Wonogiri CBHFA I Project, and the remaining 20 new villages were selected since they were considered to be most vulnerable.

The regular project activities consist of conducting training on health and DP/DR prioritized topic for staff, district facilitators and volunteers; and disseminating health and DP/DR messages through community outreach sessions, also school and, household visits. Based on the Community Vulnerability Assessment, there are six health and DP/DR topics that have been disseminated to targeted community and schools (i.e. diarrhea, dengue, TB, hypertension, road safety, flood, landslide, typhoon and fire)

II.        The Objective of the Endline Survey and Final Evaluation

Objective of Endline Survey:

The Endline Survey is expected to measure the related project indicators and to compare the value with the baseline result, which will ultimately be used to justify the changes made by the project intervention. The main objectives of the Endline Survey are:

1.      To measure the average score of respondent knowledge in the community in regards to the transmission, prevention and treatment of priority health issues by the end of the project (i.e. diarrhea, dengue, TB, hypertension) (B = 14.2 ; T= 45.8);

2.         To measure the average score of respondent knowledge in schools in regards to the transmission, prevention and treatment of priority health issues by the end of the project (B= 5.4; T=44.6);

3.         To measure the average score of respondent behavior in regards to the prevention of priority health issues at the community level by the end of the project (B=7.8; T=17.2);

4.         To measure the score of respondent behavior in regards to the prevention of priority health issues at the school level by the end of the project (B=1.0 ; T= 19.0);

5.         To measure the percentage of respondent in targeted communities who can correctly mention at least three disaster response measures by the end of the project (B=7.50 ; T=60%);

6.         To measure the percentage of respondent in targeted schools who can correctly mention at least three disaster response measures by the end of the project (B=2.70%; T= 50%);

7.         To measure the percentage of respondent in targeted villages who have implemented at least two required practices for disaster preparedness by the end of the project (B=5%; T=25%); and

8.         To measure the percentage of respondent in targeted villages and schools who can correctly mention the location of the agreed meeting point in case of emergency situation/disaster taking place by the end of the project (B=0% ; T=70%).

Objective of Final Evaluation:

The Final Evaluation serves as an instrument for change and plays a critical role in supporting accountability. The main objectives of this assignment are:

1. To measure the achievement of the project objectives based on its objective statements and indicators;
2. To determine the efficiency, effectiveness, impact, and sustainability of the project;

3.      To collect learning points and recommendations for future project design, also Integration of Health-DRR and NCDs piloting for other projects; and

4.      To collect lessons learned and success stories from the project.

III.      Methodology

Initial information will be collected through a desk review of PMI documents. The data collection for the endline survey will use RAMP (Rapid Assessment using Mobile Phone) technique with Magpi technology. Meanwhile, the data collection technique for final evaluation will include key informant interviews and focus group discussion with PMI board members, staff and volunteers, stakeholders and local government in Wonogiri; also observations with various participatory techniques to assess the knowledge and behavior patterns

IV.      Scope of Work

The consultant will be responsible for ensuring that the Wonogiri Project endline survey and evaluation is conducted in a scientific research methodology that meets the statistical requirements, with utilization of mobile phone data collection with Magpi Software. The scopes of work in details are as follow:

1.        To provide a complete proposal of endline survey and final evaluation. The proposal should contain survey objectives, methodology, sampling design, sampling calculation, sampling selection, and analysis plan. In addition, the proposal should also describe all of the detailed activities and timeline (i.e. proposal development, questionnaire development, enumerator training, data collection, data analysis and reporting).

2.        To develop and finalize questionnaires for endline survey and evaluation. This process will be done in close consultation with the Red Cross team, and the questionnaires will go through a pre-testing phase and final approval from the Red Cross team.

3.        To lead the endline survey and evaluation, to supervise the implementation of the field research, and to assist the technical expert from PMI in conducting  data collection using mobile phone. The consultant will coordinate closely with the Red Cross team, particularly PMI Wonogiri, in selecting the enumerators and field supervisors, also in designing and overseeing the listing operations for the endline survey. The data collection for endline survey will be led by a technical expert from PMI with skills on RAMP utilization with Magpi software.

4.        To conduct data management and analysis. Once the data collection has been completed, the RAMP technical expert will provide the data in excel format for the further analysis using SPSS. The consultant should ensure that complex sample design is taken into account, with calculation and use of sampling weights (as needed), also use of appropriate variance estimation technique based on the given sample design.

5.        To write reports. The consultant will develop a report of endline survey and final evaluation each, which includes narrative on the survey and evaluation processes and the result of data analysis. The consultant is also expected to present the results to PMI, Amcross, the International Federation of Red Cross (IFRC), and other relevant stakeholders such as the District Disaster Management Agency (BPBD), District Health Office (DHO), and schools, as required.

V.        Timeline 

The total working day is approximately 40 days. The consultation period to draft the proposal and to select the consultant is targeted to be completed on the 3^rd week of March. The next process will be the consultant’s working period, as follows:

No	Activity	Time
1	Desk review with the Red Cross team (including development of tool and data collection method)	30 March -3 April
2	Enumerator Training	9-11 April
2	Field data collection (endline: 14 days; evaluation:4-5 days for FGD)	12- 25 April (endline)/ possible for extension up to 28 April to conduct FGD
3	Data analysis (endline: 2 days; evaluation: 3 days)	27-28/30 April
4	Report drafting and consultancy with PMI and Amcross	4-15 May
5	Report submission, along with its annexes	18-20 May

VI.      Expected Output and deliverables

a.      Inception Survey and Evaluation design (Proposal).

b.      Finalized questionnaires, in Bahasa Indonesia and in English.

c.       All data files in SPSS & Excel (raw data sets, modified data sets, log files, code book).

d.      Sampling weights used in data analysis.

e.      Presentation of initial findings to PMI and Amcross, after field activities have been concluded.  

f.        Draft reports in Bahasa Indonesia and English within two weeks after data collection at the field is completed.

g.      Presentation materials on the findings.

h.      Submission of a comprehensive final report in Bahasa Indonesia and English within one week after receiving feedback from PMI and Amcross. The final report should include detailed information on the following:

-            Sample size calculation, sample design (domains, subclasses, stratification, clustering, number of units at each stage);

-            Allocation of units across stages, selection of units at each stage;

-            Methodology for selecting respondents at the final stage;

-            Questionnaire development;

-            Training of data collectors;

-            Listing operations;

-            Quality control measures;

-            Data cleaning and analysis (including the non-response, variance estimation technique used and how the complex sample design is accounted for); and

-            Lessons learned to inform future survey, including experiences in utilizing Magpi for mobile phone data collection.

VII.         The responsibility of Red Cross:

-            Providing data and related documents;

-            Determining the size calculations and sample design with advice from the consultant;

-            Providing equipment and logistics such as computer/cell phone (technology equipment) for survey management and data entry;

-            Training enumerators on how to collect data using cell phone;

-            Arranging meetings, discussions, transportation and accommodation in Wonogiri;

-            Providing logistic and equipment during the enumerator training; and

-            Providing timely feedback and response on the consultant’s report.

VIII. Qualifications

The required qualifications and experience are as follows:

a.      Post graduate degree from a recognized institution in related field (statistic, public health, social research).

b.      Experience in conducting and leading survey at least in the recent 7 years.

c.       Experience in survey work with similar purpose and with mobile data collection is strongly preferred.

d.      Experience in working with participatory methodologies.

e.      Excellent knowledge and skills in research methodology and statistical analysis. Demonstrated experience and expertise in designing, managing and analyzing data from complex-designed surveys (stratified, clustered, multistage, disproportionate selection probabilities, etc.).

f.        Advanced computer skills (SPSS and Excel).

g.      Excellent communication skills for training and facilitation, also leadership to manage and coordinate the data collector team.

h.      Excellent report writing skill both in Bahasa Indonesia and in English.

i.        Experience in project management.

j.        Ability to work with minimum supervision/direction.

k.       Ability to travel and stay in Wonogiri district during the data collection process.

l.        Ability to work independently, to juggle and coordinate many tasks simultaneously, to prioritize tasks, and to be well organized, reliable and trustworthy

IX.      Application procedures

Please submit your application and curriculum vitae in English not later than 24 February 2015 to recruitment.amcross@gmail.com or recruitment@amredcross.org , placing the job title in the subject line and label your CV with your name and documents.  Only short listed candidates will be notified.  Applications received after the deadline will not be considered.