Type of evaluation: Formative Evaluation (Baseline Survey)
Methodology: Quantitative method
Consultant involvement:
ü Submit Inception Report (Final Baseline Project Plan)
ü Baseline tools development
ü Baseline preparation
ü Data collection
ü Data management and analysis
ü Quality assurance and data protection
ü Submit Baseline Report
Location of work: Micronesia with travel to project sites
Expected date/period: 7 January – 15 February 2019 (40 days)
1. Project Background 1.1 Background and objectives
The Federated States of Micronesia (FSM) covers more than 600 islands with
only 702 square kilometers combined, spread across the west Pacific Ocean.
With different levels of exposure and sensitivity to typhoon and to El
Niño-Southern Oscillation (ENSO)-related impacts, FSM has experienced major
droughts and floods that affected a significant number of vulnerable
people, especially in coastal areas, where virtually the entire population
lives. The Micronesia Red Cross Society (MRCS), as an auxiliary to the
government of FSM, plays a unique role in disaster risk reduction and
emergency response in Micronesia. Since 2016, MRCS has been, in partnership
with the American Red Cross, implementing the Micronesia Community
Resilience and Capacity Development (CRCD), funded by the U.S. Agency for
International Development's. Office of U.S. Foreign Disaster Assistance
(USAID/OFDA). The goal is to reduce the number of deaths and injuries and
lessen the negative socio-economic impact caused by climate-related
disasters (e.g. typhoons) and environmental degradation in vulnerable
Micronesian islands through the Community-based Disaster Risk Reduction
(CBDRR) approach. Throughout the first phase (March 2016 – September 2018),
and NCE (September-December 2018), the project has been implementing in 21
communities in Chuuk State, and a CBDRR framework and manual has been
developed and used by MRCS to assist the implementation of community and
school-based activities. The WASH-related infrastructure and a Coastal
Green Belt Rehabilitation Initiative has also been used to enable MRCS to
enhance its CBDRR approach by piloting coastal green belt (mangrove and
other vegetation) as natural protection in five communities.
Starting in January 2019, the second phase of the project will continue to
build on community-based disaster risk reduction progress, working with
affected coastal populations, and assisting the MRCS to expand CRCD
activities. This new phase of intervention will assist 11 communities in
the States of Chuuk and Kosrae, with two main objectives that are 1) to
strengthen the resilience of vulnerable small island communities to
mitigate risk and negative impact of coastal hazards, especially typhoons
and 2) to strengthen coastal community resiliency through improved water
sources, sanitation along with corresponding hygiene practices.
Augmenting investment
in disaster preparedness and readiness capacity, the expected outcomes
include:
1. Reduced vulnerability to coastal hazards through a replicable
integrated community-based disaster risk management program;
2. Reduced vulnerability to coastal hazards through a replicable
School-based Disaster Risk Reduction program;
3. Reduced vulnerability to coastal hazards through improved Coastal
Green Belt and its management;
4. Strengthened DRR/CCA-related partnerships between MRCS and
public-private sectors through improvement of Chapter readiness to response
capacity and volunteer networks;
5. Reduced community vulnerability through improved and strengthened
water infrastructure especially protected from hazards such as typhoons;
6. Improved community sanitation infrastructure available to assist
persons seeking refuge in evacuation centers, especially during typhoons;
7. Improved hygiene behaviors and practices, leading to better
environmental and community health.
1.2. Project areas
The Phase 2 of the CRCD project targets 11 municipalities, comprising 9
municipalities (and 5 schools) in Chuuk State and 2 municipalities (and 2
schools) in Kosrae State as detailed in Table 1 and 2 presented in Annex 1.
From the tables, targeted municipalities are grouped based on exposure of
time of intervention (Phase 2 only or both phases) and the four different
type of interventions that include: Community Based Disaster Risk
Reduction, School Based Disaster Risk Reduction, Green Belt Rehabilitation,
and Community Evacuation Center with improved WASH.
1.3. Coverage of evaluation
It is expected that the formative evaluation (Baseline survey and
supplementary qualitative data collection) will cover targeted population
for Household survey at community level and student survey with some
observations at school level. Due to geographical challenge, it is
suggested that the survey to exclude Tafunsak and thus will become one of
study limitations. Municipalities from Mortlocks, Chuuk, are located in
different island from Chuuk, hence, it is expected that consultant will
consider time allocation for enumerator trainings and data collection in
for different islands.
2. Evaluation Overview
*2.1 Objectives of Formative Evaluation *
The objectives of the formative evaluation are to:
1. Provide *descriptive information of project population*
2. Measure *baseline status of project indicators*
3. Determine *existing knowledge, perception, capacity, and behavior
related to disaster risk reduction, climate change, coastal management,
safe and hygiene practices, and potential barriers for desired behavioral
changes at community and school levels*
4. Provide *recommendation for behavior change communication strategy*
(best targeted audience, suggested key messages, communication tools and
channels) and other *technical* *recommendation for project*
*implementation*.
2.2 Expected Baseline survey
i. Indicators measurement
To achieve the objectives, it is desired for baseline survey to measure the
baseline status. However, the baseline questionnaire will not limit to the
indicators only to obtain comprehensive understanding towards existing
knowledge, attitude, practices, and potential barriers on safe and hygiene
practices. Therefore, the selected consultant, under consultation with
project team, is expected to develop additional questions, particularly for
information seeking on potential barriers. In proposal, consultant has to
explain proposed methodology, whether baseline survey alone is sufficient
to meet the formative evaluation objectives or combined with qualitative
methodology considering the geographical challenge and time allocation.
ii. Survey type
Comparative – Baseline Comparative – Endline
iii. Geographic scope & target population
The survey will be carried out in all target communities representing each
phase of project. Please look at Table 1 and 2 in “Project Areas” section
and consider the evaluation coverage to determine sample size. The
consultant should provide sampling method given the best fitted formula
proposed in the proposal.
iv. Mode of data collection
It is highly desired that the consultant is willing to develop
questionnaire and lead data collection using KoboCollect (preferable) and
Data collection software based on Open Data Kit (ODK) platform and manage
the daily data collection pooled on the cloud. Consultant candidates are
allowed to use paper questionnaires for student survey, however, they
should provide data management and quality control plan and describe their
strategy on the proposal.
v. Unit(s) of analysis
Enumerators, Field Supervisors/Coordinators, and persons in charge for data
entry would be MRCS personnel (staff and volunteers) who will be trained by
the consultant.
vi. Required precision & Desired difference proportion
A standard level of required precision would be for a value of the relative
standard error (or coefficient of variation) of key variables ≤ 0.05. The
desired difference between 2 proportions (baseline and endline) is 10%.
Estimated 500 HH will be interviewed during the survey. For schools’ pen
and paper test will be conducted for selected class rooms.
3. Scope of work and Evaluation design 3.1 Scope of work
The consultant will be responsible to ensure Micronesia Community
Resilience and Capacity Development Project Formative Evaluation uses
methodologies that meet standard research and evaluation. The scope of work
must include:
1. *Provide a complete Inception Report (Final proposal and tools)*.
The inception report will contain (1) the formative evaluation objective;
(2) scope of work; (3) indicators measurement; (4) respondents with
description of target population, important characteristics, context of the
project, the proposed sample formula, sampling allocation and selection
criteria, justification of sample weight (if needed), and response rate;
(5) data collection instrument; (6) work plan provided in a table comprised
of activities, days, date, responsible consultant team member; (7) details
on work plan with a table comprised of activities and their description;
(8) analysis plan explained in a table consisted of type of analysis and
its utilization; (9) consultant deliverables; (10) additional comments (if
applicable); (11) resources listing the resources needed in a table
including list of resources needed, number, unit, and details; and (12)
Annex with all supporting documents attached on the report. This Inception
Report is made by the selected consultant based on selection results
between Red Cross team.
2. *Develop two questionnaires *for household survey at community
level and for students at school level. This process will be done in close
consultation with Red Cross team in development (draft tools will be
provided by Red Cross), and questionnaire pre-testing, and final approval
from the Red Cross team. The draft questionnaires must be tested and
finalized as part of Annexes in Inception Report. The Inception Report’s
template will be given to the selected consultant.
3. *Coordinate the utilization of *KoboCollect or ODK (or other
standard mobile data application) Once the questionnaire has been
finalized, the consultant will develop questionnaire in the ODK or other
mobile data collection methods with MRCS staff to be accessed by
enumerators for field data collection. Survey with students will be carried
out on paper questionnaire.
4. *Develop data quality control* measures to the data entry. The
consultant should propose quality control methods to ensure the high
quality of data collection and data inputs (i.e. field check, double
checking for data entry, procedure for data cleaning) .
5. *Train data collection team *(enumerators, personnel of MRCS for
data entry, field supervisor, etc.) The consultant should prepare training
materials, such as survey background, methodology and sampling design,
questionnaires, interview technique, introduction of ethical guidelines,
mobile phone utilizations, and field practices. Training will be conducted
separately for Faichuuk and Mortlocks regions and each will be conducted
for at least 3 days.
6. *Lead and coordinate data collection*. Coordinate closely with Red
Cross team, particularly MRCS in Head Quarter, Chuuk, and Krosae to
selecting the enumerators and field supervisors and design and oversight of
listing operations of the survey. The enumerators criteria will be
discussed during initial briefing. Consultant should conduct and oversight
of data collection at field, evaluate data collection in daily basis and
provide daily progress report to the Red Cross team, including faced issues
at field and actions taken to address the issues.
7. *Conduct data management and analysis. *
The consultant should report progress of data collection in daily basis.
Upon the completion of data collection at field, the consultant has to
conduct data cleaning using standard statistical package software for
analysis. The consultant should consider the baseline survey design for
end-line survey analysis in final evaluation, and use of appropriate
variance estimation technique given the sample design and method to reduce
potential bias.
8. *Write and present Project Final Evaluation Report; *develop a
Project Formative Evaluation Report and present the key finding results to
MRCS, American Red Cross delegation, and/or other stakeholders as relevant.
3.2 Logistic and Administrative support
The consultant is expected to use her/his own computer. Approved
administrative and logistic costs will be reimbursed by the Red Cross
Team. The consultant will be able to work remotely, after approval of the
work plan.
3.3 Reporting relationship
The consultancy service agreement would be facilitated by MRCS. The
consultant will report to Dino Argianto DRR delegate American Red Cross
with technical support from Madiatri Silalahi M&E officer.
4. Expected deliverables and Duration of consultancy
4.1 Expected activities and deliverables
i. Inception Report (Final proposal and tools) prior data
collection
ii. Conduct initial briefing and document review
iii. Finalized survey questionnaire
iv. Provide Training Manual for field staff
v. Train enumerators
vi. Implement data quality control (field check, double checking of
data entry)
vii. Perform data entry, cleaning, and analysis. All data files in
statistical software (preferably SPSS) and Ms. Excel (raw data sets, merged
& modified data set, log files, code book)
viii. Provide progress report of quantitative data collection at daily
basis (target vs achievement, response rate, main issues faced and actions
taken)
ix. Draft Report
x. Presentation materials
xi. Final Report. The Final Report should include contents as follow:
- Cover
- Glossary and Acronyms
- Table of contents
- List of tables
- List of graphs
- Executive summary
- Introduction, covering purpose of report, purpose of evaluation,
detailed methodology, actions to reduce potential bias
- Context, including the project’s approaches/components at
community level
- Study limitation
- Key findings with provision of the baseline scores from the
Project’s Logical Framework and baseline survey results (along with
narrative description)
- Conclusion
- Recommendation for project implementation
- Reference
- Annex including photos taken during data collection and all
supporting documents
xiv. Provide all data collected and used for analysis and report writing
(raw data, clean data set, table of sampling allocation/frame)
*All of documents/deliverables should be in English
*4.2 Duration of consultancy *
The consultancy will be provided within 40 working days, starting from 7
January to 15 February 2019.
*Estimated consultant working day*
*Activities*
*Estimated day(s)*
1. Initial briefing between consultant and Red Cross team
1
2. Inception report preparation (including questionnaire testing) and
finalization
5
3. Enumerator training and Data collection at field (including travel days)
(in Kosrae and Chuuk)
20
4. Data cleaning and analysis
6
5. Report writing
4
6. Presenting Final Baseline Report
1
7. incorporating comments from American Red Cross
2
7. Submitting all required deliverables and supporting documents
1
*Total*
*40 days*
5. Required qualifications
Required qualifications for a single consultant or consulting team are:
a. Five years of experiences conducting and leading both quantitative
researches, three of which should be with international organizations.
b. Excellent knowledge and skills in research method, particularly
for quantitative methodology. Be able to show experiences in designing
formative evaluation including baseline survey, managing and analyzing
quantitative data.
c. Experiences in leading evaluation, preferably for disaster risk
reduction projects.
d. Demonstrated expertise in disaster risk reduction, integrated
coastal management, and behavioral changes.
e. Demonstrated experiences in questionnaire development,
particularly using mobile application for data collection.
f. Demonstrated experiences in training enumerators and mobile data
collection preferably using ODK.
g. Demonstrated experiences in behavioral changes related survey.
6. Application materials and procedures 6.1 Application materials The
consultant candidates are expected to submit the proposed Project Formative
Evaluation Proposal with table of content as followed:
*1. **Consultant profile *
This section includes summary of the Evaluation team, explaining in short
paragraphs all members’ education background and relevant working and
evaluation experiences. Details of each members’ background etc., have to
be summarized in CV that is part of Annex.
*2. **Evaluation purpose*
Stating objectives of evaluation as written on the TOR.
*3. **Scope of work *
Stating the scope of work that is cited from the TOR. *I*f there is more
than one consultant on the proposed team, please provide the number of
persons needed for both for quantitative and qualitative methodologies.
Attach a table describing the level of effort (in number of days) of each
team member in each of the evaluation activities.
*4. **Enumerator training*
State who will be responsible for enumerator training and include
information on prior experience. Describe how training will be provided,
the topics covered, expected duration, and logistic and administrative
support needed from Red Cross team.
*5. **Quality control mechanism*
Provide a section detailing the mechanisms to ensure data quality by
clearly specifying steps for data validation. This section may also include
supervisory mechanism for data quality and the role of field supervisors.
*6. **Data management*
Explain mechanism for data entry, validation checks, and data processing.
*7. **Analysis plan*
The consultant team is expected to explain data analysis plan, software
used, and working duration. The consultant has to be able to explain
strategy to avoid potential bias from the proposed baseline design.
*8. **Progress updates*
This section should clearly indicate the mechanism that would be used to
communicate with MRCS and American Red Cross team within the reporting
relationships on regular update about the field activities, response rate,
etc.
*9. **Evaluation timeframe*
Complete this section with proposed work plan. Provide a table comprised of
description of Formative Evaluation activities, estimate days of
implementation, and the consultant team member responsible for each planned
activity.
*10. **Evaluation budget plan*
Provide a table that shows planned activities, estimate days of
implementation, responsible evaluation team member, and unit cost in US
Dollar.
*11. **Professional Reference *
Provide at least 3 references with details as follow:
- Name of reference
- Position and organization
- Mobile phone number and email address
- Previous consultancy service
*12. **Annex*
The Annex should be consisted of:
- CV of each Evaluation team members
- Sample of previous Baseline Survey report in English comprised
of*: Executive Summary (1 page), Analysis results and key findings (4
pages), Recommendation (1 page), and Conclusion (1 page).*
6.2 Application procedures
Please submit your proposal application in English not later than* 6
December 2018* to* recruitment@amredcross.org
<recruitment@amredcross.org> *with
*Micronesia CRCD Phase 2 - Baseline* in the subject line and label your CV,
proposal, and writing sample with your name and document type in the file
names. Only short listed candidates will be notified and contacted for an
interview. Applications received after the deadline or incomplete
applications will not be considered.
7. Selection criteria and Payment term 7.1 Selection criteria
1. Qualifications section
2. Quality of proposal
3. Experience in evaluation
4. Expertise in organizational development
5. Number of Days and Timeline availability
6. Rational budget plan
7. Previous evaluations in PMI and American Red Cross projects
7.2 Payment term
The payment will be made based on satisfactory deliverable and after
approval from the Sr. Program Advisor, American Red Cross AMEE-Regional
Office. Consultant need to submit invoice for the payment.
Payment term
ü 20% after Inception report (Final Proposal and Tools)
ü 40% after submission of Draft 1- Report
ü 40% after approval of Final Report and all deliverables.
No comments:
Post a Comment