TERM OF REFERENCE
End Evaluation: Community Managed Early Childhood Care & Development Project
Plan International Indonesia is an independent child rights and humanitarian organization committed to children living a life free of poverty, violence and injustice. We have been building powerful partnerships for children for more than 75 years, and are now active in more than 70 countries. In Indonesia, Plan International has been working since 1969 and currently works in 7 provinces and sponsors more than 40,000 children.Since 2011, Plan International Indonesia has implemented the Community Managed-Early Childhood Care and Development Project (CM-ECCD) with funding from the Australian Government through the Plan International Australia National Office. Starting in two districts, Sikka and Lembata, the project then extended to Timor Tengah Utara District in 2013 and since 2014 has further extended to Timor Tengah Selatan and Nagekeo district. The most recent phase of the project (FY16 to FY17) has focused on two outcomes which are integrated parenting activities and advocacy at district and national level. The Plan Indonesia CM-ECCD project seeks to prove that 100% of disadvantaged children aged birth-8 years in targeted high poverty communities can achieve child wellbeing indicators through effective and quality community-managed early childhood supports. The approach must be appropriate for rural early childhood center settings; user friendly for communities in Indonesia; and implementable through the Posyandu (community-run health centres available in every village) and other existing programs. In this way, Plan intends to contribute to the goals of the Government of Indonesia to improve Holistic Integrative Early Childhood Development outcomes (PAUD HI).
2) AimThe aim of the end evaluation is to review the CM-ECCD project of Plan Indonesia, to establish the extent to which the project has met/or did not meet its set objectives (as outlined above and in the Phase II project design) and to understand how the project has contributed to change for children, families/caregivers, Plan and partners and government. The key areas of inquiry are the Plan parenting program and the advocacy/influence agenda of Plan Indonesia, related to the external policy environment for ECCD and parenting in the country. The advancements that Plan Indonesia has made in terms of government engagement and advocacy, and challenges faced, are of particular interest and should be explored in depth, including through development of a case study that can be shared externally. It is intended that the overall evaluation will be used as an evidence base for both successes and challenges of the project, and will help provide a blueprint for further scale up and replication. Aligned with the new Plan global strategy, it is also important to analyse to what extent the project has been or could be gender transformative.
2. objectives and the key questions
The objectives of the evaluation are as follows:
a. Identification of early childhood parenting situation, this requires information on: Child development status (cognitive, language, social/emotional) based on Child Well-being Index priority indicators of Plan Indonesia; Parenting status and Identification of enablers and barriers, especially for increasing quality of parenting activities; Effectiveness, relevance and impact of the parenting project interventions; Issues of equality, inclusion, participation and non-discrimination, particularly in regards gender and disability inclusion.
b. Advocacy: To evaluate the effectiveness, relevance and impact of the advocacy efforts and their contribution to improving the external environment for children birth-8 e.g. policy at district and village level that supports Holistic Integrated ECCD; what programs have been conducted by partner/communities regarding Holistic Integrated ECCD; identification of enablers and barriers, especially for increasing access and quality of ECCD.
c. Sustainability, Scaling Up and Replication: To understand the sustainability aspects (not just activities but of impact as well), including what elements of the project need to develop further to enhance sustainability and to identify the challenges of sustainability, if any; To include specific recommendations on scaling up and replication.
The evaluation design should refer to the Community Managed ECCD Project outcomes (as per project designs for Phase I and II, with a focus on parenting and advocacy). To assist the study team, key questions will be built around the following aspects:
1. Effectiveness: (a).To what extent were the project objectives achieved/are likely to be achieved? What are the main factors (barriers and enablers) faced in the implementation of the project and influencing achievement/non achievement of the objectives? (b). How well has the internal monitoring system contributed to a process of continuous learning and adaptation during project implementation?; (c). How do the activities at district, provincial and national levels impact on the village level? e.g. formation of HI ECCD taskforces at district level or District Regulations about HI ECCD; (d). How does the effectiveness of project partners or stakeholders contribute to the project achievement? ; (e). How effective have parent education and awareness raising activities on ECCD and child development been i.e. what has been the impact/value of Plan’s parenting approach? Has Plan parenting bought about behavior change and parenting empowerment, particularly women’s empowerment and father engagement? ; (f). How effective has the advocacy/influence agenda of Plan Indonesia been in changing the external policy and budgetary environment for ECCD and parenting in the country?; (g). To what extent have standard guidelines for HI-ECCD been adopted and influenced policy, practice and budget allocation at village and district level? In what way? Are there accountability and transparency measures in place?; (h). What programs have been conducted by partners/communities regarding HI-ECCD, under influence from Plan?
2. Relevance: (a). Do the objectives and goals of the project match with the problems or needs being addressed? ; (b). How appropriate was the original project design for achieving the results expected? i.e. did the project logic lead to the desired changes?; (c). Was the implementation strategy of the project able to adapt and respond i.e. to changes in the quality of parenting activities and raising awareness of related stakeholders? Why/why not? How well? (d). To what extent did the project (particularly re parenting and advocacy agenda) ‘fit’ with/complement partner government policies?
3. Efficiency: Are the resources invested (financial and human) justified by the achieved outcomes?
4. Impact: (a). To what extent has the project improved parents/caregivers quality of parenting KAP? ; (b). What are the key outcomes for girls and boys related to healthy and strong; happy and smart; safe and protected? (as per Child Wellbeing priority indicators); (c). What has been the impact of community actions for child wellbeing?; (d). Do the beneficiaries take lessons learnt from the project implementation? Do they feel satisfied with it?; (e). To what extent has Plan integrated program implementation and coordination within the structure of local government administration?; (f). To what extent have local/district regulations for ECCD been enacted in Plan target areas? What impact has the project had on village and district level ECCD provision? Can this be attributed to Plan’s advocacy agenda?; (g); How successful has Plan been in building people’s/local government awareness about the importance and advantage of the parenting program and HI-ECCD?; (h). What is the evidence i.e. re budgetary, policy, human resource allocation etc.?
5. Equality and Non-discrimination: (a). To what extent has the project impacted the knowledge, Attitude, Practice (KAP) of beneficiaries related to Plan’s gender house analysis which consist of four fundamental analysis, which are: Social Relations of Gender, Gender and Power, Accountability, Gender-based child rights violations, Sustainability? E.g. Do ECCD taskforces promote gender equality (are there female task force leaders, to what extent do they reflect on and engage with gender dimensions of ECCD? Are national standard guidelines for HI-ECD gender aware or gender transformative?); (b). What needs to change to contribute to greater gender equality outcomes and gender transformative potential?
6. Sustainability and Scaling up: (a). To what extent could the project achievements be sustained? What were the major factors which might influence the achievement or non-achievement of sustainability of the project?; (b). How have changes been institutionalized at different levels?; (c). Is there any possibility of project method or lesson learnt being continued by government at macro and micro level for development of a program specific for parenting activity?; (d). Do government partners feel ownership of the project (especially re parenting and operationalizing HI-ECCD)?; (e). Is there any possibility of project method or lesson learnt being replicated or scaled up by other institutions? ; (f). What is the level of involvement of representatives of the Ministries of Health and Education, the National Family Planning Board and other relevant government agencies in targeted province/s?; (g). How well has Plan established and operationalized relevant government partnerships?
3. Methodologies to be used in the data collection and analysis; and for discussion with stakeholders
The evaluation will apply combined qualitative and quantitative approach through secondary and primary data collection. Gender-sensitive approach is compulsory in all stage of the studies. Alternative of methods are in-depth interview, closed questionnaire, and FGDs with various stakeholders on district, provincial levels and national level, community groups, representatives of business and academic, etc. Capturing the voice of children in child friendly ways, and also women, is compulsory. Applicants should outline in more detail the methodology, including response to key questions outlined above and any other proposed key research questions.
Case studies- Up to 3 case studies should be collected that reflect on change as a result of the project. This should include a detailed case study on the integration of Plan’s parenting program with that of the government, the process, challenges, lessons learnt and way forward etc. – with a view to also making this case study relevant for other Plan ECCD projects. This case study should also reflect on how successful Plan’s advocacy/influence agenda has been or can be on improving child wellbeing. Other suggestions for case studies include father engagement, girls and/or women’s empowerment.
Location : Nagekeo District, Sikka District, Lembata District, Timor Tengah Selatan District, Timor Tengah Utara District, Provincial level and National level.
4. Scope of Work
The consultant is expected to form a team for the evaluation and manage coordination and communication with Plan International Indonesia. Tasks and responsibilities of the consultant are as follows: (example)
1. Proposal that detailing the methodology and budget required to conduct the evaluation
2. Propose tools to be used in the evaluation, including sampling frame and quality control scheme and discuss the suggestion with Plan International Indonesia
3. Develop a data entry system and management
4. Supervise field research/evaluation implementation
5. Responsible for field operations, including logistics, permission to conduct the research, informed consent from individuals and families taking part in the research/evaluation
6. Coordinate with project management during field research implementation
7. Collect, compile and analyses all data (gender and age disaggregated) gathered and develop a draft and then a final report (including all findings and statistics)
8. To keep all information provided by Plan International Indonesia as well as the findings of the research/evaluation confidential.
The consultant is expected to produce and submit the following deliverables:
· Study protocol specifying a detailed study work plan and proposed study tools that will be discussed and agreed upon prior to field activity
· Presentation of initial findings to Project team and MER&D department team, after field activities have been concluded (narrative and PowerPoint version)
· Draft reports in Bahasa and English in two weeks’ time after field activities have been concluded
· Presentation of draft and reporting data flow diagram to Plan International Indonesia for discussion and feedback
· Submission of a comprehensive final report in Bahasa and English two week after receiving feedback from Plan International Indonesia
· Hardcopies of filled-in questionnaires (if available), interview transcripts and attendance lists, photos with informative subtitles and other valuable study materials (For Plan International Indonesia Documentation)
· Presentation on finding and recommendations in dissemination workshop
· Optional - Produce Information, Education and Communication (IEC) material, e.g. infographic, that will be used for advocacy purpose
6. Indicative Timeline
1 – 13 April 2017
Selection of consultant- Agreement signed
10 - 22 April 2017
Draft design and tools for end evaluation
24 April – 21 May 2017
Conduct end evaluation (training enumerator, data collection and analysis)
30 May 2017
9 June 2017
Dissemination workshop of evaluation result
7. Ethical and child protection statements
The consultant team and Plan International staff have to adhere to Plan International Child Protection Policy. The consultant team must follow Ethical Principles for involving human subjects in research and obtain written/ verbal consent from the human subjects. Permission from elder must be sought if the children under 18 years are involved as subjects. Signed informed consent of each child and his/ her parents need to take after explaining purpose of the study and its usage. Training on this will be part of the training provided for the survey team.
8. Selection criteria for evaluators
The evaluation has to be led by a highly qualified research consultant with strong capacity in conducting gender sensitive studies. Consultant/s interested in submitting a proposal should meet the following criteria:
1. Possess equal composition of qualified academic background, knowledge, experience and capacity to manage the study
2. Demonstrated experience in at least 3 comparable evaluations/end line surveys, preferably in the area of ECCD project intervention or in a relevant area
3. Extensive experience in managing studies in the context of Child or human rights in the area of ECCD, education, gender, community empowerment approaches or in other relevant areas
4. Good knowledge of the study areas
5. Good knowledge and understanding of the political and policy context in Indonesia, preferably in the area of ECCD.
6. Experience of developing, managing and/or evaluating advocacy projects
7. Understanding of project management structures and change management concepts
8. Ability to work in a focused manner and to agreed timelines.
9. Excellent in report writing, both in Bahasa and English
10. Experience of working with participatory methodologies
11. Strong experience in disseminating research findings
Minimum requirement for data collectors (where applicable): High demonstrated experience in undertaking research; Demonstrated experience of data collection; Excellent analytical skills; Good team work; Commitment in promoting participation of vulnerable groups
9. Submission Protocol
First step: Consultant send application to: Ajun.Khamdani@plan-
and cc. to Sulistiono.sulis@plan- international.org;
before 11 AM, 3 April 2017 by enclosing the following required documents:
1. Organizational Profile (for institutional consultant) or a CV (for individual consultant).
2. Proposal containing two parts:
a. Technical Proposal comprising the description of proposed activities, methodology (sampling design, data collection technique), and implementation plan which includes study schedule and timeline for data analysis. The technical proposal should demonstrate understanding of study ethic, qualification of study team members and their CVs, as well as their roles in the study. The consultant should also describe her/his experience in conducting similar survey, and financial and technical capacity (Ms. Words or pdf format)
b. Budget Proposal comprising details of budget plan such as consultant and team’s fees, transportation, stationeries and other survey necessities. (Ms. Excel format)
c. Sample of previous research or survey report
Second Step: Based on received proposals, Plan International Indonesia will shortlist candidates and invite them to present their proposals. Plan International Indonesia will afterwards proceed with selecting and appointing a consultant.
Only shortlisted consultant will be contacted for recruitment.