PLAN INDONESIA - Consultant Final evaluation project YEE Hyundai - IDN0218

TERMS OF REFERENCE

Auto-mechanic skills training project to empower for youth in Jakarta, Indonesia

(HYUNDAI∙KOICA Dream Center, Indonesia)

Pulogadung Industrial Area - Great Jakarta Province

Funded by: Hyundai & KOICA

Plan Indonesia – YEE Program

 

A.            Background of consultancy

Indonesia has the world’s fourth largest youth population with over 38 million young women and men between the ages of 15 and 24. The majority of Indonesian youth cannot reach their full potential including their right to an adequate standard of living, because they are unable to continue their schooling, find jobs, start businesses or take part in the economy at large. Indonesian youths have limited access to employment opportunities, skills, and financial services.

 Plan Indonesia has implemented a five projects of Youth Economic Empowerment , in Rembang, Grobogan (Central Java)  Surabaya (East Java ), Soe & Kefa (East Nusa Tenggara) and Great  Jakarta. Name of project in Jakarta is Auto-Mechanic Skills Training to Empower Indonesian Excluded Youth, and its location is Industrial Area, Pulogadung, East Jakarta.

Project’s gool is to economically empower at least 250 excluded youth aged 18-24 enabling them to find decent employment, become economically secure and improve their living standards. The youth will come from cities in the greater Jakarta and its arround. The project directly target 250 youths get auto-mechanic knowledge and skill.   This project is supported by Hyundai Motor Company and KOICA (Korea International Cooperation Agency).

A final evaluation of the project will be conducted at the end of the project. The purpose is to capture the lessons learned up to the time of the evaluation and so guide expansion and continuation of the project and enable the dissemination of the lessons learned to other location of project. It is also envisaged that the review will clearly recommend the actions necessary to strengthen the capacities of schools to assume responsibility, in partnership with local government, for essential basic education services. For this consultancy, Plan Indonesia seeks to procure the services of an independent, external consultant(s) to design, plan and conduct a rigorous final evaluation.

 

B.            Project Information

Project Objectives:

1.    Improve auto mechanic training environment and empowerment through modernize/ upgrade equipment

2.    Provide quality education and link to job placement include enterprise

3.    Improve house income and decrease poverty index

4.    Ensure that the project becomes sustainable and has the potential to scale up.

Project Output:

1.      Well equipped training center

2.      Trainees who has the skills for decent job

3.      Secured fund for stainable management and empowerment of center

4.      Project sustainable

 

C.            The Objective of Final Evaluation

The project Final Evaluation serves as an instrument for change and plays a critical role in supporting accountability. The main objectives are:

1. To measure the achievement of the program objectives of “Hyundai.KOICA Dream center Indonesia”.
2. To determine efficiency along the project implementation.
3. To determine the extent to which “Hyundai.KOICA Dream center Indonesia” Program  has met the objectives and planned outcomes outlined in the project document;
4. To collect learning and recommendation to develop direction and program design for next funding.
5. To assess the project implementation of CCCD approach over the 2013 to 2014 (to what extent the programme is consistent with key aspect of CCCD approach)
6. Gauge the sustainability of the outcomes and measures that can be taken to improve it in future.

 

The Final Evaluation will take stock of “Hyundai.KOICA Dream center Indonesia” Program achievements over the project period from inception in 2013 to February 2014. It will review activities and analyze the extent to which their outcomes are fulfilling planned targets. The Final Evaluation findings will serve as an update for improving project performance compare to baseline.

F.    Key question of Final Evaluation

 

The review shoud be able to comprehend the “Hyundai.KOICA Dream center Indonesia” Program framework and conceptual framework in the development of study design properly that meet to the project requirement. To assist the survey team key question is the survey will be built around the following factors;

             1.     Effectiveness :

o   What is the last status of the project? Is the “Hyundai.KOICA Dream center Indonesia” Program achieving all of project objectives from the above points? What are the main factor/challenges faced in the implementation of the project (only if the project didn’t achieve target/objectives)?

             2.     Relevance :

o   Do the “Hyundai.KOICA Dream center Indonesia”  Program objectives and goals match with the problems or needs that are being addressed?

o   Was the “Hyundai.KOICA Dream center Indonesia”  Program implementation strategy is able to adapt and respond to changes in the quality of training services (including the instructor's perception changed and the level of support)? Why/why not? How the project to adapt and respond to these changes?

             3.     Efficiency

§   Do the “Hyundai.KOICA Dream center Indonesia” Program delivered in a timely and do the budget allocate for this project is comparable with the result achieved?

             4.     Impact

o   How extent the project improve youth income

o   How extent the project improve youth quality of life  

o   How extent the project improve youth life skill

 

             5.     Sustainability

o    Is there any possibilities of project method or lesson learn to be continued by provincial education office of DKI Jakarta or other DKI Jakarta government office.

o    Is there any possibilities of project method or lesson learn to be replicated or be scaled up by other institution.

 

G.     Methodology

Initially information was gathered through a desk review of Plan Indonesia and various program partners working in the sector. Following the selection of sites meetings and discussions were undertaken with Puslatdikjur, training supervisor, instructors/trainer, youth. Data collection techniques included, key informant interviews with government staff, instructors and youth, focus group discussions with  stakeholders, observations of various participatory techniques to assess knowledge and behaviour patterns.

The consultant is required to propose a suitable study methodology as specified by this TOR. It will inform the methodology details such as sampling technique, sampling frame, data collection and analysis potentially applied in this study. The study will be conducted in “Hyundai.KOICA Dream Center Indonesia” areas where a number of activities have been planned or during the training, activities are being implemented. And therefore, coordination with PME team, YEE Team, Puslatdikjur, Training supervisor, Project Coordinator and staffs is highly required at all times.  Due to the child centered of the project, the methodology used should be child friendly and participatory wherever possible.

During the study, the consultant is expected to perform:

·           Desk-based review on existing relevant documents with Plan Indonesia team

·           Briefing session with YEE-Hyundai Project Team and Plan Indonesia PME

·           Study with questionnaires for instructors, principles, supervisors, training participant, parent and communities;

·           Focus Group Discussions with instructors, principles, training team, communities, training supervisors and project  partners, etc.

·           Separate consultations, and if required observation with youth and their family/spouse, by arranging sessions with youth group or peer counselling group established by  YEE team.

·           Key Informant Interviews particularly in-depth interview with other stakeholders selected to participate in the study, such as instructors/principles and Puslatdikjur, training center,  etc.

·           Field visits to “training center” supported by the program

·           Debriefing with YEE staff and Plan Indonesia PME

 

No.	Qualitative	Sample
1.	FGD	§  Instructor/trainer §  Peserta pelatihan §  Bengkel/industri otomotif §  Asbekindo §  Puslatdikjur §  LPTKP (Lembaga Pemberdayaan Tenaga Kerja Penyandang cacat) §  YDBA (Yayasan Dharma Bakti Astra) §  Hyundai Mobil Indonesia §  Provincial Education Department §  KNPI §  LSPTO §  BNSP §  Universitas Indonesia §  Local NGO §  Pemerintah Jakarta timur
2.	In-depth Interview	§  Kepala Dinas Pendidikan §  Kepala Suku dinas Jakarta Timur §  Bappeda DKI Jakarta §  DPRD DKI Jakarta §  Kepala Puslatdikjur §  Kepala Pelatihan §  Instruktur §  Peserta pelatihan
3.	Observation	§  Youth activities in autorepair shop or automotive industry. §  Tools, material, and facilties are used to supportthe training
4.	Call/Phone	§  Asking PPI to all alumni of traininng by using existing PPI’s form (10 questions).
5.	Document Review	§  All document, policies, budget, and report regarding to the project

 H.     Variables and indicators

 

The final evaluation is expected to generate answer to key assessment of final outcomes such as changes in attitudes / expectations, as well as measurement of receipt and levels of exposure to the project intervention, based on direct survey questions, checks through project and school records, and findings of children and community surveys.

A summary of the  project’s aspect, information need to know data type design are follows:

No.	Aspects	Information we need to know	Data Type
1.	Improve auto mechanic training environment and empowerment through modernize/ upgrade equipment	§ are workshop buildings  adequatefor automotive training practices ?(safe and comfortable) § are classrooms adequate for alearning / training? (safe andcomfortable) § are the dormitory rooms adequateto break/live in? (safe andcomfortable)   § are automotive equipmentsadequate / complete? (require the standards of the automotive training center)	§ Primary data/secondary data     § Primary/secondary data       § Primary/secondary data       § Primary/secondary data
2.	Provide quality education and link to job placement include enterprise	§ What # and % of enrolled youth finished the TVET training? § What # and % of enrolled youth finished the internship? §  What # and % finished MED training? § How many took the certification exam? § How many passed the certification exam? § How many graduates acquired work within one month of training? § How many of them received a formal contract? § What is the range of salary for youth who found jobs (for example, between 1.0 million-4.0 million rupiah)? § What is the average salary? Do these meet the minimum wage for their local area? § What kind of jobs did they take? How many are working for informal sector (small body shops) §  How many are working for formal sector (car making factories)? § How many are working in a sector outside of auto-mechanic? § How many are unemployed? What are the reasons that they are unemployed? § Where are they working? What percentage in Jabodetabek region? What % in Java and % outside Java? § What are the names of the employers, if big companies? § How many found work with Hyundai? § How many are still employed with decent work 3 months after graduation? “Decent work” means 1) have a contract or employer agreement, 2) receive benefits (like time off and insurance), 3) receive at least the minimum wage in their area. § What kind of entrepreneurial work did the MED graduates engaged in? § How far advantage of the business training project to motivate theyouth’s desire for business? § What is main factor that can cause willing   to do business? § Is automotive training methodologyin accordance with the spirit of youth? § Is time allocation enough for practice in workshop? § Changes in skill and attitude afterinternship in the auto-repair shop / industry. § Changes in skill after getting automotive training (specific skill, such as tune-up, spooring, balancing, putty, spraying/painting, etc) § Changes in self confidence /social communication of youth aftergetting life skill training. § Changes in attitude of youth after training / motivation / coaching. § Benefits / added value of dormitory  for youth. (economic value, social value, and culture) § How does extra-curricular activitiesboth physical and non-physical .(like sports, community service,religious, etc.) for developing the personality of youth? § How is the implementation of theprinciple of gender equality in thetraining? § How is the implementation of theprinciple of inclusion anddiscrimination in the training	§ Secondary data   § Primary/Secondary data   § Secondary data   § Secondary data   § Secondary data   § Secondary data     § Primary/Secondary data       § Secondary/primary data   § Secondary/primary data     § Secondary/ primary data           § Secondary /primary data     § Secondary data       § Secondary data   § primary data   § Primary data           § Secondary & Primary data   § Primary data       § Primary data   § Primary data   § Primary data   § Primary data     § Primary data   § Primary data     § Primary data         § Primary data     § Primary data   § Primary data       § Primary data           § Primary data       § Primary data
3.	Improve house income and decrease poverty index	§ What # and % of graduated youth who show an increase in living standards by the end of the project (increased PPI score of youth and family) with income allocated to their family.  *(Consultant has to ask PPI to all alumni of traininng by using existing PPI’s form. It can be conducted by phone). § What % salary that is used to meet their own needs? § What % of salary that is used tosupport a family? § What % of salary for savings?	§ Primary and secondary data         § Primary data   § Primary data   § Primary data
4.	Ensure that the project becomes sustainable and has the potential to scale up	§ Number of education office government budget allocated to thesupport project. (present) § Planning of number of education office government budget allocated to continue the project. (after the project ended in 2015) § Program planning of education office government  to continue the project. § What # automotive industry that support the project. Haw far partnership  of industry with the project?   (present) § Planning of partnership of industry to continue the project. . (after the project ended in 2015).	§ Secondary data     § Secondary /primary  data     § Secondary /primary data       § Secondary &primary data       § Secondary &primary data

 

I.       Scope of Work

The consultant is expected to form a survey team for the evaluation and manage coordination and communication with Plan Indonesia. Tasks and responsibilities of the consultant are as follows:

·           Proposal that detailing the methodology and budget required to conduct the final evaluation

·           Recruit, train and manage final evaluation research teams including enumerators

·           Propose tools to be used in the final evaluation research and discuss the suggestion with Plan Indonesia

·           Develop a data entry system and management

·           Supervise field research implementation

·           Responsible for field operations, including logistics, permission to conduct the research, informed consent from individuals and families taking part in the final evaluation research

·           Coordinate with YEE-Hyundai project during field research implementation

·           Collect, compile and analyse all data (gender and age disaggregated) gathered and develop a final report on the final evaluation results (including all findings and statistics).

·           To keep all information provided by Plan Indonesia as well as the findings of the Final Evaluation confidential.

 

J.      Outputs and Deliverables

The consultant is expected to produce and submit the following deliverables:

1)        Study protocol specifying a detailed study work plan and proposed study tools that will be discussed and agreed upon prior to field activity.

2)        Presentation of initial findings to YEE Team and PME, after field activities have been concluded.

3)        Draft reports in Bahasa and English in two weeks’ time after field activities have been concluded

4)        Presentation of draft and reporting data flow diagram to Plan Indonesia for discussion and feedback.

5)        Submission of a comprehensive final report in Bahasa and English one week after receiving feedback from Plan Indonesia. 

6)        Executive Summary (max. three to maximum five pages) in English describing the study results.

7)        Hardcopies of filled-in questionnaires (if available), interview transcripts and attendance lists, photos with informative subtitles and other valuable study materials.

K.         The Responsibility of Plan Indonesia

·             Provide a working contract

·             Provide data and related documents as per consultant’s request

·             Arrange meetings, discussions and field visits for the study team

·             Provide timely feedback and response on consultant’s study report

·             Settle payment for the consultant service

L.     Timeline

The final evaluation is to be conducted in November to December 2014, then the report should be finished the latest on January 2015.

 

M.    Report Content

Final report should be produced in Bahasa and English and should contain and be developed according to the following structure:

Acknowledgment

Executive Summary    (Having contents; Objectives, Methods, results & conclusion)

1.      Background [description of project and  activities]

2.      Literature review

3.      Methodology (having below mentioned components)

a.       Objectives of the study

b.      Operational definitions

c.       Study design

d.      Sampling Method and Sample Size

e.      Data Collection Procedure

f.        Data Analysis

g.      Significance / Importance of the study, Human subject protection

4.      Limitations

5.      Findings should be presented on study objectives covering following (CCCD principles, quality of design of the project, cost, reach, overall quality assessment) – the sections of the findings should be according to main objectives of the project

6.      Conclusions

7.      Recommendations           

8.      References

9.      Annexure [management response table[1], TOR, data collection tools, case studies…]   

Following lists must be given at the start;                                                                                         

1.      Table of contents (automatically generated)

2.      List of Figures

3.      List of Tables

4.      Acronyms and abbreviations

 

N.    Ethical and child protection consideration

The Evaluator must follow Ethical Principles for involving human subjects in a research and obtain written/ verbal consent from the human subjects. Permission from elder must be sought if the children under 18 years are involved as subjects. Signed informed consent of each child and his/ her parents need to take after explaining purpose of the study and its usage.

O.    Criteria for Consultant Selection Based on Competency and Experience

Consultant interested in submitting a proposal should have the following criteria:

1.      Possess equal composition of qualified academic background, knowledge, experience and capacity to manage evaluation.

2.      Have an extensive experience in managing large-size studies in the context of basic automechanic skill.

3.      Demonstrate understanding and sensitivity on cross-cultural, tradition and language of project target.

4.      Proven experience as an evaluator, preferably on automotive work.

5.      Understanding of project management structures and change management concepts

6.      Clear, effective writer in English.

7.      Experience of working with participatory methodologies.

 

P.      Procedure for Expression of Interest

First Phase: Interested consultant should submit an Expression of Interest to Plan Indonesia at: HRD.Indonesia@plan-international.org before 5 pm on 2 November2014, enclosing the following required documents:

1.            Organizational Profile (for institutional consultant) or a CV (for individual consultant).

2.            A proposal containing two parts:

·         Technical Proposal comprising the description of proposed activities, methodology (sampling design, data collection technique), and implementation plan which includes study schedule and timeline for data analysis. The technical proposal should demonstrate understanding of study ethic, qualification of study team members and their CVs, as well as their roles in the study. The consultant should also describe her/his experience in conducting similar survey, and financial and technical capacity.

·         Budget Proposal comprising details of budget plan such as consultant and team’s fees, transportation, stationeries and other survey necessities.

Second Phase: Based on received proposals, Plan International Indonesia will shortlist candidates and invite them to present their proposals. Plan International Indonesia will afterwards proceed with selecting and appointing a consultant, and produce a working contract with the select consultant.

Only shortlisted consultant will be contacted for recruitment.