ACTED Indonesia is seeking evaluators interested in conducting the External Evaluation of the Consolidation of the Livelihoods Recovery Programme on Nias.
The selected evaluators or institution will be responsible as following terms of reference and desired qualifications:
Terms of Reference for External Evaluation of the
Consolidation of the Livelihoods Recovery Programme on Nias
1. Introduction
1.1.Rationale:
The Canadian Red Cross (CRC)-funded project, "Consolidation of Livelihood Program on Nias" was implemented by ACTED in three sub-districts of the northern part of Nias Island was the second phase of livelihood intervention in 24 communities of fishermen and farmers. The project activities will end as of 30 November, 2011 and thus a final evaluation is required to:
. Identify the outcomes against targets of project implementation in the targeted area.
. Evaluate achievements of stated objectives and results of the program, their relevance, and potential sustainability.
1.2.Purposes:
a. To assess to what extent the program contributed to the overall objective, "Improved livelihood sustainability for the inhabitants of 24 villages in North Nias District,[1] Nias."
b. To measure to what extent the program achieved the specific objectives:
. Increased sustainability of effective local governance and support on village development initiatives in the 24 target villages;
. Increased sustainability of effective Self- Help Groups and successful livelihoods business projects implemented in 24 target villages;
. Increased livelihood opportunity and sustainability for fishery groups and marine carpenters of 24 target villages.
c. To assess the potential for project results to be sustainable after project completion.
d. To identify which methods and approaches used by ACTED have been the most successful in improving livelihood sustainability.
e. To document lessons learned and best practices developed during the project.
f. To assess CRC visibility during the project.
g. To document lessons learned about the partnership between CRC and ACTED.
1.3. Estimated dates: 1st to 30th of November 2011
1.4. Location of consultancy: Indonesia (Jakarta, Gunung Sitoli and North Nias)
1.5.Users and Intended Use:
Users
Intended Use
CRC Management and ACTED Coordination
CRC Management
1. To evaluate the results of ACTED's CRC-funded project in Nias.
2. To assess CRC's visibility throughout the project.
3. To document and collect good practices, approaches and tools related to beneficiary engagement and community mobilization for internal and external knowledge sharing.
4. To gain lessons learned for future project development and implementation.
5. To contribute to CRC policy on partnerships with INGOs.
2. Background of the Program/Project
ACTED implemented a CRC-funded livelihoods recovery project in 26 villages in Lahewa district from January 2008 to December 2009 as part of the post-disaster response to the 2004 tsunami and 2005 earthquake that devastated the island. This project contributed to the restoration and development of the agricultural and fishery sectors, while supporting good local governance through capacity-building work with village-level governance structures in the target area. Project activities inlcuded the mobilisation of Self-Help Groups (SHGs), agricultural and fishery technical training, asset (seeds, tools, and boats) distribution, as well as technical training for members of BPDs (village parliaments). It was clear to ACTED and CRC however, that further activities were needed to consolidate the livelihoods recovery to sustainably improve the welfare of beneficiaries; thus this became the first phase of two-phase project and CRC funded ACTED to implement a second intervention.
The second phase project continued capacity-building activities targeting rubber, cacao, vegetables, and fisheries in 24 villages, but the focus of the 160 Self-Help Groups involving 3,900 beneficiaries shifted from basic production to post-harvest processing and marketing to increase household income and livelihood opportunities for the SHGs and their members. ACTED provided business and post-harvest training to the SHG groups to support them in developing business proposals, which were included in applications for small grants provided by the project for post-harvest processing and savings and loan activities. Governance activities continued capacity-building of BPD members at the village level, and strengthened networking among villages with the establishment of three sub-district forums to improve transparency and accountability through information sharing. Training was also provided to marine carpenters in Nias to provide sustainable improvement to asset production for the fishing sector; this training produced 26 additional boats that were distributed to fishery SHGs targeted during both phases of the project to increase their asset-to-member-ratio.
3. Evaluation Scope
Scope of this evaluation:
1. Sectoral scope: local governance capacity-building, livelihoods (fishing and farming), and community mobilization (Self-Help Groups).
2. Scope of evaluation criteria: effectiveness, sustainability, partnership between ACTED and CRC and coordination between ACTED and government and relevant local authorities.
3. Scope of stakeholder consultation: local administrative authorities (district, sub-district, village), local technical authorities (Agriculture, Fisheries, and Cooperative Departments), financial stakeholders, and beneficiaries.
Cross-Cutting Themes to be considered in the Evaluation:
Area of Enquiry
Guidance Notes
Gender Equity
The extent to which gender equity issues were integrated throughout the project management cycle in line with the gender policy or strategy of CRC and implementing partners
Capacity Building
The extent to which capacity-building of the targeted communities is done and its positive and negative effects. This could be part of sustainability evaluation criteria.
4. Scope on stakeholders:
a) Beneficiaries: agriculture and fishery Self-Help Group members, household representatives;
b) Village level: Head of Villages, Head of LPM (Village Development Institution), BPDs (Village Parliament), Self Help Groups, any relevant local Community Based Organizations;
c) Sub-district level: Camat secretary, BPD forum members;
d) District level: Bupati, Head of Dinas Kelautan, Head of Dinas Pertanian, Head of Dinas Koperasi, BPD association, financial institutions (banks, INGOs, Bapeda);
5. Scope of program sites:
. Three sub-districts in Northern Nias District: Afulu, Lahewa and Lahewa Timur;
. 25 villages (24 Phase II intervention villages, and 1 Phase I intervention village where SHGs received boats distributed during Phase II);
. Meulaboh, North Sumatra (marine carpenter training and boat-building activities took place here, but marine carpenters will be interviewed in Nias);
4. Evaluation Criteria and Key Evaluation Questions
Effectiveness:
. To what extent were the project's expected outcomes achieved?
. Were the activities carried out as originally planned?
. Were there any unforeseen negative or positive side-effects?
. Did ACTED make the right and timely adaptations in response to the changes in the project context?
. To what extent were the overall objectives achieved?
Partnership and Coordination:
. How effective was coordination with local authorities?
. Were all relevant stakeholders involved?
. What worked and what didn't work with the partnership between CRC and ACTED, in supporting the achievement of expected objectives?
. Compliance with the Code of Conduct as per #29 of CRC's Agreement with ACTED (the Red Cross Supplier Code of Conduct)
Sustainability:
. Is the project environmentally and financially sustainable?
. What were the major factors which influenced the achievement or non-achievement of sustainability of the project structures and results?
. To what extent did beneficiaries take ownership of the project?
5. Evaluation Methodology and Process
Evaluation management team will be comprised of ACTED Coordination and ACTED's Appraisal, Monitoring, and Evaluation Unit (AMEU), with additional guidance provided by CRC management staff. The management team will oversee implementation evaluation activities by the consultant to ensure quality and efficiency. ACTED and CRC management will make a decision on the final ToR, selection of the consultant evaluator, evaluation matrix, tools and methodologies, and the final report.
The evaluation will be conducted by an external national consultancy team,. They will be responsible for the overall evaluation from design, preparation of tools, data collection, coordination with relevant parties, and finalising the report (to be revised accordingly).
The specific review of methodology will be detailed in close consultation with the management team, but will be drawn upon following primary methods:
. Desktop review of project proposal, interim reports, relevant organizational background and history, and any relevant sources of secondary data;
. ACTED end line survey, which will be compared with the initial baseline and includes socio-economic data. Survey questionnaire implementation and data entry into the database will be conducted by ACTED staff; the consultant will be in charge of analyzing the data and producing a comparison report;
. As part the quality control of CRC, the end-line survey methodology will be reviewed by an external consultant to provide understanding of the level of confidence can be given to the findings. This step does not require the ACTED to change its ongoing data collection. The external consultant will be hired by CRC-Ottawa and ACTED will submit a description of the methodology (e.g., how sample size is calculated, etc.). Depending on the methodology, an external review may also be done on the statistical analysis carried out by the national consultant.
. Qualitative evaluation to answer the evaluation purposes, which should include site visits and interviews with relevant stakeholders;
. A briefing at the beginning of the evaluation and a debriefing at the end of data collection with ACTED and CRC coordination and/or AMEU;
Key milestones of the evaluation are
11 November : Submission of an inception report
9 December : Submission of first draft report
6 January : Submission of final report
The evaluation process will ensure stakeholder input while maintaining the integrity and independence of the evaluation according to the following guidelines:
. Inaccuracy: Inaccuracies are factual, supported with undisputable evidence, and therefore should be corrected in the evaluation report itself.
. Clarifications: A clarification is additional, explanatory information to what the evaluators provided in the report. It is the evaluators' decision whether to revise their report according to a clarification; if not, the evaluation management response team can decide whether to include the clarification in their management response.
. Difference of opinion. A difference of opinion does not pertain to the findings (which are factual), but to the conclusions and/or recommendations. These may be expressed to the evaluators during the evaluation process. It is the evaluators' decision whether to revise their report according to a difference of opinion; if not, the evaluation management response team can decide whether to include the clarification in their management response.
6. Evaluation Deliverables
Inception Report - Once the consultant is appointed, he or she should submit the inception report within one to two weeks upon signing contract. The inception report will include the proposed methodologies, data collection and reporting plans with draft data collection tools such as interview guides, a timeframe with firm dates for deliverables, and the travel and logistical arrangements for the team. This inception will be reviewed by the Evaluation Management Team.
The First Draft report - A draft report, to the Evaluation Management Team, consolidating findings from the end line survey and evaluation, identifying key findings, conclusions, recommendations and lessons for the current and future similar program, will be submitted within two weeks of the evaluation team' completion of data collection in Nias. The consolidated feedback from the Evaluation Management Team will be sent to the evaluator two weeks after submission of this first draft.
Final report - The final report will contain a short executive summary (no more than 2500 words) and a main body of the report (no more than 60 pages) covering the background of the intervention evaluated, a description of the evaluation methods and limitations, findings (to be presented by evaluation criteria), conclusions, lessons learned, clear recommendations. Recommendations should outline recommendations that the project staff and the reviewer(s) have in common or different views based on the workshop to discuss the findings. The report should also contain appropriate appendices, including a copy of the ToR, cited resources or bibliography, a list of those interviewed and any other relevant materials (e.g., tools). The final report will be submitted 2 week after receipt of the consolidated feedback from the Evaluation Management Team.
7. Evaluation Quality & Ethical Standards
The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of the people and communities involved and to ensure that the evaluation is technically accurate and reliable, is conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. The evaluator will sign and adhere to the Canadian Red Cross Code of Conduct.
The evaluation standards are:
1. Utility: Evaluations must be useful and used.
2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost effective manner.
3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with particular regard for the welfare of those involved in and affected by the evaluation.
4. Impartiality & Independence; Evaluations should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders.
5. Transparency: Evaluation activities should reflect an attitude of openness and transparency.
6. Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.
7. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate.
8. Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation.
It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality. Further information can be obtained about these Principles at: www.ifrc.org/what/values/principles/index.asp
8. Qualifications
Selection of the external evaluation consultant will be based on the following qualifications:
1. Must be based in Indonesia.
2. Demonstrable experience in conducting similar project final evaluations.
3. Good understanding of local governance and livelihoods issues in rural Indonesia.
4. Experience in a large-scale household surveys and qualitative data collection as well as data analysis techniques; consultant must be capable of producing data analysis report, or include clear sub-contracting of this activity in the proposal.
5. Strong analytical skills and ability to clearly synthesize and present findings in English, draw practical conclusions, make recommendations, and prepare well-written reports in a timely manner.
6. Knowledge and experience working with INGOs.
7. Excellent English writing and presentation skills, with relevant writing samples of similar evaluation reports upon request.
8. Immediate availability for the period indicated.
9. Application Procedures
Interested parties should send a maximum two-page EOI indicating their:
. Experience in project evaluation (required) and governance/livelihood issues in Indonesia (desirable);
. Proposed methodology or tools in conducting this assignment;
. Proposed timeline with clear indication of number of billable days;
. Proposed daily rate inclusive of meal and per diem costs (excluding field transportation and accommodation, which will be provided by ACTED);
. Sample of previous work (project evaluation).
To indra.yani@acted.org & indonesia@acted.org before 4 November 2011 and at 17.00 WIB
Application material is non-returnable, and we thank you in advance for understanding that only short-listed candidates will be contacted for the next step in the application process.
No comments:
Post a Comment