Terms of Reference for a Real Time Evaluation of the IFRC

Terms of Reference for a Real Time Evaluation of the IFRC response to the North Africa / Middle East crisis 2011

1. Summary

1.1. Purpose: This real time evaluation (RTE) will assess the ongoing IFRC response to the North Africa / Middle East crisis to inform continued response and preparedness in MENA, as well as future global emergency relief operations. Particular emphasis should be placed on overall management and performance of the response and related preparedness actions, and how effective they were / are in delivering services throughout the ongoing response.

1.2. Commissioners: This RTE has been commissioned by the USG of the Programme Services Division, IFRC, Geneva.

1.3. Audience: This RTE will be used by the IFRC in MENA Zone and IFRC headquarters in Geneva; it will also inform all RC/RC actors contributing to or affected by the operation and the wider International Federation in future disaster response.

1.4. Duration of consultancy: approximately 30 days (with approx 15 days in the field)

1.5. Estimated dates of consultancy: June - July, 2011.

1.6. Location of consultancy: Geneva, Amman, Egypt, Tunisia and other relevant countries in the MENA Zone.

2. Background

The IFRC is committed to ensuring quality assurance, standards and a strong culture of lesson learning in its disaster response and, as such is committed to carrying out RTEs in the wake of all major disasters requiring an international response and meeting certain criteria. Effective RTEs can identify key systems, processes, issues and lessons in a timely manner that directly improves service delivery and are therefore a key mechanism for accountability to intended beneficiaries, donors and other stakeholders supporting IFRC work.

Since January 2011, the MENA region has witnessed weeks of civil unrest in a number of different countries. This has provoked concern related to the insecurity for the population in each specific country and also to ongoing population movement, primarily from Libya into neighbouring countries. The political conditions and rapidly changing events in Libya have caused around 850,000 people to flee Libya, the majority into Tunisia and Egypt, and there is no immediate end in sight to this situation. In addition, other countries, such as Yemen and Syria have experienced political unrest and confrontation with existing governmental systems. The Red Cross Red Crescent Movement is working together to support the National Societies in these countries, to respond to and/or prepare for escalating tensions and humanitarian needs.

• The IFRC has responded in Tunisia by supporting the Tunisia Red Crescent Society (TRCS) to provide medical / first aid assistance, food, non-food items, water & sanitation and some psychosocial support. With support from a 14-person FACT and four ERUs, the RC has set up a transit camp for up to 2,000 persons to provide support to key groups, such as sub-Saharan Africans. There has also been support to build the capacity of the TRCS. A Plan of Action has been developed to outline the strategy and contingency planning for the operation in Tunisia. There is also a strong focus on building the capacity of the TRCS to be better prepared for future disasters. This plan will focus mainly on: logistics system, volunteer management and disaster management programming.

• In Egypt, the Egyptian Red Crescent Society (ERCS) has been working closely with the Government and other organizations to provide support to those crossing the border from Libya. ERCS volunteers have provided relief assistance (food and non-food items), water and first aid / basic medical care to those in transit or remaining in the border area of Saloum.

• In Libya, meetings have been held with the LRCS and a plan of action has been drawn up to provide key assistance and capacity building / organisational development support to the NS.

• In both Syria and Yemen, the Zone has carried out assessment missions and in Yemen has provided additional human resources (DM and PSP specialists) who have provided training and technical advice. Assistance is being provided to both NSs to support the targeting and delivery of relief items / logistics systems, in first aid and in other emergency health services in coordination with the ICRC.

• At a regional level, the MENA Zone is supporting National Societies in the three sub-regions to build or reinforce contingency planning at a national and regional level to ensure that they can provide effective emergency response in a timely, neutral and impartial manner. This includes the provision of technical support, training and the planning for pre-positioned stocks.

The IFRC launched a Preliminary Emergency Appeal (MDR82001) on 1 March 2011 for CHF 4,438,090 to support up to 100,000 beneficiaries in the affected countries, in particular in support of the population movement from Libya into Tunisia and for contingency planning across the region. In addition, three DREF grants were (CHF 150,000 was allocated directly to Tunisian Red Crescent, CHF 107,672 to Egyptian Red Crescent, and CHF 59,374 to region for the start up of the appeal). A Revised Emergency Appeal was launched on 24 March 2011 for CHF 12,269,102 to extend to operation in Tunisia and support for regional contingency planning in countries where the risk of humanitarian crises is increasing and again 17 May for CHF 14,840,345 to include support to the Yemen and Syrian Arab Red Crescent Societies.

3. Evaluation Purpose & Scope

The primary purpose of this real time evaluation (RTE) is to assess the ongoing IFRC response to the North Africa and Middle East crisis to inform continued response and preparedness in the MENA region, as well as future global emergency relief operations. Particular emphasis should be placed on the overall management and performance of the response and related preparedness actions, and how effective they were / are in delivering services throughout the ongoing response. This should focus according to the objectives and activities outlined in Emergency Appeal.

The RTE will look at the appropriateness, timeliness and effectiveness of preparedness and response planning and actions carried out for the affected countries in North Africa and subsequently for the wider MENA region. The RTE will consider the "IFRC response" to include the NSs affected by the current crisis, the Red Cross / Red Crescent NSs supporting the operation within the IFRC Secretariat system and the Secretariat at all levels (country, regional, zone, and Geneva). The RTE will look at links to ICRC activities, but will refrain from pronouncing on those activities. In addition to these RCRC stakeholders and the intended beneficiaries of RCRC services, the RTE may consult with any other key stakeholder (partner organizations, governmental, etc) relevant to the evaluation objectives.

Geographically, the scope of the evaluation will include the affected countries included in the Emergency Appeal: - the countries of North Africa (particularly Tunisia and Egypt), of the Middle East (particularly Yemen and Syria) and those NSs covered by relevant preparedness actions (including Bahrain). It will take into account the work done on regional contingency planning and the support from the Regional Logistics Unit in Dubai. The evaluation will cover the period from the outset in January until the time the evaluators collect the data.

4. Evaluation Objectives and Key Questions

Specific objectives and key questions to be answered in this RTE are listed below. In addressing these objectives and key questions particular emphasis should be placed on considering the performance and delivery of the IFRC Secretariat and the wider IFRC in both the response to and preparedness for the developing crisis. Please note that these are guiding questions and the RTE team are not limited to those below.

1. Efficiency & Effectiveness of operational management: to what extent have internal systems, structures and mechanisms affected the management and timely and cost-effect service delivery?

a. Was the Federation' operational structure, internal systems and processes well geared to deliver timely, efficient and effective disaster response in an equitable manner, proportionate to need?

b. How effective and efficient were the systems to mobilize resources – financial resources / fundraising (including DREF, PEAR, ECHO etc), human resources (including surge capacity and delegate recruitment), logistics etc.? Was resourcing adequate and what could have been done differently? How effective and relevant were the initial needs assessment? Who was involved, and what methodologies / tools were used? What (if any) should have been done differently?

c. How did the assessment(s) link to planning? Were plans / emergency appeal timely and relevant in each situation?

d. How effectively were plans turned into relevant and timely programming or assistance and how effectively was this assistance delivered? What constraints were there? How could the delivery of programmes or assistance have been improved and what lessons are to be learnt?

e. What levels of beneficiary consultation were used during the assessment, planning, implementation and monitoring stages?

f. How effective were the systems to provide programme and management reporting? And how timely, relevant and informative was both management and public reporting?

2. Coordination: to what extent the IFRC response was managed in a cohesive and effective manner, including communication, collaboration, and coordination among key stakeholders:

a. How timely and effectively has the IFRC internally communicated and coordinated in its response within the Secretariat – including country, region, zone, and Geneva? Coordination and communication covers all stages from initial assessment to planning and service delivery)

b. How timely and effectively has the IFRC communicated and coordinated its response with Movement partners? And with external actors?

c. Were the resources present to support and ensure effective coordination and communication at the necessary levels?

d. Was coordination between levels and across programmes effective and transparent?

3. Preparedness: Contingency planning and basic response preparedness is an important element of this slowly evolving and changing crisis across the MENA region and there should be a focus on different levels of preparedness, including country, regional and Zone level. To what extent were contingency planning and basic response preparedness measures planned and carried out in an effective, efficient, relevant and appropriate manner - at country, regional and Zone level.

a. What contingency plans and early warning systems were in place before the crisis (or were developed in the early stages of the crisis) to prepare NSs and Secretariat teams?

b. How was the early warning and monitoring of the ongoing situation carried out? How timely and effective were the steps taken to put in place contingency plans for emerging crises and how effective were these plans?

c. What steps were taken to ensure a coherent approach across the MENA Zone towards contingency planning and response preparedness measures? Were global contingency planning guidelines and technical support used and were they effective? What level of contingency planning information was shared with National Societies and other partners?

d. What risk management was taken into account?

e. What steps or recommendations would be made to improve or reinforce the regional contingency planning process?

5. Evaluation Methodology & Process

The methodology will adhere to the draft IFRC Framework for Evaluations , with particular attention to the processes upholding the standards of how evaluations should be planned, managed, conducted, and utilized.

An IFRC evaluation management team will oversee the evaluation and, with the evaluators, ensure that it upholds the IFRC Management Policy for Evaluation. The evaluation management team will consist of three people not directly involved with the MENA operation; one of which is from the Secretariat Planning and Evaluation Department, the other two who have direct experience in emergency operations and assessments – one from the Zone and one from Programme Services Division.

The evaluation team will consist of two – three people: one an external evaluator and one-two internal IFRC evaluators (ideally all candidates should have regional experience and Arabic language skills). The external evaluator will provide an independent, objective perspective as well as technical experience to the evaluation, and will be the primary author of the evaluation report. S/he should not have been involved or have a vested interest in the IFRC operation being evaluated, and will be hired through a transparent recruitment process, based on his/her professional experience, competence, ethics and integrity for this evaluation.

The internal evaluator(s) should provide sound knowledge, understanding and experience of IFRC disaster response. They will assist the external evaluator in the evaluation process, and to best interact with the various RCRC actors involved in the operation. It is expected that this two-three person team will be able to conduct a reliable and informed evaluation of the emergency operation that has legitimacy and credibility with stakeholders.

The specific evaluation methodology will be detailed in close consultation between the RTE team and IFRC, but will draw upon the following primary methods:

1. Desktop review of operation background documents, relevant organizational background and history, including prior IFRC RTE evaluation reports, and any relevant sources of secondary data, such exist surveys from IFRC participants in the operation.

2. Field visits/observations to selected sites and to the Zone office.

3. Key informant interviews (institutional and beneficiaries as appropriate).

4. Focus group discussions, (institutional and beneficiaries as appropriate) as time and capacity allow.

An initial draft report will be prepared for a review process. This process should occur within 3 weeks of submitting the draft report to the evaluation management team, and will involve the following stakeholders in the following order:

• Week 1 of review: the evaluation management team to check content is in line with this TOR and IFRC evaluation standards.

• Week 2-3 of review: stakeholders who participated in the evaluation for feedback on inaccuracies or clarifications (differences of opinion would not be put forward here but outlined in the management response). Following this, a final draft is prepared

• Week 4 of review: an evaluation management response team (to be determined, but consisting of key relevant stakeholders from within the IFRC) will review the report and compile a management response to be included as an appendix to the final RTE report.

The draft IFRC Real-time Evaluation Management Guide will be piloted for this RTE and that the report review process and development of management response will be undertaken as described in the draft guide, to be made available to the evaluation team.

6. Evaluation Deliverables

Inception Report – The inception report will be a scoping exercise for the RTE and will include the proposed methodologies, data collection and reporting plans with draft data collection tools such as interview guides, the allocation of roles and responsibilities within the team, a timeframe with firm dates for deliverables, and the travel and logistical arrangements for the team.

Debriefings / feedback to management at all levels – The team will report its preliminary findings to the IFRC in Tunisia, Amman (Zone Office) and in Geneva, in a timely manner and will adhere to the above mentioned review process.

Draft report – A draft report, identifying key findings, conclusions, recommendations and lessons for the current and future operation, will be submitted by the team leader within two weeks of the evaluation team's return from the field.

Final report – The final report will contain a short executive summary (no more than 1,000 words) and a main body of the report (no more than 10,000 words) covering the background of the intervention evaluated, a description of the evaluation methods and limitations, findings, conclusions, lessons learned, clear recommendations. Recommendations should be specific and feasible. The report should also contain appropriate appendices, including a copy of the ToR, cited resources or bibliography, a list of those interviewed and any other relevant materials. The final RTE report will be submitted one week after receipt of the consolidated feedback from IFRC.

All products arising from this evaluation will be owned by the IFRC. The evaluators will not be allowed, without prior authorization in writing, to present any of the analytical results as his / her own work or to make use of the evaluation results for private publication purposes.

7. Evaluation Quality & Ethical Standards

The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of the people and communities involved and to ensure that the evaluation is technically accurate and reliable, is conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. Therefore, the evaluation team should adhere to the evaluation standards and applicable practices outlined in the IFRC Management Policy for Evaluation.

The IFRC evaluation standards are:

1. Utility: Evaluations must be useful and used.

2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost effective manner.

3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with particular regard for the welfare of those involved in and affected by the evaluation.

4. Impartiality & Independence; Evaluations should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders.

5. Transparency: Evaluation activities should reflect an attitude of openness and transparency.

6. Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.

7. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate.

8. Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation.

It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality. Further information can be obtained about these Principles at: www.ifrc.org/what/values/principles/index.asp

9. Qualifications

Selection of the external evaluation consultant will be based on the following qualifications:

1. Demonstrable experience in leading evaluations of humanitarian programs responding to major disasters, with specific experience in RTEs preferred;

2. Knowledge of strategic and operational management of humanitarian operations and proven ability to provide strategic recommendations to key stakeholders;

3. Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner;

4. Experience in qualitative data collection and data analysis techniques, especially in emergency operations;

5. Knowledge and experience working with the Red Cross Red Crescent Movement preferred;

6. Demonstrated capacity to work both independently and as part of a team;

7. Excellent English writing and presentation skills in English, with relevant writing samples of similar evaluation reports.

8. Knowledge of the MENA region required

9. Ability to communicate in Arabic preferred.

10. Minimum qualification of a master's degree or equivalent combination of education and relevant work experience.

11. Immediate availability for the period indicated.

9. Application Procedures

Interested candidates should submit their application material by 31st May 2011 to the following email: misgana.ghebreberhan@ifrc.org. Application material is non-returnable, and we thank you in advance for understanding that only short-listed candidates will be contacted for the next step in the application process.

Application materials should include:

1. Curriculum Vitae (CV)

2. Cover letter clearly summarizing your experience as it pertains to this RTE, your daily rate, and three professional references.

3. At least one example of an evaluation report most similar to that described in this TOR.