Mercy
Corps Somalia
Terms of Reference
School Environment and Education Development for Somalia (SEEDS) Program
Job Title: Consultant as Final Program Evaluator
Length of Consultancy: 30 Days (Exact dates to be determined during
the signing of the contract)Terms of Reference
School Environment and Education Development for Somalia (SEEDS) Program
Job Title: Consultant as Final Program Evaluator
1. Background and Context
School Environment and Education Development for Somalia (SEEDS) is a USAID funded Education Program that was initially designed for three years (from 30th September 2008 to 29th September 2011).
The program was jointly designed by a consortium consisting of Mercy Corps (as the prime), Centre for British Teachers (CfBT) Education Trust, African Rescue Committee (AFREC), International Medical Corps (IMC) and International Aid Services (IAS). IMC and IAS dropped off the consortium leaving Mercy Corps, CfBT and AFREC in the consortium.
The goal of the SEEDS program is to achieve improved access to basic education and water services in Somalia.
The program was intended to achieve improved access to basic education, water services and sanitation in school communities in Somalia.
It adopted a holistic approach to improving education environment; encompassing infrastructure development, supplies provision, as well as capacity development for school management that targets CECs, teachers and education officials.
The focus has been to create more learning spaces for children to expand the availability of basic education opportunities through construction of new and rehabilitation of existing classrooms, and water and sanitation infrastructure, in-service training of teachers, and improving school management and provision of technical assistance to MOE and other government officials.
In the course of implementation, two major cost modifications were approved. The first modification, two years (September 2010) after the start of the program expanded the scope of the program to include Maternal and Child Health Care.
The second modification towards the end (July 2011) of the implementation, expanded the scope of the program to include secondary education and livelihood opportunities and the modification extended the life of the program to 21st July 2012.
Program Objectives:
a) Primary Education
Objective 1: Pupils have improved access to basic (Primary) education
Objective 2: People in school communities have access to safe water and basic sanitation
Objective 3: Improve teaching and learning in about 100 schools by the end of the project
Objective 4: Improve management in about 100 school communities at the end of the project
Objective 5: Improve institutional capacity of MoE
b) Maternal and Child Health
Objective 6: Improve maternal and child healthcare in Somaliland
c) Secondary Education
Objective 1: Improve access to and equity of secondary education
Objective 2: Improve delivery of secondary education services
Objective 3: Improving capacity of MoE to manage education system
Objective 4: Support creation of education and economic opportunities for in-school youth
Since the Secondary Education has only been implemented for a very short time, this component will not be included in this final program evaluation.
At the end of SEEDS, all the secondary interventions will be transferred to the new Somali Youth Leaders Initiative (SYLI), another Mercy Corps program. The outcomes and impact of the Secondary Education activities will therefore be captured during the evaluation of SYLI.
3) Purpose of the Final Evaluation
The final evaluation is meant to take stock of SEEDS’ achievements over the past three and a half year period of implementation. The evaluation will review activities and analyze the extent to which outputs and outcomes have been met.
The specific purpose of the final evaluation is to carry out a process evaluation of SEEDS to establish the extent to which the program has achieved its stated objectives, milestones, and targets which were set for the life of the program.
Evaluation objectives
In terms of specific objectives the evaluation will focus:
a) To evaluate whether program objectives were achieved with high quality outputs and accountability to beneficiaries;
b) To conduct an in-depth qualitative evaluation of the effects of the program on selected Mercy Corps school communities in comparison with other communities; and
c) To identify and document best practices, lessons learned for future programming (examples of potential best practices include: monitoring of school construction processes by the CEC community led tendering processes, and CEC led school improvement plans. Lessons learned to be explored include working with the local partners in reaching wider and non-secure areas, quality of program outputs achieved through the local partners’ implementation, and use of local resource persons in facilitating trainings).
In addition to the above evaluation objectives, the evaluation will also cover the following aspects of the program.
a) Relevance: the extent to which objectives, implementation strategies, activities and methodologies were adapted to the needs of the beneficiaries.
b) Effectiveness: the extent to which SEEDS has done what it was intended to do for the beneficiaries.
c) Efficiency: the results achieved in relation to time, efforts and resources expended.
d) Sustainability: the extent to which results achieved can continue after the end of the program.
e) Impact: the overall effects SEEDS has had on the beneficiaries and participating communities and on their methods of working in the future.
f) Ownership: the level of stakeholder and community participation/involvement and their feeling of who controls and makes what decisions.
4) General Evaluation Questions
Primary Education:
i. How has access to primary education improved within the SEEDS target areas and what factors have influenced this? Is this sustainable?
ii. How has the quality of primary education improved within the SEEDS program target areas and what factors have influenced this? Is this sustainable?
iii. How has the management of primary education system been improved within the SEEDS target areas?
iv. How has the community participation in the management of primary schools improved within the SEEDS program area?
Maternal and Child Health Care:
i. How has access to maternal and child health care services improved within the SEEDS target areas?
ii. How has the quantity and quality of maternal and child health care services changed/improved within the SEEDS program area
iii. How has the management of maternal and child health care system improved within the SEEDS target areas?
iv. How has the community (men and women) participation in the management of Primary Health Care facilities changed/improved within the SEEDS program area
5) Evaluation Methods
The process will be participatory and the evaluation will employ a combination of different quantitative and qualitative methods.
In addressing evaluation Objective 1, the consultancy should systematically assess program outputs and their contribution to achieving objectives in a reasonable sample size of school and health communities considering timeline and security risks.
In addressing Objective 2, the evaluation should take a qualitative participatory impact assessment approach. Key school and health communities should be chosen and an in depth evaluation of program impact done in those communities across the 6 program objectives in order to address the link between achieving the program objectives and goal. A school and health community without Mercy Corps support in each zone should also be studied in order to make a comparative assessment.
The specific evaluation methods are outlined below.
a) Conduct a desk review including an analysis of documents, policy papers, national surveys and reports that have been produced in connection with the project thematic area. Review internal Mercy Corps documents related to the SEEDS program including data already gathered.
b) Semi structured key informant interviews with respective Government Ministries, CSOs, NGOs, in the selected SEEDS program area.
c) Focus group discussions with various community groups; CECs and health facility management committees.
d) Structured individual/household survey interviews with sampled community members.
6) Existing Sources of Program Information
- SEEDS program
documents including log frame and performance monitoring plan (PMP)
- The program
quarterly and annual reports
- Baseline survey
report (education and health baseline survey reports)
- Mid-term review
report
7)
Team Composition and Participation
The Evaluation team will consist of:
The Evaluation team will consist of:
- External
Consultant, as lead, with experience in evaluation of education and
maternal and child healthcare programs
- Mercy Corps
M&E Specialists
- Technical
Specialists from the Consortium; CfBT Education Trust and African Rescue
Committee
- Mercy Corps
Sector Specialists Education and Infrastructure Manager
- Government
Ministry representatives (Ministry of Education and Ministry of Health)
The
key responsibilities of the team are highlighted below
The external consultant:
The external consultant:
- Will lead the
evaluation team and will be expected to train selected research assistants
who will assist in primary data collection (facilitation of focus group
discussions and the administration of individual interview questionnaire).
- Conduct the desk
research/review of relevant literature.
- Develop the
assessment tools which include but not limited to semi-structured
questionnaires for key informant interviews, focus group discussion guide,
structured questionnaires for the individual interview and tools for
collecting quantitative data from the Secondary schools and health
facilities.
- Conduct key
informant interviews with the respective Government Officials, other
community leaders, CSO and NGO representatives.
- Facilitate focus
group discussions with the various groups.
- Conduct
survey/collect quantitative data from the sampled community members.
- Key in data,
analyze, and write the first draft report for initial review by team
members listed above.
- Incorporate the
comments from the team members, prepare and present the second version of
the draft report to the Stakeholders in Somalia.
- Incorporate the
feedback from the stakeholders and prepare the final report.
Mercy
Corps M&E Specialist will take the overall coordination role of the evaluation
as outlined below:
- Coordinate with
the other team members in the review of the evaluation tools developed by
the consultant and provide a consolidated feedback to the consultant.
- Review the final
version of the evaluation tools and ensure that the consultant has
adequately incorporated the feedback from the all the team members.
- Coordinate with
the team in the review of the draft final evaluation report and provide a
consolidated feedback to the consultant.
Technical
Sector Specialists (Mercy Corps, Consortium members and Government line
Ministries)
These will provide technical backstopping to the Assessment Team with regard to their respective sectors as follows:
These will provide technical backstopping to the Assessment Team with regard to their respective sectors as follows:
- Review the
assessment tools developed by the consultant to determine their
suitability in measuring the sector specific indicators.
- Mobilize/Coordinate
the key informant interviews with the line Government departments and
other target informants.
- Review and
provide feedback on the draft final evaluation report prepared by the
Consultant.
8)
Duration of the Evaluation
Mercy Corps and the consortium members expect that the entire evaluation process will cover a period of 30 days during the months of May and June 2012.
9) Timetable
The Consultant will develop a draft timetable for the assessment, which will be discussed with the Chief of Party and M&E Specialist for comments and approval before the commencement of work.
10) Evaluation Report Format
The final evaluation report format will be agreed with the consultant and CoP/M&E Specialist upon signing of the contract with the successful consultant.
11) Consultancy Outputs
The consultant should produce the following outputs:
a) Hard (bound) and soft copies of final evaluation report (ready to print/publish), preceded by two draft copies of the report for review by Mercy Corps and one final draft presentation for Mercy Corps.
b) Softcopy of final evaluation power point for 20 minute presentation.
c) Presentation of final report for the Somalia Education Committee.
d) Presentation of final report for USAID.
11) Consultancy Cost
The cost of the consultancy will be based on the number of days and cost of travel. This is however subject to negotiations with Mercy Corps.
12) Qualification and Experience
The consultant selected should have proof of competence, expertise, and track record of conducting assessments of similar scope and related programs.
13) The Bidding Process
The application should contain:
Mercy Corps and the consortium members expect that the entire evaluation process will cover a period of 30 days during the months of May and June 2012.
9) Timetable
The Consultant will develop a draft timetable for the assessment, which will be discussed with the Chief of Party and M&E Specialist for comments and approval before the commencement of work.
10) Evaluation Report Format
The final evaluation report format will be agreed with the consultant and CoP/M&E Specialist upon signing of the contract with the successful consultant.
11) Consultancy Outputs
The consultant should produce the following outputs:
a) Hard (bound) and soft copies of final evaluation report (ready to print/publish), preceded by two draft copies of the report for review by Mercy Corps and one final draft presentation for Mercy Corps.
b) Softcopy of final evaluation power point for 20 minute presentation.
c) Presentation of final report for the Somalia Education Committee.
d) Presentation of final report for USAID.
11) Consultancy Cost
The cost of the consultancy will be based on the number of days and cost of travel. This is however subject to negotiations with Mercy Corps.
12) Qualification and Experience
The consultant selected should have proof of competence, expertise, and track record of conducting assessments of similar scope and related programs.
13) The Bidding Process
The application should contain:
- Not more than 3
pages technical proposal stating suitability and capability, and proposed
methodology, and team composition;
- Itemized
financial estimates;
- Indicative work
plan; and
- Not more than 5
pages CV of the lead consultant.
Send
the above application documents to vacancy@so.mercycorps.org by close of
business 20 April 2012.
Logistics
Mercy Corps will provide the Consultant with transport and other logistic needs during his/her visit to the field.
Logistics
Mercy Corps will provide the Consultant with transport and other logistic needs during his/her visit to the field.
