One of the challenges that evaluation teams face is finding evidence of whether a project’s results made a difference. The most straightforward way to address this challenge is to replicate a baseline study to measure the difference directly. Unfortunately, this is not always possible even when a baseline measurement exists. The Marawi Response Project (MRP) endline performance evaluation faced this challenge and developed partial solutions to overcome it by creating a comparison group because comparisons with the baseline group were not possible.
USAID requested that CLAimDev conduct an endline evaluation of MRP, implemented by Plan International from September 2018 – March 2022. MRP was USAID’s medium-term response to the 2017 Marawi Siege. USAID designed MRP to support the transition of internally displaced persons (IDPs) to social and economic stability and establish the conditions for local governments and communities in and around Marawi to address their long-term rehabilitation needs after the siege. MRP aimed to solidify a social contract between the local population and the local government and to enhance social cohesion between IDPs and HCMs as a means of alleviating latent social conflicts.
MRP conducted a baseline study of IDPs and HCMs in 2019. One intent of CLAimDev’s endline performance evaluation was to replicate the baseline study to determine whether there had been a change in IDP and HCM perceptions of self-reliance, polarization, and public participation and representation attributable to MRP’s interventions.
However, the evaluation team experienced difficulty in involving the baseline respondents for the endline survey due to contamination from other non-USAID interventions in the area, difficulty in establishing contact, and refusal to participate. This affected the already small sample size of the baseline sample. As a result, it was not feasible to perform statistical analyses that compared the baseline and endline groups. Instead, the evaluation team created a comparison group to the MRP beneficiaries (treatment group) to determine whether there were differences in the endline perceptions of a group similar to the baseline respondents and MRP beneficiaries, some of whom were included in the baseline study. The evaluation team used this endline survey, along with key informant interviews, focus group discussions, and document reviews to answer the evaluation questions on the relevance, effectiveness, and sustainability of MRP.