MRP

One of the challenges that evaluation teams face is finding evidence of whether a project’s results made a difference. The most straightforward way to address this challenge is to replicate a baseline study to measure the difference directly. Unfortunately, this is not always possible even when a baseline measurement exists. The Marawi Response Project (MRP) endline performance evaluation faced this challenge and developed partial solutions to overcome it by creating a comparison group because comparisons with the baseline group were not possible.

USAID requested that CLAimDev conduct an endline evaluation of MRP, implemented by Plan International from September 2018 – March 2022. MRP was USAID’s medium-term response to the 2017 Marawi Siege. USAID designed MRP to support the transition of internally displaced persons (IDPs) to social and economic stability and establish the conditions for local governments and communities in and around Marawi to address their long-term rehabilitation needs after the siege. MRP aimed to solidify a social contract between the local population and the local government and to enhance social cohesion between IDPs and HCMs as a means of alleviating latent social conflicts.

MRP conducted a baseline study of IDPs and HCMs in 2019. One intent of CLAimDev’s endline performance evaluation was to replicate the baseline study to determine whether there had been a change in IDP and HCM perceptions of self-reliance, polarization, and public participation and representation attributable to MRP’s interventions.

However, the evaluation team experienced difficulty in involving the baseline respondents for the endline survey due to contamination from other non-USAID interventions in the area, difficulty in establishing contact, and refusal to participate. This affected the already small sample size of the baseline sample. As a result, it was not feasible to perform statistical analyses that compared the baseline and endline groups. Instead, the evaluation team created a comparison group to the MRP beneficiaries (treatment group) to determine whether there were differences in the endline perceptions of a group similar to the baseline respondents and MRP beneficiaries, some of whom were included in the baseline study. The evaluation team used this endline survey, along with key informant interviews, focus group discussions, and document reviews to answer the evaluation questions on the relevance, effectiveness, and sustainability of MRP.

  • Relevance

    Were MRP’s development interventions relevant to the needs of the IDPs and HCMs and adaptive to the complex and changing external environment in Mindanao, especially in the project areas?

    The evaluation team found that MRP’s intervention was highly relevant to USAID’s and the national government’s objectives. Even more, the interventions were highly relevant to the local government and neighborhood (barangay) leaders as well as the community solidarity groups whose members were the activity’s primary beneficiaries. MRP was the first USAID/Philippines activity that was developed through a co-creation process. This process was key to ensuring MRP’s relevance to those it was to benefit. MRP’s highly collaborative approach with local stakeholders and flexible design that allowed for adaptive management throughout the life of the project also enhanced its relevance.

    BOX 1. HARNESSING LGU PARTNERSHIP FOR RELEVANCE, EFFECTIVENESS AND SUSTAINABILITY

    The Municipality of Bubong, Lanao del Sur is a second class municipality with a population of 26,514. At the height of the Marawi Siege, Nabillah R.H. Abdulhakim, served as the Municipal Administrator and worked closely with Maradeca, MRP’s implementing partner. She lauded how MRP went down to the communities and asked people about their priority needs. “MRP asked people about their needs. The women expressed they needed support for dressmaking to earn income. During the graduation ceremony, I saw the products of women beneficiaries. The women worked hard and committed to supporting their families.”

  • Effectiveness

    Did MRP accomplish the project’s expected outcomes of improving self-reliance, such as increasing business and livelihood opportunities, strengthening social cohesion, and increasing participation such as in civic and economic activities among IDPs and HCMs?

    MRP bolstered beneficiaries’ self-reliance in the face of external challenges, including economic challenges brought about by COVID-19 pandemic. Amidst the ramifications of the pandemic, the business grants, community micro grants, and training that beneficiaries received allowed them to generate income by starting businesses and engaging in gainful employment. Thus, MRP created a stimulus for the restoration of individual and community capacities to engage in economic activities. Moreover, the project’s interventions contributed to the reduction of gender and social inequalities, particularly those that were noted during the baseline study.

    MRP’s social cohesion grants and training fostered civic engagement and community improvement among the IDPs and HCMs. The project contributed to improvements in the beneficiaries’ perceptions of social relations, connectedness, and orientation toward the common good. It has improved the perceptions of the IDPs and HCMs regarding the existence of polarization in their communities and their level of public participation in civic activities.

    The evaluation found that the proportion of endline survey participants who reported that their income is sufficient to meet the basic needs of their families (e.g., food, medicine, children’s education) was higher among participants who received MRP interventions (44.5%) than among the participants from the comparison group (38.0%). In addition, those who reported that they have savings is higher among survey participants who received MRP interventions (15.3%) than among the comparison group (11.3%). The evaluation also found that the proportion of MRP beneficiaries who indicated they have sufficient income to meet the needs of the basic needs of their family (44.5%) is higher compared to the general population in Bangsamoro Autonomous Region in Muslim Mindanao (19.0%).

    The endline survey data also suggest that more MRP beneficiaries than members of the comparison group are engaged in paid livelihoods (55%), have the right skills to find a job (83%), and are confident in finding a job should they lose their present job or livelihood (59%). An inferential analysis suggests that the proportion of MRP beneficiaries is significantly higher in terms of engagement in paid work/livelihood (p=0.024) and having the right skills to find a job (p=0.000). These findings imply that MRP’s interventions in the form of business/livelihood grants and training workshops have expanded employment opportunities of the IDPs and HCMs.

    MRP interventions also supported increased social cohesion. The evaluation found that MRP beneficiaries’ overall perception of polarization significantly improved at the endline (p=0.020). The significant increase in the scores of the beneficiaries from the baseline to the endline suggests that the beneficiaries are perceiving significantly less polarization or fragmentation in the communities they are living in at present. MRP beneficiaries’ level of trust also significantly increased in the endline (p=0.046). MRP beneficiaries’ level of trust was significantly higher than the comparison group (p<0.001). The effect size generated in the analysis (d=0.481) suggests that MRP interventions have a medium or average effect contribution to the improvement in the trust level of the beneficiaries.

    Based on the endline survey, the evaluation also found that MRP beneficiaries have a high level of civic participation compared to the non-intervention group. Further analysis revealed that the level of participation of MRP beneficiaries is significantly higher than the comparison group. The effect size from the analysis suggests that MRP interventions have a large effect (d=1.113) on the level of participation of the beneficiaries.

  • Sustainability

    Did MRP establish mechanisms to promote sustainable improved economic conditions and strengthen the social cohesion between IDPs and HCMs?

    MRP has established concrete sustainability mechanisms that will be essential in sustaining the outcomes of the project. MRP strengthened the buy-in of strategic stakeholders by harnessing partnerships between and among them, which resulted in institutional complementation.

    MRP laid the foundation for the Community Solidarity Groups (CSGs) to qualify for future government support and linked them to potential markets for their products. MRP developed building blocks for CSGs to continue to operate, serve their members, and strengthen livelihoods after the activity’s conclusion by empowering and formalizing the CSGs into cooperatives and civil society organizations.

    The capacity and partnership-building mechanisms that MRP developed provided an impetus to sustain the social and economic outcomes of the project.

    The strengthening, consolidation, and social network expansion of the CSGs transformed them into advocacy/sectoral organizations, serving as a strong foundation for sustaining economic and social cohesion gains. With these mechanisms, it is likely that the outcomes of the projects will be sustained after MRP ends .