Findings, Conclusions, and Recommendations

Two questions guided the evaluation of ABC+’s relevance: the first asked about how the program innovations aligned with DepEd objectives and policies, and in contrast, the second asked about how ABC+ improved four learning aspects of children in the early-grades.

RELEVANCE

To what extent does the innovation introduced by ABC+ on improved early-grade instruction and delivery, improved access to quality materials, and improved education capacity and commitment to manage and implement interventions supporting early-grade instruction correspond with the objectives of the Philippines DepEd (priorities, policies, needs, and the practical requirements)?

FGD and survey data findings show that ABC+ innovations aligned highly with DepEd policies. Its activities were instrumental in partner schools’ implementation of learning continuity, especially for literacy instruction. Data show that at the institutional level, ABC+ activities have aligned much or somewhat with DepEd programs and policies. Annex 15 presents a list of national and regional DepEd policies that ABC+ activities align with, according to informants.

Figure 1 below unpacks policies and programs that align with ABC+ innovations. Informants explained that ABC+ is so aligned with their activities that it does not appear like a “separate program because all the initiatives support what we are already doing.” Another explained, “What ABC+ has been doing is that they made sure that all their undertakings are aligned with the BARMM’s Ministry of Basic, Higher, and Technical Education (MBHTE) and DepEd.” Results revealed that partners enhanced or contextualized existing policies due to their ABC+ experience or confidence in ABC+ support.

Among DepEd policies and programs, ABC+ was particularly relevant in supporting DepEd’s learning continuity and learning recovery programs. As schools grappled with remote learning and the new demands of education, ABC+ supported schools’ learning continuity programs by providing supplemental reading materials (SRMs) to support reading practice. ABC+ teacher training also provided teachers with strategies they applied in preparing the modules for learners. ABC+ ensured that instructional materials were distributed on time with the help of stakeholders. Teacher training and materials development continued during the pandemic through online platforms and virtual communication.

The pandemic heightened the need for teacher training and materials development, and ABC+’s ability to supply these heightened its relevance, mainly because the regular training for teachers stopped and schools did not have adequate resources to support learners remotely to develop their reading skills.

According to one interviewed teacher, “ABC+ joined us at the height of the pandemic, so the experience was challenging, but at least, with their help, everything became easier.” ABC+’s relevance extends to the present, as schools try to recover learning lost during the pandemic. The materials that ABC+ provided and the training knowledge and skills teachers gained are valuable in learning recovery programs. As explained by a teacher, “ABC+ is very relevant, and it is what we need right now [learning recovery].”

To what extent has ABC+ contributed to improving early-grade literacy, the better transition of reading ability to Filipino and English, more solid acquisition of foundational math concepts and skills, and improved social and emotional learning for children in the early grades (kindergarten to Grade 3) in the Philippines?

The data presented could be more nuanced for transitioning reading skills to Filipino and English. Due to the learning gap incurred during the pandemic, the data imply that much of the reading skills described by informants are only for the mother tongue and Filipino, and transitioning the reading skills from the mother tongue to the second language will be taken care of after establishing foundational reading skills.

Survey results show that 80 percent of teachers in all the target regions feel that ABC+ initiatives have been instrumental in improving early-grade literacy instruction. These findings are supported by qualitative data where informants described ABC+ as their “partner” in literacy teaching. Figure 1 above unpacks the extent of ABC+’s contribution to literacy improvement across individual, institutional, and organizational levels.

ABC+ initiated collaboration with parents by capacitating home learning partners (HLPs) for remote learning and using SRMs that targeted literacy development. For teacher training, informants identified the content on pedagogy and technology use as crucial topics that improved their reading instruction. Teachers reported that the provision of literacy instructional materials improved their literacy instruction because materials were accessible via the Learning Resource Portal and were ready to print and easy to use. One informant shared that “learning became easier for children because of the materials provided by ABC+.”

At the institutional level, ABC+ contributed to school programs supporting literacy, learning continuity and recovery, materials development and distribution, and assessment-informed instruction. ABC+ contributed to school programs by providing teacher training and materials.

The most significant ABC+ contribution was its support of learning continuity and recovery. A teacher shared that ABC+ helped them design and implement an especially relevant intervention during learning recovery because there were so many non-readers. This effort to focus on fundamental skills such as literacy skills for learning recovery aligns with the World Bank’s Reach, Assess, Prioritize, Increase, Develop (RAPID) Framework for learning recovery, and the National Learning Recovery Plan was presented in DepEd’s Basic Education Report 2023.

On materials development and distribution, ABC+ assistance supported teachers’ skills development in materials writing and evaluation or quality assurance. Moreover, SRMs, which are leveled readers in the local language, help children practice their reading skills and develop an interest in reading. From a target of 576 titles, ABC+ produced 1,402 titles of EGR materials. These materials were made available to 750,479 learners (from a target of 523,201).

Schools implemented assessment-informed instruction through ABC+’s provision of the comprehensive rapid literacy assessment tool, which diagnosed learners’ reading skills and provided guidance for differentiated instruction tailored to learners’ diagnosed reading levels. Starting with an assessment to inform recovery learning intervention is consistent with the World Bank’s RAPID framework to regain learning lost during the pandemic.

ABC+’s collaboration with teacher education institutions (TEIs) resulted in the establishment of a pre-service course in literacy instruction. Through this partnership, ABC+ prepared pre-service teachers (including current and incoming teachers) early on for reading instruction.

Collaboration with the community involving the barangay, the police, private radio stations, and others provided the wraparound support needed to ensure learning continuity. The barangay and police helped ABC+ and partner schools deliver instructional materials to learners, while the partnership with radio stations enabled the broadcasting of lessons. ABC+ also initiated meetings to establish memorandums of agreement between schools and some stakeholder groups to ensure learning continuity. Meetings initiated by discussions paved the way for some stakeholder groups to express their commitment to supporting school programs.

Some of the following unexpected outcomes emerged from ABC+ activities:

  1. Teacher training and materials development produced a ripple effect. FGD informants from Regions V and VI said that trained teachers went on to train intermediate-grade teachers in their school and early-grade teachers in other schools. One teacher informant shared that the training has “enriched [me]…in so many ways,” She planned to cascade these skills with other teachers in the Division.
  2. FGD informants reported that learners at home used the materials that ABC+ produced, unlike the pre-pandemic practice where children were not allowed to bring home instructional resources. An informant from Region V shared that “learning became easier for children through the materials provided by ABC+.” Informants explained that materials benefited not only the learners but also their siblings and parents, some of whom had been illiterate and learned to read along with their children through the materials. An informant from Region VI said that “parents learned together with their kids” and that “illiterate parents were able to learn through reading materials put up [made available through hubs] in sitios and haciendas.”
  3. In Region V, informants shared that schools reproduced ABC+ materials. Any excess they received was shared with other schools, so schools beyond partner schools also benefited.

Although literacy has traditionally been ABC+’s primary focus, numeracy development is also becoming an additional target since it has been identified as a skill for which children likewise need help. However, the project’s numeracy focus has been initiated only recently.

In 2021, ABC+’s efforts to improve numeracy began with research in collaboration with the University of the Philippines National Institute for Science and Mathematics Education Development (UP-NISMED). The study looked into the state of early-grade math instruction, and results were shared with DepEd. Based on the research results, math instruction pivots based in the research results were designed, piloted, and evaluated. In 2022, ABC+ conducted teacher training on numeracy in Regions V and VI (Teacher Training I). Consultations with the Early Numeracy Technical Working Group began, and Teacher Training 2, which trained participants to connect formal and informal mathematics for children, was conducted in Regions V and VI. Math instructional strategies in the two regions were piloted by distributing ABC+ home learning resource kits that included materials for numeracy and learning action cell modules on early-grade mathematics.

Survey findings show that five out of ten teachers in the regions reported moderate ABC+ contributions to developing math skills. These contributions were reported on training, materials, and program support. These findings are validated by a comment shared by a Region VI informant at the evaluation team’s online Results Validation Workshop with DepEd. The informant explained that more activities on developing numeracy could be organized in the following stages of implementation.

Activities for developing mathematics skills were only recently implemented, and their impact will only be noticed after some time. Informants shared that numeracy materials developed are slated for distribution to partner schools soon. Training support was scheduled for late 2022 at the time of this evaluation. This training involves integrating vital instructional strategies and collaborative lesson planning, which will be demonstrated to children or peers and revised based on trainer feedback. Class observations for developing numeracy skills are also scheduled for early 2023.

ABC+ integrated efforts to strengthen socio-emotional learning (SEL) through instructional leadership training, believing that school leaders are critical in fostering learning environments that encourage SEL. The ABC+ Quarter 3 report states that the training taught DepEd leaders to foster conditions resulting in a positive learning environment. The training provided resources and toolkits that featured SEL in school management and showed supervisors and school heads how to integrate SEL and GESI into their school programs and interventions. The report also noted that ABC+ embedded SEL in SRM development by featuring an SEL orientation and workshop in materials and training. Moreover, SEL messaging was strongly encouraged in the themes of produced SRMs. Teacher Training 2 also taught teachers how to integrate SEL/GESI themes in book selection and other class exercises.

Based on the survey of teachers, ABC+ helped in improving learners’ SEL to a large extent. Almost eight out of ten teachers in BARMM stated this was the case, while nearly six out of ten Regions V and VI teachers expressed the same. Qualitative data show that ABC+ helped improve SEL among learners through teacher training, which taught strategies for this.

However, teachers’ explanations of SEL integration in surveys and FGDs were mainly about the values inculcated in learners through SRMs. They explained that SRMs helped in learners’ value formation. According to teachers, “Children learned moral lessons and applied these to their life.” Given the cessation of classroom face-to-face instruction during the pandemic, it appears that values in SRMs became the primary source for SEL integration. Future training on SEL can be used to check the depth and accuracy of teachers’ understanding of SEL integration, and actual SEL integration can be monitored and confirmed in face-to-face classes.

Program evaluation on relevance has two sub-questions: one is about the inclusion and implementation of GESI, and the other is on whether ABC+ considered the capabilities and capacities of recipients in response to the changing context of the COVID-19 pandemic.

How did ABC+ incorporate GESI analysis? To what extent has ABC+ implementation supported GESI?

Surveyed teachers claimed that ABC+ training helped them integrate GESI into their instruction. Annex 13 shows that almost eight out of ten teachers in BARMM stated that ABC+ contributions in their GESI integration were helpful to a large extent. In comparison, nearly six out of ten teachers in Region V and VI evaluated ABC+ initiatives as useful to a great extent. While these data are favorable, teachers from Region V and VI could be further supported in the following years of program implementation so that more teachers in these areas could integrate GESI into their instruction.

At the individual level, ABC+ incorporated and promoted GESI through training that developed teachers’ and HLPs’ awareness of the concept. At the institutional level, the contribution of ABC+ GESI training was manifested in GESI integration in the SRMs that teachers and writers developed and in teachers incorporating GESI in their instruction. One informant explained that featuring different tribes, different colors, and skin “was a way of supporting GESI.” Another informant said ABC+ training “made the teachers aware of gender stereotypes or potentially discriminating ideas in pictures and words in books given to learners.” Informants also explained that the provision of materials and activities that benefited struggling and special learners provided a means of living up to the principles of social inclusion.

Another informant expressed that their improved awareness of gender stereotypes has also influenced how they assign tasks to learners. One informant said, “girls are not the only ones asked to sweep the floor. Even boys are asked to sweep the floor.”

Did ABC+’s design and implementation consider the available capability and capacities of the recipients (individuals and organizations) and responses to any change in context (e.g., public health emergency such as COVID-19)?

The most significant change in context that the ABC+ program had to adapt to was the COVID-19 pandemic. Indeed, ABC+ made several adjustments in consideration of the capabilities and capacities of recipients in the face of COVID-19. All adjustments were due to the prohibited or restricted face-to-face interactions, which were most strict in early 2020. Though program activities had to be conducted remotely, ABC+ continued its activities by adapting to restrictions on physical interactions. The instructional and technological support provided by ABC+ aligns with World Bank’s RAPID framework for learning recovery and DepEd’s Basic Education Report 2023, particularly the advice to increase the efficiency of instruction by supporting teachers through upskilling in technology used for remote learning. Key project pivots included the following:

  1. Because there was no face-to-face training and physical contact at the onset of the pandemic, ABC+ provided pre-training orientation on the technology used to prepare partners for online training platforms.
  2. ABC+ also provided communication and Wi-Fi (internet) support or allowance to beneficiaries to ensure they could sustain online training participation.
  3. ABC+ reached out to recipients, monitored their progress, and sought to identify concerns through online means (virtual chat groups, emails).
  4. ABC+ utilized an adaptive training schedule that accounted for teachers’ busier schedules and other pandemic-related concerns. Training activities were made more flexible because they could be completed online or offline.
  5. ABC+ provided modules and materials for the training to give teachers more time to prepare.
  6. ABC+ also provided a longer time to finish training outputs.
  7. Eventually, some areas accommodated limited face-to-face training, allowing ABC+ to conduct onsite training only with smaller groups of trainees and with adjustments to fit into the “new normal” setup. Health protocols were followed strictly during training, and onsite COVID-19 testing was conducted to protect the health of participants and staff.
  8. Because a limited number of people were available to help with materials distribution, and due to limited transportation resources, ABC+ collaborated with the community, like the barangay and the police, for systematic distribution of materials from schools to the houses of learners to ensure the continuity of learning activities.

Key Message on Relevance

The emerging key message on relevance is that ABC+ teacher training, materials development, and community collaboration efforts played a crucial role in capacitating schools to ensure learning continuity during the pandemic. With ABC+ support, partner schools’ learning continuity and recovery efforts would have been unrestricted and less than optimal in terms of timeliness, efficiency, and quality.

Conclusions on Relevance

  1. ABC+ provided a complete and streamlined package of activities spanning teacher training, materials development, and system capacitation that was highly relevant to DepEd policies and programs, particularly on learning continuity and learning recovery. While other programs typically target only one goal, ABC+ recognized the value of the relationship among training, materials, and the overall system and worked on all three seamlessly and simultaneously. The efficient streamlining is seen in the complementary design and activities for teacher training, materials development, and system capacitation.
  2. ABC+’s highest level of support went toward literacy instruction. This affirms the achievement of the program’s primary goal, which is to improve children’s literacy skills. Initiatives for the three IRs benefited the improvement of children’s literacy skills.
  3. ABC+ initiatives for developing numeracy skills have just started and have yet to mature. Learners’ mathematics skills are expected to improve once initiatives are implemented more fully.
  4. SEL integration was more evident in developed materials than in instruction, possibly due to the absence of face-to-face instruction. Now that learners and teachers are back in their physical classrooms, the effects of initiatives on SEL integration in teaching can be monitored and enhanced.

Recommendations on Relevance

  1. ABC+ and DepEd could help regions give ideas to enhance or contextualize national or regional policies that ABC+ already supports. Data revealed that due to partner schools’ confidence in ABC+ support, some regional policies that involved only literacy had been expanded by partners to include numeracy. Some policies were also enhanced in terms of budget allocation. Processes for initiating these local policy enhancements can be shared with other regions through fora or meetings so that other regions can also identify local policies that can be enhanced or further contextualized.
  2. ABC+ may assist schools in disseminating best practices, especially in implementing recovery programs concentrating on literacy development. Findings show that some partner schools in Region V have developed implementation efficiency and achieved targeted literacy outcomes for their learning recovery programs. Schools can be supported in creating platforms for disseminating or sharing good practices, such as school visits for benchmarking or an online or F2F forum or presentation.
  3. Because numeracy instruction is designed to be a pilot model that will support the plan for developing a National Framework on Numeracy, ABC+ may strategically develop and design a supplementary framework that targets numeracy instruction in the key stages. This framework may emerge from the lesson development and implementation with UP-NISMED as their partner institution.
  4. ABC+ and DepEd regional office could bolster efforts for SEL integration, especially in the instruction of Region V and VI teachers. Generally, findings showed teachers’ explanation of SEL integration was limited to values development via SRMs. This may have been due to the absence of opportunities to implement SEL integration in F2F classes. With the resumption of F2F classes, teachers’ accuracy of understanding SEL integration can be checked in their class instruction, and appropriate training can be offered based on class observation data. While Regions V and VI data show that the extent of the help of ABC+ training for SEL integration was achieved to a great extent, these results (57.8 percent for Region V and 59.5 percent for Region VI) are much lower compared to BARMM’s (72.9 percent). F2F class observations can help determine the training support needed to help teachers in these regions increase their level of SEL integration.