2021-2023 Analysis
Katie James, Ph.D., Director of Research
Kelsey Rocha, Ph.D., Research and Data Consultant
Neha Pandit, Senior Manager, Research and Program Operations
Summary
This study examined whether PowerMyLearning’s Social Emotional Learning (SEL) program improved elementary grade student and teacher SEL competencies, teacher retention, and academic achievement across the 2021-22 and 2022-23 school years.
The results of this study indicate that PowerMyLearning’s SEL program was related to significantly better student SEL skills, teacher confidence in supporting SEL, teacher retention, teacher-family relationships, and school climate. The findings also suggest a significant impact on SEL for Multilingual Learners (MLLs). The program was also associated with significant differences in math and reading achievement.
These two-year findings suggest that PowerMyLearning’s impact can be sustained and deepened across multi-year partnerships leading to lasting changes for our school partners.
SEL Outcomes
Our two-year SEL partnership was associated with significantly better student SEL from both the teachers’ perspective (+45pp, p<.001, ηp2=.82) and the students’ (+26pp, p<.001, ηp2 =.53). After the SEL program, teachers had significantly better confidence in supporting their students’ social emotional learning (+36pp, p<.001, ηp2 =.68).
During our partnership, the partner school also had significant improvements in teacher reported school climate (+39pp, p<.001, ηp2 =.84) and staff-family relationships (+46pp, p<.001, ηp2 =.89).
There is also evidence that we are successfully having an impact on the learning experiences of Multilingual Learners (MLLs). The results indicate significantly better overall SEL for MLLs after the program (+28pp, p<.001, ηp2 =.55).
Teacher Retention Outcomes
PowerMyLearning’s coaching integrates research-backed strategies that strengthen retention and includes: providing regular, positive feedback to teachers; helping teachers identify areas of development; and giving informal critical feedback about performance (Wu et al. 2012).
The partner school had an 8-percentage point increase in teacher retention during our partnership. Indeed, 96% of teachers retained into 2023-24 as compared to 88% in 2022-23.
Academic Outcomes
With respect to academic achievement, in the second year of the program there was a statistically significant difference in the proportion of students who exceeded or met district benchmarks before and after the program for Star Reading (+7pp, p=.001), Star Math (+14pp, p<.001), and Next Step Guided Reading Assessment (NSGRA) (+4pp, p=.003).
Research Context
This study examined the impact of PowerMyLearning’s SEL Program on grade 1-6 teachers, students, and families at one partner school in San Jose, California during the 2021-22 and 2022-23 school years. The student demographics of this school were 67% Hispanic, 29% Asian American and Pacific Islander (AAPI), with 78% of students qualifying for free or reduced lunch.
The school was eager to focus on SEL during the 2021-2022 school year to address behavioral and interpersonal challenges that arose when students returned to in-person instruction following a year of remote learning during COVID. They chose to continue focusing on and deepening their SEL practice during the 2022-23 school year.
Principal Magdalena Moore and Curriculum Support Specialist Rosalie Yu shared that the two-year partnership had made SEL an integral part of the school culture and created space for academic learning to be the focus rather than behavioral work.
“Our SEL work with PowerMyLearning has put us in a position where academics can now be the focus. Especially to see an increase in math, math is one of those mindset subjects where you hit a wall and you need to know how to problem solve through it. That makes sense with the work we have been doing with PowerMyLearning." – Magdalena Moore, Principal
During the 2021-2022 and 2022-23 school years, 26 educators, including K-6 teachers and other school staff and leadership, participated in the
program with a total reach of 523 students. This study focuses on data collected from 12 grades 1-6 teachers. Details of the study and our impact are provided in the research methods section of this report.
"We've noticed a significant positive change in how this year is starting. Classrooms are calmer, already able to dive into learning, and require less social-emotional intervention due to ingrained practices. Students now show greater understanding for differences, thanks to our partnership with PowerMyLearning.” – Rosalie Yu, Curriculum Support Specialist
PowerMyLearning SEL Program
PowerMyLearning’s SEL Program includes workshops that focus on SEL and culturally responsive instruction. Our workshops for teachers are always followed by small group coaching to extend learning into daily practice.
Across the 2022-23 and 2021-2022 school year, the partner school received the following:
- Eight Educator Workshops for 26 teachers, leadership, and other school staff. The workshops focused on both SEL and culturally responsive education and were designed to provide strategies for interpreting and addressing challenging behaviors in and out of the classroom, building an inclusive community of motivated learning, and advancing equity in the classroom. Workshop titles included: Trauma-Informed Practices that Benefit All Learners, Mindfulness and More for Kids and Adults, Build a Classroom Community Where All Student Feel They Belong, and Transform Discipline into a Learning Opportunity, Mitigate the Impact of Implicit Bias, Build Trust, Apply an Equity Lens, and Rigor and Relevance.
- Coaching for 18 teachers in grades 1-6 to extend learning from the Educator Workshops into daily practice. Each teacher participated in three cycles of three coaching sessions per year (18 total sessions). Coaching sessions were 45 minutes each, resulting in 13.5 total coaching hours per teacher across the two years. The coaching cycles followed the first three Educator Workshops each year.
- Eight 60-minute Family Workshops delivered in Spanish and English that were designed to provide strategies for families to support their children’s well-being, as well as their own. Workshop titles included: Self-Care for Family Wellness, Strategies to Foster Your Child’s Motivation, Navigate Cyberbullying and Social Media, Understanding Your Child’s Developmental Milestones, Family Wisdom, How to Share Expertise, Planning for the Future, and Strengthen Community with Empathy and Dialogue.
Research Methods
In this section, we will describe the study design, data collection, data analysis, and study results. We will then discuss the implications and limitations of the results and next steps of this work.
Design
We used a pre-post study design to examine PowerMyLearning’s impact on SEL competencies and academic achievement. In both year 1 and year 2, SEL survey data was collected from 12 teachers in grades 1-6 in October 2021(pre) and March 2023 (post). SEL student survey data was collected at the same timepoints as the teachers, student data was collected from 136 students who started the 2021 SY in grades 3-5 and ended the 2023 SY in grades 4-6. In year 2, academic achievement data was collected from 372 students in grades 1-6 at the beginning and end of the school year. We also looked at SEL outcomes for 79 grade 4-6 students who identified as Multilingual Learners.
Data Collection
SEL Survey. Students completed a 39-question SEL survey adapted from the Panorama student survey. PowerMyLearning researchers modified the original survey to focus on questions related to hypothesized outcomes on student SEL competencies. The SEL competencies included self-efficacy, growth mindset, self-management, and social awareness. The survey also included questions on student engagement, sense of belonging, teacher-student relationships and sense of school safety. Teachers also completed a 20-question SEL survey adapted from the Panorama teacher survey. PowerMyLearning researchers modified the original survey to incorporate questions related to additional hypothesized outcomes, including teachers’ perceptions of school safety and students’ sense of belonging. In addition, we adapted questions from the Panorama SEL survey to pertain to the whole class, rather than individual students, and we added questions to the survey to examine teacher perceptions of student SEL competencies. Thus, the SEL survey included questions about student SEL competencies, teacher confidence in supporting SEL, teacher-family relationships, and school climate. The SEL competencies included self-efficacy, growth mindset, self-management, and social awareness. The teacher confidence questions focused on supporting those same student SEL skills.
Teacher Retention. Pre-post teacher movement data was collected from school administrators to understand teacher movement. Movement data was collected from the 2020-21 to 2021-22 school year as baseline and then again from the 2022-23 to 2023-24.
Academic Assessments. Academic achievement data was based on school-administered achievement tests. The assessments include Renaissance Star Reading (grades 2-6; N=283), Renaissance Star Math (grades 1-6; N=361), and Next Step Guided Reading Assessment (NSGRA) (grades 1-3; N=128). The reading test data were norm-referenced scores that provide an indicator of students’ reading level (Instructional Reading Level for Star Reading and Scholastic Reading Level for NSGRA). The Star Math data were norm-referenced grade equivalent scores.
Analysis
SEL Survey Analysis. Responses to each survey question were scored on a 5-point scale. In cases where multiple survey questions pertained to the same SEL construct, such as teacher-family relationships, the individual question scores were averaged to create an overall score for that construct. We conducted paired samples t-tests on the pre/post-data for each of the SEL constructs.
Teacher Retention Analysis. We calculated the percent of teachers retained at baseline and after the SEL program.
Academic Assessment Analysis. We evaluated beginning of year student data relative to the beginning of year district benchmarks for each assessment. We evaluated end of year student data relative to the end of year district benchmarks for each assessment. We analyzed whether each student exceeded or met the relevant district benchmark for each assessment at each timepoint, resulting in categorical data. We then conducted McNemar’s tests on the beginning and end of year data for each assessment. We also calculated the percent of students who exceeded or met the relevant district benchmark for each assessment at the beginning and end of the year.
Results
SEL Results. With respect to student SEL, the two-year SEL program was associated with significantly better student-reported overall SEL (+26pp, p<.001, ηp2 =.53) and specifically self-efficacy (+41pp, p<.001, ηp2 =.42), social awareness (+35pp, p<.001, ηp2 =.55), self-management (+32pp, p<.001, ηp2 =.52), and growth mindset (+32pp, p<.001, ηp2 =.32). The program was also associated with significantly better student-reported student engagement (+24pp, p<.001, ηp2 =.29), sense of belonging (+23pp, p<.001, ηp2 =.31), and teacher-student relationships (+16pp, p<.001, ηp2 =.17). Teachers were also asked to report on student SEL outcomes. Overall the two-year SEL program was associated with better teacher reported overall SEL (+45pp, p<.001, ηp2 =.82) and specifically self-efficacy (+57pp, p<.001, ηp2 =.80), social awareness (+41pp, p<.001, ηp2 =.67), self-management (+52pp, p<.001, ηp2 =.70), and growth mindset (+36pp, p<.001, ηp2 =.70).
Multilingual Learners were the only group for whom the SEL program was related to significantly better sense of safety at school (+8pp, p<.03, ηp2 =.06). For MLLs, the program was also associated with significantly better overall SEL (+28pp, p<.001, ηp2 =.55), student self-efficacy (+40pp, p<.001, ηp2 =.38), social awareness (+37p, p<.001, ηp2 =.58), self-management (+34pp, p<.001, ηp2 =.56), growth mindset (+37pp, p<.001, ηp2 =.41), student engagement (+24pp, p<.001, ηp2 =.29), sense of belonging (+23pp, p<.001, ηp2 =.26), and teacher-student relationships (+18pp, p<.001, ηp2 =.20).
The findings also suggest a significant impact on teacher confidence in supporting student SEL (+36pp, p<.001, ηp2 =.68). Specifically, the SEL program was associated with better teacher confidence in supporting student self-efficacy (+48pp, p=.001, ηp2 =.63), social awareness (+28pp, p=.03., ηp2 =.39), self-management (+48pp, p=.001, ηp2 =.63), growth mindset (+31pp, p<.001, ηp2 =.67), student sense of safety (+38pp, p<.001, ηp2 =.70) and sense of belonging (+78pp, p=.001, ηp2 =.60). The results also indicate a significant effect of the program on teacher-family relationships (+46pp, p<.001, ηp2 =.89) and teacher perceptions of school climate (+39pp, p<.001, ηp2 =.84).
Teacher Retention Results. The partner school had an 8% increase in teacher retention during PowerMyLearning’s partnership. Indeed, 96% of teachers retained into 2023-24 as compared to 88% in 2022-23.
Academic Achievement Results. In the second year of the program, the change in the proportion of students who exceeded or met district benchmarks before and after the SEL program was statistically significant for Star Math (+14pp; p<.001), Star Reading (+7pp; p=.001), and Next Step Guided Reading Assessment (+4pp, p=.003).
Discussion
PowerMyLearning is committed to building evidence around the impact of our work. This study found statistically significant impact of PowerMyLearning’s SEL program on multiple aspects of SEL. The program was associated with a significant increase in Multilingual Learners’ sense of safety, which was not found for the overall student population. This suggests that the SEL program is particularly successful at reaching high needs students, including MLLs. The program was also associated with significant differences in math and reading achievement.
The two-year findings suggest that our impact can be sustained and deepened across multi-year partnerships leading to lasting changes for our school partners. One limitation of this study is that the results reflect the impact of all learning experiences during the 2022-23 and 2021-2022 school year and cannot be solely attributed to the SEL program.
Beginning in Fall 2023, a randomized control trial is planned to examine the impact of PowerMyLearning’s SEL program and Family Playlists, which will enable us to more precisely determine the extent to which our programs are causing observed effects.
Acknowledgments
This study was generously funded by the Leon Lowenstein Foundation.
About PowerMyLearning
Founded in 1999, PowerMyLearning is a national nonprofit that partners with schools and districts to advance educational equity and accelerate student success. The organization’s evidence-based programs build teacher and leader capacity to accelerate student learning and support social emotional learning. As a recognized leader with extensive experience serving districts in underserved communities, PowerMyLearning helps schools activate the exponential power that occurs when teachers, students, and families collaborate on the same learning goals—the “Triangle of Learning Relationships.” The impact from partners is unanimous: when students are supported by an active learning community, they thrive academically and in their well-being. To learn more, visit PowerMyLearning.org.
