The COVID-19 pandemic led to dramatic shifts in the way that most, if not all, educational institutions in the nation operate. The shift to distance learning sent a shock wave through the field of education, making even more visible how our current system is failing many of our students, particularly students of color, and their families. It led educational institutions to pause conventional activities built on long-held (and often harmful) assumptions, such as equating “seat time” with learning and an over-reliance on the results of standardized tests to measure student achievement. School systems, educators, and policy makers then began to ask anew: “What is most important for students to learn? How do we know students are learning? What are the most important metrics and tools we have for students to effectively demonstrate what they know?” In other words, over the last year, the education world has been engaged in a redefining of student success.
Since the beginning of the pandemic, SPR has led a range of evaluation projects in schools and with youth that provided a firsthand look at the impact of COVID-19 on student learning. As evaluators who care deeply about opportunities for all students to thrive, we understand that in order for there to be permanent changes in how student academic success is defined, the broader system for measuring student success must also change—and that we have an important role to play in that change. Thus, we have been working in collaboration with our clients to generate new ways to think about and measure student success at the school and district levels.
To highlight a few lessons learned that have surfaced out of our own lived experiences and evaluation work, a few SPR staff members share how we, as evaluators, attend to new and different ways of measuring and defining student success.
For education researchers and evaluators, School Year 2020-21 was a puzzle. Our tried-and-true methods—like in-person site visits and observations—suddenly became impossible, and we were forced to rethink how we approach school systems in virtual ways, particularly in a time of chaos and trauma when research is (rightfully) very low on their priority list. So, we adapted. We honed our skills around virtual data collection, conducting interviews, focus groups, and even classroom observations via video conferencing. We changed the way we think about engaging with schools, and especially with students and families, to respond to this unique moment in time in more culturally relevant and intentional ways. But, as a field, even though we have navigated and adapted the how of our work, we have yet to shift the why.
As the COVID-19 pandemic continues to illuminate the depth and breadth of inequities woven into the fabric of our education system, the education research and evaluation world is being pushed toward a similar reckoning as it becomes even more clear what many have known all along: student learning is not confined to the walls of the school building. So why, despite all that the pandemic visibilized, are we still using standardized, in-school assessments as the measure of student success? What do we lose by not seeing students as their full selves, understanding their success as a complex and nuanced amalgam of adaptive skills and culturally-grounded understanding—one that includes but is so much more than academic content knowledge? Over the last year and half, we’ve changed the way we do our work. Let’s also change why.
– Daniela Berman, Assistant Director
Student academic success is commonly conceptualized in terms of standardized testing and high-stakes tests. While heavily prioritized federal, state and district-level assessments have traditionally been used to track and compare student academic achievement at a high level, formative school-level assessments (including classroom assessments) offer value in gauging where students are in their learning. The closer we get to the school-level assessments, the more quickly teachers can collect and analyze assessment results to make appropriate instructional adjustments for students on an individual level. COVID-19 has challenged long-held testing prioritizations as several states and districts abandoned testing. For many, what remained was a reliance on school-level assessments to track student progress. One can imagine the many opportunities and challenges this shift created within the COVID-19 context. For instance, I am working on a project where the acceleration of schools’ investment in technology to support distance learning has expanded school-level assessments that have been used to encourage student individualized learning and self-pacing. However, I have also heard of stories regarding the technical challenges and inequities that impact assessments and the consequential reduced quality of the data that measure student academic success. Even so, I suspect that, with these assessments coming from the school or classroom, we are seeing an influx of data that teachers value to improve individual student learning than we would if standardized and high-stakes testing continued.
Relatedly, COVID-19 has forced education stakeholders to not only pay attention to but invest in the outside the school building connection to student academic success. Initiatives investing in meals, clothing, technology, and SEL have become common across my projects. I think for many the shift is a long-term one. As we begin to redefine student academic success, I hope we remember these lessons around how we measure it and the many factors that impact it.
– Nikkolette Hunter, Associate
Over time in the K-12 system we have seen a shift in focus to critical thinking and communication, which employers have identified as essential skills for thriving in the modern workplace. While basic technology skills have also become increasingly important in the workplace, the shelter-in-place orders required much of the American workforce to rapidly increase their tech savvy. Remote working for most companies is becoming increasingly common, and many think in-person workplaces will never be the same. In a McKinsey Global Institute report cited by an Education Weekly article, 20-25% of employees will likely be able to work from home at least three days a week after the pandemic. As part of SPR’s evaluation of the Oakland Fund for Children and Youth and Oakland Unified School District’s expanded learning programs, I have been interviewing expanded learning programs here in Oakland. In these interviews, I’ve been hearing stories about how youth have successfully adapted to a more technology-based world. These comments have left me to wonder, “Is anyone measuring students’ technology skills throughout the pandemic?”
There have been a lot of drawbacks from distance learning and it is clear that not everyone is benefitting from the current virtual model. Personally, I know through the pandemic some of my own skills have decreased (e.g., in-person communication) and some of my other skills have increased (e.g., facilitating Zoom meetings). Knowing that a similar phenomenon is happening in the classroom (some skills are decreasing, and others are increasing), I am wondering how we can move away from a deficit lens and ensure we are capturing other skills that have traditionally not been a part of regular schooling assessments. Whether it’s new technology skills or something else, I encourage evaluators and our partners to do some digging around new skills youth might be developing that are not currently captured in assessment and evaluation tools.
– Savannah Rae, Associate
The Link Between Language Development Interruption and Academic Success
When I think about the impact COVID-19 has had on students’ academic success during the past 2020-21 academic year, I often think about the language learning loss for English Learners (ELs). As a former EL myself raised by non-English speaking parents, it is hard to imagine that I would ever become proficient in speaking English, much less be able to improve my reading, writing, and comprehension if I were not in a classroom setting. When we think about language acquisition, we often hear, “If you don‘t use it, you lose it” and I would agree with that statement. Learning a foreign language is a complex and dynamic process that requires ongoing exposure and practice with peers, educators, and interactive pedagogical activities. Also, the time, effort, and resources it takesfor EL students to become fluent English-speakers is like other traditional academic-based subjects (e.g., Math, History, Science, etc.). For EL students, mastery of the English language is a building block for learning other subjects and resource allocation should reflect adequate supports. That said, school districts that only look at traditional academic-based subject matter in their student assessments are only scratching the surface in describing a root cause that hinders learning, as well as being able to quantify the short-term and long-term academic and non-academic impacts of COVID-19.
In response, my hope is that school districts expand their assessments in how they conceptualize student academic success because there are many EL students who may not have interacted with people outside their immediate families over the course of the last academic year. It’s important to recognize that over the past year, many students have strengthened their home language – an important skill to have given the global economy that we live in, where bilingualism is seen as an asset in the job market. As part of SPR’s evaluation of the Road Map Project (RMP), through interviews I learned about how RMP partners came together to pass the Seal of Biliteracy – a prestigious academic acknowledgment on a student’s high school transcript and diploma indicating they are bilingual and biliterate. I believe that the RMP demonstrates how school districts can shift from deficit-based to asset-based definitions of student success to further educational justice.
– Juan Carlos, former Associate
To say that COVID-19 has affected nearly every aspect of our education systems for students, families, educators, and administrators is not an exaggeration. A window of opportunity is open around Elementary and Secondary School Emergency Relief (ESSER) funds and investing in alternative forms of assessment. How are we, as evaluators going to use this moment to help broaden definitions of student success?