With little fanfare last month, Chicago Public Schools released its second edition of “value-added scores” for every elementary school in the system—offering parents what is theoretically a fairer way to gauge school performance and sounding a note of caution on school turnarounds.
Secretary of Education Arne Duncan is pressing districts across the country to emulate Chicago’s “turnaround” approach, in which teachers are replaced wholesale at struggling schools as a way to jumpstart academic changes. Yet the latest value-added scores (download in Excel) paint a decidedly mixed picture.
Schools run by the Academy for Urban School Leadership make a strong showing in math. But one CPS-led turnaround effort raises red flags.
With little fanfare last month, Chicago Public Schools released its second edition of “value-added scores” for every elementary school in the system—offering parents what is theoretically a fairer way to gauge school performance and sounding a note of caution on school turnarounds.
Secretary of Education Arne Duncan is pressing districts across the country to emulate Chicago’s “turnaround” approach, in which teachers are replaced wholesale at struggling schools as a way to jumpstart academic changes. Yet the latest value-added scores (download in Excel) paint a decidedly mixed picture.
Schools run by the Academy for Urban School Leadership make a strong showing in math. But one CPS-led turnaround effort raises red flags.
The value-added scores were developed by University of Wisconsin researchers and compare a student’s year-to-year growth on the Illinois Standards Achievement Test with growth made by demographically similar students from across the district. A school receives a green, neutral or red light at every grade level based on how well its students collectively “gain” on reading and math tests.
What makes the score arguably fairer—but also far more complex—is that students are only scored against others who share background characteristics like race, poverty status and previous academic performance. All comparisons are then summed up and schools receive red or green marks if average gains are lower or higher than district averages.
About 60 percent of the scores handed down to schools are neutral, as there is no way to tell if the gains are statistically significant.
But at Harvard Elementary, which AUSL took over as a “turnaround” in 2007, three out of five grades made significant gains in math and—as a school—reading gains were also in the green. Howe Elementary, an AUSL turnaround in 2008, also scored heavily green in math.
Overall, three out of five AUSL turnarounds posted significantly higher math scores compared to about one in five schools across the district. In reading, the schools posted average marks, with one—Morton Elementary, a 2008 turnaround—clearly struggling.
Sherman, a 2006 turnaround, earned mostly neutral marks. And at Dodge Elementary, which was closed for an entire year before opening as an AUSL school in 2005, math scores were excellent but reading results mixed.
The scores for Dodge reflect the power of the value-added measure to shed light on the argument that the turnaround process ends up driving out needier students.
The percentage of students who have met state standards has steadily risen at Dodge since the AUSL takeover. But detractors have argued that the school’s closure resulted in a wholesale changeover in the student population. In essence, they argue, the school’s scores are higher only because it is servicing a different set of students who face far fewer social problems at home and in their community.
The value-added score, however, only compares students who were in the school from year-to-year with similar students across the district. “The value-added score kind of renders the argument moot,” as Laura Smith, AUSL’s knowledge manager, puts it.
Smith says the scores have struck AUSL principals as fairly good indicators of what’s going on in their schools. If a “rock star” teacher is stationed in the fourth grade, she says as an example, principals generally saw green marks there.
The schools have also adopted a new bi-weekly assessment in math—developed in house by AUSL educators—that she credits with the overall strong showing at schools like Harvard and Dodge.
Still, the results may be informative to outside observers but they come too late to be of much use to educators, she adds. At Morton, for example, administrators realized the “turnaround” effort was flailing early last year and decided to replace the principal and, later in the year, four out of 10 teachers. AUSL adminstrators were not surprised by the lowly value-added scores posted by Morton several months later.
While AUSL marks were generally strong, one of the two elementary schools that underwent district-led “turnarounds” in 2008 raised major red flags. Fulton Elementary scored red marks in four out of five grade levels on reading tests and two out of five grade levels in math.
On the bright side, the districts efforts did result in an overall green light in math at Copernicus.