CPS officials called the unveiling of a new teacher evaluation system a “historic opportunity” and said they were surprised that Chicago Teachers Union leadership wouldn’t stand with them in support.
Starting this fall, the current checklist system that administrators and teachers have long said is virtually worthless for improving instruction will be replaced at all schools with the new system. However, many teachers won’t be affected until fall 2013; the new system will only be used initially for probationary teachers and for tenured teachers with satisfactory and unsatisfactory ratings.
Mandated under state law, the new evaluations will factor in measures of student growth on standardized tests; progress on district-designed tests known as “performance assessments”; principal’s observations of teachers using a modified version of Charlotte Danielson’s Framework for Teaching. Student feedback will be incorporated in the second year of the program.
District officials dubbed the new evaluation system REACH, for Recognizing Educators and Advancing Chicago’s Students.
The union quickly weighed in, saying the new system is seriously flawed because it relies on value-added test scores–although state law mandates that student achievement be part of all new evaluations–does not provide enough checks and balances on a principal’ evaluation, and doesn’t give teachers an appeal process. But the CTU noted in a statement that “the new observation system is an improvement over the checklist system currently in place.”
Tim Daly, president of TNTP (formerly known as The New Teacher Project) and a member of the state Performance Evaluation Advisory Council, said that several districts around the country weigh achievement more heavily than Chicago’s new plan.
Factoring in achievement has become a nationwide trend in recent years, as the U.S. Department of Education has been pushing states to make student growth measures a part of teacher evaluation.
Neither New York City nor Los Angeles public schools have adopted new evaluation systems yet. A state law will require New York schools to do so in the future.
“The CPS approach is more moderate,” Daly says. “Under the law, the district could have pressed for quite a bit more weight [on value-added measures].”
However, Daly says, that will put more onus on CPS to make sure that teacher observations are done well – something that has not been the district’s strong suit in the past.
“The weight [on achievement] starts out so small that it’s unlikely to affect many teachers’ ratings,” he notes. Even when this percentage rises, “the ratings of the vast majority of teachers in CPS will continue to be determined primarily based on observations.”
The same is true of the district- and teacher-designed performance assessment.
“It’s easy with state tests to say the test makers are at fault, or the state is at fault,” Daly says. “These are going to be local tests, so the responsibility falls on the community of educators to make them good. And it’s going to be tough.”
Disney Magnet Elementary Principal Kathleen Hagstrom says that she already supplemented the district’s checklist with additional evaluation forms. With the new system, she is concerned about how she will find time to do detailed observations of her school’s 86 teachers.
“Some accommodations will have to be made for large staffs,” she notes. “I know those who did the pilot for the Danielson said the data input was extremely difficult.” But, she adds, she looks forward to finding out whether the new system will be helpful.
“I am hoping our questions will be answered,” she says. “Many people felt the past evaluation wasn’t effective. We will have to see.”
Because legally mandated negotiations between the union and the district had been going on since December- well past the minimum 90-day period specified in state law – the district was free to implement the evaluation procedures it most recently discussed with the union.
“I have mixed feelings, and I think it remains to be seen how this will play out,” said Carol Caref, coordinator of the CTU’s Quest Center. “We felt like it was better than what they started with, but there were many things in it we did not agree with.”
Robin Steans, executive director of Advance Illinois, praises the district’s work. “They deserve credit for this,” she says. “It’s an awfully good first step”
The law mandating the new evaluations was passed in 2010 to make Illinois a more viable candidate for the federal Race to the Top grant competition, which Illinois did not win.
Phasing in more reliance on student growth
The weight given to students’ learning gains – the most controversial part of the new evaluations, and one mandated by state law – will increase gradually over five years, until it accounts for 35 to 40 percent of a teacher’s evaluation. Student feedback will comprise 10 percent of the evaluations, and observation will remain the largest factor.
High schools will begin phasing in student growth measures based on the EXPLORE, PLAN and ACT assessments in September 2013. For the coming year, only teachers in core subjects will be rated on student growth, and it will be done through performance assessments that only account for 10 percent of a teacher’s ratings. (This formula does not meet the minimum state requirements for implementing the new evaluations, but CPS has some flexibility because the district is only required to use the new evaluations in 300 schools this year.)
For elementary schools, a portion of the student growth measure will be based on value-added test scores from the NWEA, a test that students take three times a year. The NWEA (for Northwest Evaluation Association, a non-profit group that created the assessment) is more difficult than the state’s ISAT, and the results for Chicago can be compared to those nationally. Using this test is a major divergence from past practice and is another signal of the district’s move away from the ISAT and toward tests that provide a better picture of how CPS compares on a national scale.
Chief Education Officer Noemi Donoso pointed out that the NWEA is more aligned to the Common Core, new learning standards that the district will be phasing in during the next several years. An assessment based on the Common Core is still being developed by a consortium of states that belong to PARCC, the Partnership for Assessment of Readiness for College and Careers.
Creating a value-added formula
Value-added test scores try to isolate a teacher’s impact on student performance. The district’s current value-added formula, for the ISAT, takes into consideration a number of factors including where students start, a school’s racial makeup, students’ native language, and the number of low-income children in a school.
Donoso said district officials and union leadership still need to talk about the factors that will be included in the new NWEA value-added formula and how much weight each factor will have. The union’s criticism of value-added scores notes that scores can fluctuate from year to year and that teachers are rated against one another.
The new performance assessments will be developed by groups of teachers and the district over the summer. Donoso stressed that these will not take the form of multiple-choice, standardized tests, but can be a list of items that students should be able to do. Across the district, the performance tasks will be the same, regardless of type of school, Donoso said, and will be given to students at different times throughout the year.
However, she said there might need to be adjustments, if, for instance, everyone in a class walks in able to accomplish all the performance tasks. “These will be rigorous assessments so we don’t anticipate this happening very often,” Donoso said.
One key question is how the evaluations will affect teachers for grades or subjects–such as kindergarten through 2nd grade, physical education, and the arts—in which standardized tests like NWEA are not given. CPS officials decided that the teachers will develop a performance task assessment, but they also will be judged on school-wide literacy scores – even though those scores may be coming from students a teacher has never instructed.
“Increasing literacy is everyone’s job,” Donoso said. She noted that currently, only 17 percent of CPS 3rd-grade students are reading on grade level.
Using school-level value-added scores in cases where teacher-level scores are not available is not uncommon,says Sandi Jacobs, vice president of the National Council on Teacher Quality.
Washington, D.C. public schools do. Tennessee kicked off a teacher rating system that way this year but aims to gradually replace the school-level scores with alternative assessments of students. A number of districts in Texas use school-level value-added scores to determine teacher merit pay awards in non-tested subjects, as does the Teacher Advancement Program (TAP).
“The key thing is how much you’re going to count that, and it seems like what Chicago has proposed is pretty low,” Jacobs says. “When you start to talk about it as a percentage that could influence the overall rating, then I think it becomes more complicated.”
But a policy brief on the issue from the National Comprehensive Center for Teacher Quality, a teacher-quality resource center run by the U.S. Department of Education, notes in a research brief that “this approach presents some additional challenges for a number of reasons, including questions about rigor and comparability when judgments are made about individual teacher performance based on students they never taught.”
Danielson, student achievement linked
CPS’ previous teacher evaluation system had been criticized for decades. Since the 1970s, teachers had been rated on a checklist and the vast majority of teachers received good ratings. The vast number of superior and excellent teachers seemed out of sync with the fact that CPS students, on average lagged in performance.
Winckler and Donoso stressed that, though they didn’t win the support of the union, they took into account many of the union suggestions and recommendations made by teachers through surveys. “We started with the voice of teachers,” Winckler said. “We were listening and learning what excellent teaching looks like.”
Donoso also pointed out that studies have shown a strong correlation between a teacher’s score on the Danielson and how they do on student growth measures. The union had called for using the Danielson framework.
On top of observation and student growth, starting in 2013, students in grades 4 to 12 will be surveyed as to what they think of their teachers. Donoso said that a Bill & Melinda Gates Foundation study also showed that students are good judges of the quality of their teacher.
“This combination will give us an accurate picture,” she said.
It is not clear yet how much the new rating system will cost. But CPS is hiring up to 18 central office employees to help train evaluators and make sure the new ratings are accurate. The advertised salary for the position is $78,700 to $111,000.
Here’s how the rating system will work:
Elementary school ratings
–For the 2012-13 school year, the ratings of elementary teachers in tested subjects will be based 75 percent on observations under the Charlotte Danielson Framework for Teaching, 15 percent on the individual teachers’ value-added scores on the NWEA assessment, and 10 percent on non-multiple-choice student performance assessments designed by teachers.
–In this example, teachers would be rated on a scale of up to 400 points, with 100 points being the lowest possible score. A teacher would get 10 to 40 points for performance assessments and 75 to 300 points for his or her rating on the Danielson observation.
–The 60 possible value-added points would be parceled out according to how far above or below average a teacher is. The lowest fewer-than-1 percent of teachers will get 15 points (the lowest possible score), the middle two-thirds will get 30 to 45 points, and the top fewer-than-1 percent will get 60 points, according to Caref.
–Teachers earning 100 to 219 points would be rated unsatisfactory. Those earning 220 to 284 points would be rated needs improvement. Those garnering 285 to 339 points would be rated proficient and those with at least 340 points would get a rating of excellent.
— Starting in fall 2013, 10 percent of teachers’ ratings will come from student survey results.
— For elementary teachers in tested subjects and grades, the proportion of the ratings determined by student growth will increase to 25 percent for value-added measures and 15 percent for performance tasks over the next five years.
–Elementary teachers in subjects that are not tested, and in kindergarten through 2nd grade, will have 15 percent of their ratings determined by the performance assessments and 10 percent by school-wide value-added scores for literacy. The percentages will gradually increase to 20 percent for performance tasks and 15 percent for school-wide literacy scores by the fifth year.
High school ratings
–For the coming school year only, high school teachers in core subjects will have 90 percent of their ratings based on the Danielson framework and 10 percent of their rating based on student performance assessments. Those in non-core subjects will have their entire rating based on observations.
— Starting in fall 2013, the formula changes. Teachers in tested subjects will have 10 percent of their ratings based on student survey results; 15 percent based on student growth on the EXPLORE, PLAN and ACT standardized tests; 10 percent based on performance tasks; and 65 percent based on classroom observations. Those in non-tested subjects will have 15 percent of their ratings based on performance tasks and 10 percent on school-wide test scores.
— For all high school teachers, the portion of evaluations determined by observations will decrease gradually until the fifth year of the program, when it will be 50 percent for teachers in tested subjects and 55 percent for those in non-tested subjects. The portion of the ratings based on student growth (including both test scores and performance assessments) will increase to 35 percent for teachers in non-tested subjects and 40 percent for those in tested subjects. Student surveys will still account for 10 percent.
Related story: A principal gives thumbs-up to Danielson