This fall, CPS launched “performance tasks” – short assessments for nearly every subject and grade level that will be used in teacher evaluations.
The new tests are part of a state law requiring that teacher evaluations incorporate student performance, and most of the public discussion so far has focused on the use of standardized test scores in evaluations. But districts around the state will soon have to roll out similar assessments as part of the Performance Evaluation Reform Act of 2010.
Teachers contacted by Catalyst Chicago said the new tests may end up being a better gauge of student learning than standardized tests like the ISAT.
But they also cited several problems, including:
*Mistakes in answer keys and student instructions.
*Problems with delivery of materials and with formatting—for instance, a 1st-grade assessment in small print students were not accustomed to reading, with little space for young students’ oversized handwriting.
*Concern that some teachers might cheat, since they are responsible for scoring the assessments and reporting their own data.
The performance tasks will count for 10 to 15 percent of elementary teachers’ evaluations this year, and 10 percent of the evaluations of high school core subject teachers. But, since this is a pilot year for the new evaluations, only untenured teachers will be formally evaluated using them.
Roosevelt High School chemistry teacher Theo Pinson, a participant in the Chicago Teachers Union’s Quest Center professional development program, says that the new assessment “was a lot more directly tied to what they were doing in the classroom” than standardized tests are, though he felt the activity didn’t incorporate enough inquiry and critical thinking – asking students to follow a set procedure, instead of designing their own experiments.
Based on Common Core
CPS says that 450 teachers applied to help develop the performance tasks, and 151 participated in the entire process. In some cases, according to teachers who helped develop the assessments, they did not write the questions themselves but rather selected them from a bank of test questions CPS purchased.
The 93 assessments they developed span the elementary grades, as well as core subjects like reading, math, science and social science in high school.
Because the tests are short, they can only touch a small slice of the material teachers are supposed to cover in a year. They are based on the Common Core State Standards for each grade level in reading and math, and in other subjects are based on similar nationally recognized standards.
The assessments will be at the same level at the end of the year as at the beginning, and will cover the same skills, since students are being tested on how much they learn from the start of the year to the end.
Students’ scores will have to increase at least one point on a 4-point rubric to count as showing growth. If a student earns a perfect score at both the start and the end of the year, that will also count as growth.
Teachers point out gaps
Special education students can be given accommodations like extra time, or having a text read aloud to them. “They are more like an assignment given on a daily basis in a class,” said a CPS official who worked with teachers to develop the assessments.
In cases where students have severe and profound disabilities, teachers may have to create their own performance tasks around a student’s goals for the year.
But Samantha Randall, an early childhood special education teacher at Fermi Elementary, says there was “minimal to no support” in how to modify preschool performance tasks to fit her students.
LaWanda Taylor, a math teacher at Gwendolyn Brooks College Prep, says that there was no performance task for pre-calculus, so her class had to take a performance task in advanced algebra – a different subject – instead.
In some instances, teachers with multiple sections of students were instructed to pick just one class of students to assess. In others, teachers were instructed to assess all their classes and then choose which data to enter.
The district is spot-checking the scores for reliability and consistency, and says it has found no problems so far.
Reliable measure of learning?
CPS says the problems stem from the short time frame for developing the assessments. The district says just half a dozen tests had errors, and most were in difficult high school math and science subjects.
Carol Caref, Quest Center Coordinator at the Chicago Teachers Union, says that the union has gotten “mixed feedback” so far about the tests from teachers.
“The problem is not the concept of using a performance task rather than a multiple choice test. The problem is that there was not time to adequately test them in the field and modify them accordingly,” Caref says. She points out that CPS may modify some tasks before the spring, but that the district’s ability to do this is limited because the tests are supposed to measure student growth over time.
And as with other assessments, the performance tasks’ accuracy may be frustrated by the wide gaps in CPS students’ learning.
Lillian Kass, who teaches high school English at Austin Polytech, points out that some students may be so far below grade level that they miss the performance task goal entirely.
“The reading level was so far above where my students were at they were unable to complete the task,” Kass says. “My students can grow, and still get a zero again at the end of the year. I am not necessarily going to be able to show growth.”
Paul Zavitkovsky, a leadership coach at the University of Illinois-Chicago’s Center for Urban Education Leadership, says that the fact the assessments are being used for teacher evaluation may detract from teachers’ ability to use them to measure learning.
He explains that it could shift the focus to, “Did my kids get it right or did they not get it right?” rather than “What can I learn as an adult in order to help my students learn more effectively?”
He adds that the assessments “are designed to provide more balance in the assessment process. Everybody gets that you need to have multiple data points to come up with a reasonable assessment for adults and kids.”
But particularly for those performance tasks that end up being designed by individual teachers, Zavitkovsky says, so few students are taking them that it might be hard to tell whether the questions are useful.
Having teachers grade the test themselves, as is the case in CPS, opens the door to unreliability, Zavitkovsky says.
He adds that problems like those pointed out by Kass are typical until a test can be tried out on students.
“It is really hard to know if you’re assuming too much about children’s knowledge… if it’s a poorly written item, or too hard, or makes kids and teachers span too big a range of learning,” he says. “Assessment is a lot more complex on its face than it seems… (it) requires some deep knowledge about how students grow into levels of complexity within a discipline.”