How do you judge an external partner? Not surprisingly, it depends on whom you ask. Here are some of the answers Catalyst got to that question.
By school change. “The evidence of our external partner’s success is in the building,” says Louis Hall, principal of Raymond Elementary in the Douglas community. “You see it in the classroom because there’s more collaboration between teachers.”
By test scores. “Principals are looking at test scores, so we are looking at that, too,” says John MacDougall, associate director of Roosevelt University’s Chicago Education Alliance. “Still, we are trying to create a culture in the school and train teachers to change their mindset, and that may not immediately show up on test results.”
By students’ work. “We’ve been working on school reform for a long time,” says Steve Zemelman, project director of National-Louis University’s Center for City Schools. “If I see kids that are more engaged in writing and in math, but the scores are not up. …” He pauses. “Well, I’m accountable to myself. If I see good stuff happening, I’ll stick with what I’m doing. The kids are what’s most important, and sometimes we lose sight of that.”
By contract compliance (and test scores). The Office of Accountability holds monthly meetings with probation managers and external partners to see whether partners are doing what they said they would do in their service contracts. In addition, 12 intervention staff members are assigned to nine schools each, and they visit their schools daily. “There were some instances, where services didn’t happen and that agreement had to be prorated before the external partner was compensated,” says intervention director Albert Foster.
Chief Accountability Officer Philip Hansen notes that the probation manager also keeps a separate checklist on the external partner’s progress.
However, Marie Ann Donovan, an assistant professor of early childhood and reading education at DePaul University and a former external partner, says it doesn’t take a lot to be seen as accountable using this system.
“An external partner can check off that they were at the school weekly, but if you have a big school with over 1,000 students, are all those classrooms going to be visited and monitored? Probably not, but the checklist says they were there,” she says. “The checklist can be checked off as satisfactory, but the reporting format doesn’t describe what the external partner is doing.”
Another school observer, who asked not to be identified, says that some probation managers “don’t show up often and when they do, sometimes they have a totally different philosophy from the external partner about how to do things, and schools get conflicting messages about what to do.”
Hansen acknowledges that he’s heard similar complaints and that his office is looking at other ways to monitor external partners.
“We have been concerned. so this year, we are actively looking for an outside evaluation to be done for elementary schools,” says Hansen. “We also asked Fred Hess, [a professor at Northwestern University] to look at external partners at the high school level as part of his high school restructuring study [which has not yet been released].”
By whether schools want them. “The real test is if schools will continue their relationships with partners after they are off probation and off the [state] watch list,” adds Foster.
For example, Chase Elementary, which got off probation in September 1997, still has a relationship with former external partner Bernard Spillman from New York University’s School Change & Inquiry Program. “I still meet with the principal and assistant principal once a month for breakfast, and I plan to observe a couple of classrooms and meet with the administrative team,” says Spillman, adding, “It’s all strictly pro bono.”
Samuel Stringfield, a principal research scientist at the Center for the Social Organization of Schools at Johns Hopkins University, agrees that it’s hard to judge external partners and assign credit or blame.
“A school may be dysfunctional and when an external partner comes in, it is rejected,” he notes. “Or the external partner may be dysfunctional. Or it could be a good partner, but not a good partner for a particular school. Each of these combinations is happening somewhere all over the nation. The question is to find out which scenario is playing out.”
“Before I got married, I’d gone out on dates and while the person may have been a lovely person, we were not well matched, even though our friends thought we would be,” he observes. “The same things can happen to schools.”