Teachers’ education and experience make little difference in students’ math scores, according to a recent study by the Federal Reserve Bank of Chicago.
Analyzing math test score gains for three years worth of Chicago freshmen, researchers found that teachers had an enormous impact on test scores but that their credentials, including their advanced degrees and years of teaching experience, did not.
Whether a teacher had three years in the classroom or 30, a bachelor’s degree or a Ph.D., a degree from a modest local college or an Ivy League university—none of that made a significant difference in student outcomes, the study found.
G. Alfred Hess Jr. of Northwestern University says the findings don’t surprise him. His 2001 study of Chicago high schools also found no link between teacher experience and student achievement. “We saw experienced teachers who were great and experienced teachers who were dreadful,” he recalls.
“We saw new teachers who were great and [ones] who couldn’t control a class to save their lives.”The study raises questions about pay scales, like Chicago’s, that reward teachers based solely on education and experience, says Lisa Barrow, a study co-author and Federal Reserve Bank senior economist. “The compensation structure isn’t set up to reward good teaching,” she observes.
The findings should not be construed to mean that teachers never benefit from additional education. Other research has found that students learn more from high school math teachers who hold a master’s degree in mathematics.
The Fed’s study did not distinguish between master’s degrees in math and master’s degrees in other subjects because that information was not available to researchers. But when it comes to pay raises, the Chicago Public Schools doesn’t make distinctions either. A master’s degree brings an increase in pay regardless of whether it is in the subject area the teacher teaches.
The Fed’s study analyzed data on 53,000 freshmen and 650 math teachers whose classes they attended sometime between 1996-97 and 1998-99.
There was one exception to the overall finding that credentials don’t matter: Students whose teachers had majored in math, science or education as an undergraduate performed slightly better than those whose teachers had other majors.
Individual teachers made a big difference. Students assigned to a teacher who ranked in the top third of the group studied, gained, on average, an extra six months in test score growth, compared to students with teachers in the bottom third. Teachers were rated by how well their students progressed, after accounting for factors in students’ families or in the school environment that might impact test score gains.
Using test scores to judge teachers is tricky, Barrow and co-author Daniel Aaronson agree. For one, a district or principal would need to collect data on at least 200 students for each teacher. Otherwise, outcomes might be due to chance, they say. And an evaluation would have to control for outside variables known to impact student gains, such as family background. “Just looking at raw data is going to get you into tons of trouble,” says Aaronson.
An evaluation must also ferret out those who game the system, he continues. In this study, researchers were able to confirm that teachers rarely encouraged low-achievers to stay home on test day. Aaronson says they still are looking for evidence that teachers may have helped students cheat. They are using a method developed by University of Chicago economist Steven Levitt, who found that cheating on the Iowa Tests of Basic Skills is relatively rare in Chicago. (See Catalyst, September 2001)
Nonetheless, Aaronson believes that student test score gains should be factored into teacher pay and tenure decisions because the same teachers tended to get the best results year after year. “In my mind, this is a no-brainer.”
Allan Odden, a professor of educational administration at the University of Wisconsin-Madison, calls it impractical to judge teachers on test scores. “It’s not only politically controversial, it’s technically difficult—if not impossible,” he says.
Among other complications, he explains, most districts would need to add standardized tests in subjects not traditionally tested—such as music, art and gym—or put those teachers on a separate pay scale.
Instead, Odden suggests retaining a traditional pay scale with extra raises for teachers who receive superior evaluations. Odden helped develop an evaluation system in Cincinnati that rated teachers on their instructional practices and was found to correlate strongly with student test-score gains. Schools in Arlington, Va., have adopted a similar system, and other districts are considering it, he says.
“If you tie pay [only] to education and experience, you tell teachers, ‘To get paid more you have to live long and take any kind of university credit hours you want,'” he adds. “You don’t have to become a better teacher.”