The Los Angeles Times might indeed be the nation’s worst newspaper, but yesterday, they did a real service for Los Angeles residents: somehow they got data out of LAUSD, comparing the relevant test scores by individual teacher before the students entered these teachers’ classes, and after. The idea is to make comparisons of teachers based on how the students’ scores change, and to do so within a school. Such a method, while hardly foolproof, helps isolate important aspects of teacher quality, because it helps to control for the fact that so many teachers teach in low-income neighborhoods with social pathologies. The story found several teachers in low-income neighborhoods doing amazing work in bringing up their students’ test scores. In fact, it found that many of the most effective classroom teachers work in the poorest neighborhoods. As they say, read the whole thing.
The story was particularly heartening for me, not simply because of the results, but because a couple of the teachers who did not do so well in the results didn’t complain. They didn’t dismiss the findings. They didn’t act defensively. Here’s one:
Told of The Times’ findings, Smith expressed mild surprise.
“Obviously what I need to do is to look at what I’m doing and take some steps to make sure something changes,” he said.
Here’s another, known as an involved, energetic and caring teacher:
Caruso said the numbers were important and, like several other teachers interviewed, wondered why she hadn’t been shown such data before by anyone in the district.
“For better or worse,” she said, “testing and teacher effectiveness are going to be linked.… If my student test scores show I’m an ineffective teacher, I’d like to know what contributes to it. What do I need to do to bring my average up?”
This is exactly what you want to see. These teachers aren’t complaining. They aren’t making excuses. They are dedicated, and they seem to care about their students. Something seems not to be working, and they want to know more.
And then there is A.J. Duffy, president of United Teachers Los Angeles, who reacted to yesterday’s story this way:
The Los Angeles teachers union president said Sunday he was organizing a “massive boycott” of The Times after the newspaper began publishing a series of articles that uses student test scores to estimate the effectiveness of district teachers.
“You’re leading people in a dangerous direction, making it seem like you can judge the quality of a teacher by … a test,” said A.J. Duffy, president of United Teachers Los Angeles, which has more than 40,000 members.
Duffy said he would urge other labor groups to ask their members to cancel their subscriptions.
This is a classic in what some have called “reactionary liberalism” — a defense of an interest group with bromides and talking points.
If progressives want to reinstitute faith in government, then we must demand the best possible results from public institutions. And we also need to confront directly dinosaurs like Duffy who simply refuse to accept any accountability for his profession. President Obama deserves a lot of credit for taking on the education establishment nationally, and the Times deserves a lot of credit for publishing this report.
It would be nice to have an adult conversation about precisely what these scores mean and what they do not mean, how we can help teachers who are underperforming despite their real dedication, and how to weed out those who simply cannot perform well. And we will not have that conversation until we can get rid of people like AJ Duffy.
I think you make a good point. But you also have to acknowledge that teachers are under heavy attack right now, and that in such an environment people tend to get suspicious, reactionary and acrimonious. Much of the liberal establishment seems to have turned on us as well, and are pushing reckless policies without listening. But I agree with you, as a teacher the number one priority I have - and I know this may sound shocking - is that my kids are actually learning. If a standardized test can help facilitate this, then I'm all for it.
Of course, one of the big issues is, as you point out, being able to drill down and separate out the causal factors. Largest among them, yet routinely ignored, is the incredible difference in teaching environment that two schools in the same district might hold. What this means for teachers is that not only is performance going to be much different owing to demographics, but any progress made will have been much more hard won at one school versus another. This one single problem is something that neither NCLB or Race to the Top have dealt with in any serious way.
So while I completely agree with the teacher, and have a similar reaction to AJ Duffy's response, I also know that he is on the front lines, trying to do the heavy lifting that is protecting teacher's genuine and reasonable interests in an increasingly hostile policy environment. I think the type of testing in the story sounds promising, and it ought to be looked at more closely.
What everyone needs to remember is that what the modern public school system is trying to do is nothing short of revolutionary. It is essentially asking generational poverty to be broken on the backs of teachers. And I love this. This is why I became a teacher. But if we are really serious about doing this, we need to take a moment and look at what we are trying to force the system to do. Some teachers at poor schools will be achieving amazing things. The teacher next door may not be. But she may actually be just as competent as the teacher a few schools over who doesn't have to deal with nearly as many issues. The reality is that you just can't expect to put relatively equal resources into schools with wildly different demographics and expect every teacher to be amazing enough to compensate. It isn't fair. You can't build a transformational system around the idea that every teacher in a poor school has to be amazing.
And I guess that's kind of the rub of how teachers are being treated today. We signed up to do this job because we wanted to help, and a certain amount of sacrifice goes with that. But it feels like society in general has seen so many Jaime Escalante movies that they think seem to think if we all aren't working 14 hour days and coming in Saturdays we aren't good enough. Maybe we shouldn't have to, right? Maybe society ought to invest a little more money in its poor schools so that the job requirement isn't super man, but maybe average man.
Imagine if we approached other public sectors this way? What if wars were won, streets were policed, fires were put out, mail delivered, etc. by spending as little as possible (indeed *not* enough to effect real reform), then complaining when the job wasn't getting done that the workers just weren't doing their jobs?
Please offer Eli a regular spot on the RBC blog. Thank you.
AJ Duffy is awful. He is a detriment to children, although he miraculously gets media coverage in which he uses those very children as hostages and shields while he tries to keep ineffective teachers employed doing essentially nothing. He is a detriment to the profession by making it impossible for schools to dismiss teachers who have demonstrated to be ill-suited for the classroom, or have a pending disciplinary action, and fighting tooth and nail for every teacher, regardless of culpability. He is also all that is wrong with the current coalition of Democrats, especially in Sacramento. He is a demagogue who fights against every charter school, fights every attempt to chase after competitive educational grants because it undermines the UTLA, and worse of all, preys on the frustrations and fears of low-income parents to bolster his standing within low income communities as if he was fighting for them, and not his 40,000 members.
I'd love nothing more than to see his own members tell him to STFU, and other labor groups to turn a cold shoulder to his needless rabble rousing. A newspaper publishes a story that actually attempts to start the conversation about teacher evaluations, and Duff's response is to burn the paper? What a nutjob.
I'm with JMG: give Eli a regular slot. Attacking Duffy, the lame duck UTLA president, is pointless. Perhaps the adult discussion on testing can start with Richard Rothstein's seminal work on why testing is sad excuse for measuring teacher performance — indeed, why teacher evaluation is such a state of constant disrepair.
Teacher evaluation certainly is in a sad state of disrepair (with a few exceptions that don't seem to spread as they should). But teacher unions are fighting a rear guard action on this because they have never taken the lead. They have not been proactive, they have not moved the issue forward, and they have ended up in the default mode of defending bad teachers instead of at the negotiating table working out a better solution. The AFT's positions have sometimes been better than the NEA's; but on the whole, the issue has been swept under the rug. And I speak as a union supporter, in general.
Ditto on Eli's post.
… I am curious, not being from L.A. — how *does* Mr. Duffy think a teacher's quality can be judged? And what is he doing to make that happen?
I agree with Eli and will state a similar point in (I hope) a slightly different way.
Teachers, and teachers' unions, are reflexively hostile to the idea of teacher performance evaluation because consistently across the last century, those calls were a proxy for making all teachers' lives worse, by cutting pay, cutting staff, and increasing workload.
Eli: "any progress made will have been much more hard won at one school versus another". How come? I would have though that a good teacher in a bad school can more easily make a testable difference than the same teacher in a good school, on the low-hanging fruit principle. That is, once the teacher can get the kids learning in the first place, a not insignificant hurdle. Ideal pupils (bright, self-starting, well-resourced at home) are more or less unteachable; no teacher should be expected to raise the test score of young Einstein fooling around at the back. This isn't to say that we shouldn't focus on improving the bad schools and recognize the difficulties of the teachers working in them.
Eli said:
"Largest among them, yet routinely ignored, is the incredible difference in teaching environment that two schools in the same district might hold. What this means for teachers is that not only is performance going to be much different owing to demographics, but any progress made will have been much more hard won at one school versus another.
Eli, that's true, but it isn't true for the measurements that the Times was using in this survey. They were comparing children in one classroom versus another in schools that were not officially tracked. Now, there may be other factors at work here, but demographics doesn't seem to be one of them.
Jeff, the problem is that AJ Duffy is all-too-typical of UTLA: his predecessors have all been just like him, and his successor will be as well. It's the political culture of that union. Not all teachers' unions are like UTLA, but it is really, really bad.
I think the "low-hanging fruit" theory may be wishful thinking in some of the schools Eli is talking about. If most of the preconditions for decent learning (safe environment at school and outside, decent nutrition, availability of texts and time to study them usw) are being met, a relatively small set of teacher-effectiveness improvements will get you over the last remaining hurdles needed for kids to start learning seriously, but if they're not, a huge improvement in teaching effectiveness may still give you only a tiny improvement in scores.
The last two paragraphs show wisdom.
The dangers of data come when only a narrow set of metrics come to represent the full effect of any institution. A simple test: does the reason why we educate our children relate to the measurements we create to determine if and how well they have been educated? While these tests definitely give us some insight into what happens in a classroom, they do not lead us to evaluate if our schools succeed at why we have them.
Jonathan, that's right. I realized that I hadn't been clear enough. My critique there was of standardized test scores in general and the way most policy uses them, not this specific analysis, which seems pretty promising. When we tend to talk about "good schools vs. bad schools" and "good teachers vs. bad teachers", the demographic realities (of which Paul kindly pointed out a few examples) are ignored.
So while I'm interested, a few things stood out at me in the piece as pretty major errors in interpretation. And that worries me. From my blog post:
"For instance, they made this claim:
Although many parents fixate on picking the right school for their child, it matters far more which teacher the child gets. Teachers had three times as much influence on students' academic development as the school they attend. Yet parents have no access to objective information about individual instructors, and they often have little say in which teacher their child gets.
This doesn't make much sense. We know for a fact that schools in more affluent districts dramatically outperform those from poor ones. Yet while teachers in poor schools do tend to be less experienced, on average you would expect similar rates of good teaching across districts. The Times story supports this. However, the fact remains that poor schools perform worse. The only possible explanation for this is socioeconomic differences, which has be found time and again to have dramatic effects on student readiness and what I like to call Student Capital - a given student's measure of human and social capital resources that facilitate academic agency.
This site, which maps individual school test score data across multiple states, shows clearly that the number one factor driving overall school performance is socioeconomic demographics. I'm not sure how the article can say that teachers had 3x the influence on a student as the school. My guess is that this is a misreading of the data. A poor student may on his own do better at an affluent school, although I'm pretty sure I've read this benefit is actually pretty marginal. But in order to properly test such a claim, you would have to take every student at a poor school and swap them out for every student at an affluent school. Although from what the research tells us, the results just wouldn't be that different. Not only are the teachers going to be of generally similar quality on average - as the findings in this story back up, but the students are still going to possess the same levels of Student Capital as they did before."