Advancing the Teaching Profession

Amid Evaluation Controversies, Florida Teachers Model Effective Reform

In the world of school reform, there is a lot of talk about teacher evaluation these days. Hardly anyone believes the systems in place are worth much, given that many districts report that administrators often rate almost all teachers “satisfactory” and pay little attention to student results. And most everyone knows the Race to the Top states are rocketing toward implementing new systems, replete with more intensive classroom observations, mechanical use of value-added ratings, and four-point rating scales.

However, the teacher-evaluation engines are already beginning to stall. Tennessee, for example, is struggling to find time for “micromanaged” principals to conduct effective and useful observations. And New York City is rolling out wildly inaccurate* value-added test scores results to judge (and then fire) teachers.

And now bright-eyed enthusiasts for R2T-esque teacher evaluations are raising hard questions about the rigid use of test scores to judge teachers. Many point out that if principals and teachers don’t trust one another, then these evaluation systems won’t do much to improve teaching and learning. Robin Lake, for example, of the Center on Reinventing Public Education (CRPE) at the University of Washington, recently wrote a guest post on Rick Hess's blog. There, she makes the good point that the system requires not just new “tools,” but serious “attention to organizational capacity and relationships.”

Making these relationships a priority will require not just more collaboration between administrators and union leaders, as Robin suggests, but developing new systems so teachers themselves can lead the way. This would allow more teachers like Ryan Kinser of Hillsborough County, Florida (and many of his 15,000 teaching colleagues in the Tampa-area school district), to drive reforms that will affect their classrooms. Ryan, in his powerful blog posts over at transformED, points to a number of huge issues with using value-added results to improve teaching and learning. He also suggests specific strategies for using VAM in ways that make sense for teachers and the students they serve.

Soon, Ryan and his CTQ-supported New Millennium Initiative team will release a powerful multimedia product that focuses on leveraging promising teacher evaluation reforms in their district to transform the teaching profession. In her blog post, Robin Lake cites the research of Tony Bryk on the importance of organizational trust and coherence in much-needed evaluation reforms. In Tampa, the school district and union—and now a growing group of solutions-oriented teachers—are showing just how it can be done.

* It has been shown that the use of “value-added assessments” in New York is grossly imprecise. For example, the margin of error spans 35 percentiles in math and 53 in English. Some teachers' evaluations were based on as few as 10 students.

A Good Laugh--A Sigh--And a Revolutionary Suggestion

The Good Laugh

Did you catch Matthew Di Carlo’s perfect April Fools’ Day piece on “Measuring Journalist Quality”? Observing that journalists have been leading the way in “outing” teachers based on unreliable value-added measures (VAMs), Di Carlo flips the picture. What if journalists were measured by similarly unstable “reader-added models” (RAMs)?

The Sigh

Like all fine satire, Di Carlo’s piece carries serious heft.  

Many VAM systems determine teaching quality on the basis of a one-time estimate of a gain score based on students’ performance on a single multiple-choice, standardized test. And researchers have pointed out that even the most sophisticated value-added calculations produce substantial errors: as many as one in four teachers is misidentified as effective or ineffective.

In a recent study of VAM, Heather Hill, Laura Kapitula, and Kristin Umland warned: “Although we do recommend the use of value-added scores in combination with discriminating observation systems, evidence presented here suggests that value-added scores alone are not sufficient to identify teachers for reward, remediation, or removal.”

Sobering—especially when VAM systems are being used to make or break teachers’ reputations.

I’m cheered by the fact that the Measures of Effective Teaching project, sponsored by the Bill & Melinda Gates Foundation, is assembling an array of measures that reflect the complexity of teaching. And Bill Gates, of late, has written his own eloquent plea for journalists to cease their “shameful” approach of using VAM rankings to identify which teachers are effective or not, irrespective of the rating’s accuracy.

Meanwhile, it’s important to note that teachers aren’t opposed to being held accountable. In fact, many teacher leaders we work with are eager for their profession to be more results-oriented.

Teachers are even open to VAM as an element of a comprehensive teacher development and evaluation system. What disturbs them is the idea of VAM as a predominant, automatic marker of their effectiveness—especially if today’s inadequate assessments are its basis.

The Revolutionary Suggestion

Our nation’s schools could cling to the limitations of VAMs, passing swift judgments based on questionable data. Or—prepare yourself for the revolutionary—we could invest in teacher leadership to develop a comprehensive teacher development and evaluation system. (A recent paper, commissioned by the Bill & Melinda Gates Foundation, makes this argument.)

We could draw on the expertise of accomplished teachers to craft better student assessments, additional evaluation measures (like peer observation), and systems that support the professional growth of all teachers (whether in their first year or their twentieth).  

Teachers are well-suited to inform teacher development and evaluation. They understand the nuances of the work they do each day—and can help create equally nuanced systems for gauging and supporting teaching effectiveness. They grasp the complexity of their profession.

Do we get a Day-After-April-Fools’-Day wish? If so, here’s mine: that more journalists grasp the complexity of teaching (and the potential of teacher leadership, too).