A Good Laugh--A Sigh--And a Revolutionary Suggestion

The Good Laugh

Did you catch Matthew Di Carlo’s perfect April Fools’ Day piece on “Measuring Journalist Quality”? Observing that journalists have been leading the way in “outing” teachers based on unreliable value-added measures (VAMs), Di Carlo flips the picture. What if journalists were measured by similarly unstable “reader-added models” (RAMs)?

The Sigh

Like all fine satire, Di Carlo’s piece carries serious heft.  

Many VAM systems determine teaching quality on the basis of a one-time estimate of a gain score based on students’ performance on a single multiple-choice, standardized test. And researchers have pointed out that even the most sophisticated value-added calculations produce substantial errors: as many as one in four teachers is misidentified as effective or ineffective.

In a recent study of VAM, Heather Hill, Laura Kapitula, and Kristin Umland warned: “Although we do recommend the use of value-added scores in combination with discriminating observation systems, evidence presented here suggests that value-added scores alone are not sufficient to identify teachers for reward, remediation, or removal.”

Sobering—especially when VAM systems are being used to make or break teachers’ reputations.

I’m cheered by the fact that the Measures of Effective Teaching project, sponsored by the Bill & Melinda Gates Foundation, is assembling an array of measures that reflect the complexity of teaching. And Bill Gates, of late, has written his own eloquent plea for journalists to cease their “shameful” approach of using VAM rankings to identify which teachers are effective or not, irrespective of the rating’s accuracy.

Meanwhile, it’s important to note that teachers aren’t opposed to being held accountable. In fact, many teacher leaders we work with are eager for their profession to be more results-oriented.

Teachers are even open to VAM as an element of a comprehensive teacher development and evaluation system. What disturbs them is the idea of VAM as a predominant, automatic marker of their effectiveness—especially if today’s inadequate assessments are its basis.

The Revolutionary Suggestion

Our nation’s schools could cling to the limitations of VAMs, passing swift judgments based on questionable data. Or—prepare yourself for the revolutionary—we could invest in teacher leadership to develop a comprehensive teacher development and evaluation system. (A recent paper, commissioned by the Bill & Melinda Gates Foundation, makes this argument.)

We could draw on the expertise of accomplished teachers to craft better student assessments, additional evaluation measures (like peer observation), and systems that support the professional growth of all teachers (whether in their first year or their twentieth).  

Teachers are well-suited to inform teacher development and evaluation. They understand the nuances of the work they do each day—and can help create equally nuanced systems for gauging and supporting teaching effectiveness. They grasp the complexity of their profession.

Do we get a Day-After-April-Fools’-Day wish? If so, here’s mine: that more journalists grasp the complexity of teaching (and the potential of teacher leadership, too).

Comments

[teaching assistant courses guru]

Wherever in the world you might live in, it’s truly a common problem worldwide that there’s a shortage of teachers. Classrooms are often flooded with students yet there are not enough educators that will be able to at least suffice the overwhelming number of pupils. As a consequence, the quality of education that these learners would receive would probably be greatly compromised.

[Ann O'Connell]

I KNOW how hard it is for those who are charged with observing and collecting data about what teachers do in a classroom, but the simplest of procedures would give myriad information. ASK THE TEACHER WHAT THINKING WENT INTO DESIGNING THE LESSON, MAKING DECISIONS DURING THE LESSON AND WHAT NEXT THE TEACHER WILL DO BASED ON THE ASSESSMENTS S/HE MADE IN THE MIDST OF THE LESSON/S. This is what the National Board does. There is no cynicism about the motives of teachers in the questioning. It recognizes the science of teaching as well as the art and the "caring" accomplished teachers display. And, finally, it's all about the science of teaching as designed by the teacher based on what she diagnoses her many students need given the universal standards of the discipline they teach. No "fly-by" walk-throughs or checklists will ever replace the act of asking what 1500 decisions a teacher made in order to enable learning for each student...because that is how many decisions teachers make per day. If we were CEOs in industry, we'd have all kinds of bonuses and perks. Instead, the current trend is to discount the complexities of what teachers do in order to make learning happen. And, by the way, it is only the teacher who has the responsibility for ACADEMIC progress...despite the fact that they LOVE their students and "care". What teachers REALLY do that no other institution does is make learning happen.

[Barnett]

Ann. So well said. Believe it or not the wisdom of teachers like you is beginning to be heard - and soon to be embraced. Linda Darling-Hammond always reminds me of the words of Langston Hughes in his description of our collective quest to build a better world: “Keep your hand on the plow. Hold on.”

[Claire Rehkopf]

Nowhere in current pedagogy is there significant attention paid to content mastery. Data analysis means little to a bright alternative student relegated by socioeconomic subtleties to a "general" class where differentiation based on Bloom's digital taxonomy enables adolescent teacher bullying, i.e., redirection of teacher attention to behavior management one-on-one in a class of 30+. It's an unnerving, counter-productive burlesque that students have dress rehearsed ad nausea-um for a decade before they even reach high-school.

The single critical determining factor for fostering success will most frequently be passion for content. Not for the student. Not for the student's data. Passion for the content of that discipline being introduced and explored is the single most deterministic factor for student motivation.

We purport to make institutional incarceration palatable by subjecting ascending tiers of wardens to review by highly paid RttT program/project managers. These government-endorsed pseudo-academic psycometricians are just our century's version of the perennial political/academic charlatan. They are our earnestly naive, yet ambitious carpet baggers, redirecting much needed teacher pay increases and facility improvement dollars into the Italian leather wallets/designer bags of those who could never survive in an American classroom and never intended to try. Neither curriculum specialists, nor software content developers, not "Google"-savvy formative assessment formulators, nor any other element of out-of-the-classroom 50-percenters will ever reform education. This is why.

No one individual is responsible for an others education. Responsibility lies with the learner. Consequences for failure to meet entry-level requirements to a university, an apprenticeship, a residency, an honors program are meted out to those who fail, not to those who prepared the curriculum. Not to those who rated the rate of a failure's locus of origin.
You can turn a student on by ignoring the demands of pop pedagogy and demonstrating awe. You can strike a chord and elicit resonance if you know the vehicle and the tenor. A student with desire, focus, and a knowledgeable guide will earn entry. The masses who have been differentiated into myriads of unique explicable products of faulty teaching will EACH fail by an act of will. They were trained up to it so to speak. Please. Go inside a classroom and practice whatever you can manage to love. Up the student-teacher ratio. Think of it. All of those theorists and analysts INSIDE the classroom. Stop bludgeoning the idiot savants who don't want to lead teachers, who just want to teach. Suspend the Common Core re-education Camps. Hire the knowledgeable who pass a reasonable Subject Praxis and a pedophilia profile bar. It doesn't take a village, or thousands of university departments of education. "It is not in our stars, but in ourselves . . ."