Advancing the Teaching Profession

Pay for Performance: Old Thinking Thinks Anew

Am I dreaming? Does Jay Greene now believe that standardized test scores (and value-added analyses) alone cannot be used to judge teachers? Maybe so. Reading his blog, I am struck that Mr. Greene -- a long-time proponent of most-everything-rests-on-test-scores in educational accountability, including how to identify and reward effective classroom teaching -- may be doing a “180.” (Even the folks at Education Sector appear to be moving their thinking.)

Mr. Greene picks up on writer Michael Lewis's description of basketball player Shane Battier. In his New York Times Magazine piece, Mr. Lewis notes that Battier, one of my Duke basketball-crazed wife’s all-time favorites, may not have NBA all-star stats, but when he plays, “his teammates get better, often a lot better, and his opponents get worse — often a lot worse.” Battier “may not grab huge numbers of rebounds, but he has an uncanny ability to improve his teammates’ rebounding….(and while) he doesn’t shoot much, when he does, he takes only the most efficient shots.” Lewis's analysis seems to have hit the bottom of the net for the policy wonks of education. At least the sports-minded ones.
 
Mr. Greene notes that “as we move into the era of value-added analysis for teacher merit pay,” the Lewis piece “provides much food for thought,” and teacher rewards “should not just be based on individual learning gains.” Schools, according to Greene, are “more complex social organizations than basketball teams, so education sabrematicians have a great (deal) of work ahead of them.”  To find out more about how to pay teacher leaders for highly valued, but not necessarily statistically defined performances, Mr. Greene could return to the framework defined by our TeacherSolutions team, who, collectively, have designed a strategy to reward both the Kobe Bryants and Shane Battiers of the teaching profession  -- and find ways to pay more for those teachers who help others get better.

More Evaluation of Mathematica's AC/TC Evaluation

So I have been digging deeper into the recent Mathematica study, "An evaluation of teachers trained through different routes to certification.” In a February 11th post, I pointed out several problems with the study -- which looks at how students of alternative (AC) and traditionally certified (TC) teachers fare academically -- based on their standardized test scores. By assembling findings on the pre- and post-tests that are found in different tables in the report, it appears that the students of AC teachers actually lost ground in most cases while those of TC teachers generally gained -- but only slightly. Frankly, these results would indicate that neither preparation path is very good. Ok. But if you look even more carefully into the study details, you find that the researchers did not calculate measured gain scores for the AC and TC teachers. Instead they looked at whether or not there were statistically significant differences in adjusted post-test scores -- not such a kosher approach if you are looking for meaningful effects.

In addition, the study examined 87 AC teachers, who were in programs that require “low” or “high” amounts of coursework, and compared their effects with the TC teachers. I wonder what the researchers may have found if they actually looked at the content of the coursework the AC and TC teachers took -- not just what was listed in the syllabus. Unfortunately, the researchers drew on the formal description of the programs and the amount of coursework required for certification, not the quality or approach to content the teachers ACTUALLY experienced through the coursework. It seems, for the study's hefty $9 million price tag, the Mathematica team could have addressed these issues, and in doing so, improve our understanding of the effects of teacher education. I wonder why they did not.

Missing the Point on How to Improve Teacher Recruitment and Preparation

Education Week highlighted a just-released study by Mathematica Policy Research, summarizing the report’s findings that student test scores are “unaffected” by whether or not teachers matriculated through university-based traditional (TC) or more short-cut alternative (AC) certification routes. The study’s lead researcher, Jill Constantine, concluded “when students are placed with teachers with alternative routes versus traditional routes [for certification], there’s no harm in terms of student achievement.”  

This is a curious headline: Granted, the study found lots of “no effects” — but the study press release did note that “students of alternatively certified teachers who were taking coursework while teaching scored lower in math than students of their traditionally certified counterparts.”  This may suggest that learning on the job may have its drawbacks. On the other hand, buried in one of the appendices I found that AC candidates, compared to their TC counterparts, were more likely to have a mentor to help them learn to teach on the job — making a mess of trying to compare which approach is better than the other.

In addition, the researchers compare student achievement results from AC and TC teachers who have several years of teaching experience under their belts. However, because AC teachers have much higher attrition in their initial years on the job than TC novices — the researchers may have inadvertently excluded the weakest AC teachers from their sample (more complications).

Also, the researchers compare AC and TC teachers from the same schools. However, AC teachers tend to teach in schools with lesser prepared and qualified colleagues — even ones those who are more traditionally certified. The "same school" design can create a "no effects" finding.

I wonder what the researchers would have discovered if they studied high quality TC programs like those at Stanford, UCLA, Bank Street, and University of Virginia or high quality AC programs like those at the Boston Teacher Residency or the Academy for Urban School Leadership — and compared them to the typical TC and AC programs in their samples.

I wonder what the researchers would have discovered if they had used more than one-year’s worth of student achievement results as measured by one standardized test.

The question should not be whether AC or TC is better — but what do teachers need to learn before they begin to teach — and how they can continue to learn as they develop as expert teachers. No Child Left Behind, flawed as it is, did raise the expectations for students’ preparation to enter the world of work. It’s time for us to raise the expectations for teachers’ preparation to enter the work of teaching. We need to do more than conclude that one teacher recruitment and preparation approach or another will do “no harm” to students. Our students expect better from us, as they should.

And BTW, the study cost $9 million. We could recruit and support a large number of well-prepared teachers with that kind of cash.   

How and Why Teacher Preparation Makes a Difference

At the annual meeting of the American Association of Colleges of Teacher Education (AACTE) this weekend, researchers Jim Wyckoff (who specializes in labor economics) and Pam Grossman (who specializes in teacher learning) presented more findings from their “pathways” project. The project combines a wide range of data on who enters teaching and from what route (university, Teach for America, emergency credentialing, etc.) — and where they teach and how good they are.

The research is still in its “infancy,” and the researchers offer a wide number of caveats. However, they presented compelling insights into how and why teacher preparation makes a difference —drawing on data on the characteristics of programs and the value-added student achievement gains of their graduates who now teach in New York City schools.

It appears that effective teacher education programs provide:

• Well-supervised student teaching where there is strong “congruence” between the training experience and the first year teaching assignment;

 • Opportunities “to engage in the actual practices involved in teaching”  (e.g., lesson study);

 • Opportunities to study local school curriculum; and

• A capstone experience where action research or data-based portfolio are used to make summative judgments about the quality of the teacher candidate.

Linda Darling-Hammond delivered the conference's keynote address on Sunday — admonishing teacher educators to begin designing programs with these characteristics or get out of the business. It is time to stop debating whether or not teachers need to be prepared before they begin to teach. It is time to make sure all programs — traditional or alternative — give teacher candidates opportunities to be effective.

Most of all, it's time to truly understand the conditions that allow well-prepared teachers to get better from year to year.

NCTQ Earns a D+ for State Teacher Quality Report

Last week, the advocacy group National Council for Teacher Quality released its report card on state policies that advance toward or retreat from efforts to increase teaching effectiveness. The NCTQ report raises important issues and surfaces relevant facts -- but quickly loses focus in its pursuit of a rather narrow agenda. 

The report’s authors point out that “not a single state encourages local districts to provide significant pay increases to teachers when they are awarded tenure, a milestone in a teacher’s career that should be significant, but is instead automatic.” Good point. And, as noted in Education Week’s summary , there’s no question that “districts need to better align compensation, professional development, and other aspects of the teacher-quality continuum to student-achievement goals.” 

But NCTQ, from its myopic position inside the beltway, also offers up some poorly focused analyses, eyeing state teacher quality policies with a pronounced ideological squint.  For example, NCTQ report:

•  correctly points out that states do little to emphasize student learning in evaluating teachers and granting them tenure — but ignores the lack of capacity of most principals to get the job done right, or the role that accomplished teachers could play in assessing their peers;

•  correctly points out that states have not done enough to create longitudinal data systems that can identify which teachers are more successful in raising student achievement — but ignores the fact that current standardized tests are insufficient measures to gauge effective teaching in the 21st century;

•  correctly points out that states still allow teachers to teach with emergency licenses — but ignores that they also allow a plethora of poorly designed, short-cut approaches to teacher preparation that leave many students in high needs schools taught by a revolving door of underprepared, inexperienced novices;

•  correctly points out that states short-change new teachers when it comes to induction and mentoring — but ignores the fact that our public schools are not funded sufficiently to support novices, and that long-standing bureaucratic structures that keep our best teachers from routinely spreading their expertise to their less experienced colleagues;

•  correctly points out that states do not provide incentives (differential pay or loan forgiveness) for teachers to teach in high-needs schools —  but ignores the role that professional working conditions play in both recruitment and retention; and, finally,

•  correctly points out that states and their pension systems “disadvantage teachers early in their careers” and inordinately value veterans — but ignores that teaching is complex work, that good teachers become better teachers over time, and that high needs schools require highly experienced and expert teachers to mentor and support their novices.

NCTQ surfaces important information that policymakers and practitioners should consider. However, its flimsy evidentiary base and narrow-eyed assessments leave us with an incomplete story in the current narrative on teacher quality. I would give NCTQ a grade of D+ — recalling this comment from a long-ago teacher of my own: “We both know you can do better.”