Last week another study surfaced suggesting students of National Board Certified Teachers do not perform any better on standardized tests than their peers taught by non-NBCTs. I suspect that the friends of the National Board are fretting, after a spate of research reports affirming the positive influence NBCTs have in helping students achieve higher standardized test score results. Similarly, I suspect that the foes of the National Board are smiling, hoping to have any ammunition to derail efforts in this country to professionalize and empower teachers.
The most recent study, a well-thought out investigation conducted by Wendy McColske of SERVE (regional education lab in the Southeast), and James H. Stronge of William and Mary College, revealed there was little, if any, statistical difference between the teaching performances as well as the math and reading test scores of NBCTs and non-NBCTs in three North Carolina school districts. Before writing a post-mortem for the National Board, education journalists and the usual policy pundits may want to take a deeper look at the study itself.
Granted, the researchers went to great lengths to design a study that not only examined test score results but also the teaching practices of NBCTs as well as those of non-NBCTs who both produced high test scores and those who did not. But, the researchers offered several caveats to their readers — one of which was their small sample sizes. The research team sought to include all 5th grade teachers from 4 school districts (two urban and two rural), but only three could surface matched test data. The ensuing test score analysis, designed to measure gains of the teachers’ students from the 4th to 5th grade, could only be performed on 307 teachers, of which only 25 were NBCTs. One should not jump to too many conclusions from a quantitative study that draws on test scores data from just two dozen NBCTs. Dan Goldhaber, in his study that found students of NBCTs outperforming those of non-NBCTs, over 600,000 student and teacher records were examined over a three-year period, spanning multiple school grade levels.
To be sure, the SERVE and William & Mary study, went to painstakingly lengths to go beyond test score results by examining the quality of teaching (e.g, planning and assessment, time on task) and student assignments through interviews, surveys, artifacts, and observations. However, this part of the study also suffered from the difficulties getting sufficient numbers of teachers to participate. From the four districts only 51 teachers played a parted, of which 21 were NBCTs (and 16 were non-NBCTs whose students scored in the upper quartile and 14 were non-NBCTs whose students scored in the lower quartile). In interpreting the analysis of the teaching practices, very small sample sizes made statistical significance a remote possibility.
Nevertheless, the non-NBCTs looked slightly better on a range of teaching observations while the NBCTs consistently assigned students work that presented greater “cognitive challenge” and then offered more “meaningful feedback.” Both NBCTs and the upper-quartile scoring non-NBCTs consistently used more sophisticated and known effective teaching practices than those of the lower-scoring non-NBCTs.
However, several other methodological issues are worth mentioning:
1. In using standardized test score data from a state like North Carolina to judge the effects of individual teachers, policymakers (and researchers too) must be cognizant of the fact that in many cases no one individual teacher is totally responsible for the results of a group of students — even in elementary schools with self-contained classrooms. Often times, special education and bilingual teachers play a major role in improving the test scores of students who are in the homerooms of their official math and reading teachers. Not accounting for the assistance that some homeroom teachers receive and others do not can have a huge influence on the outcome of studies like this one (especially with only 300 subjects).
2. In interpreting the test score results careful consideration must be given to the fact that the researchers had no way to control for the expertise of the non-NBCTs. These teachers who produced greater one-year gains could have been identified as NBCTs, if they chose to seek the advanced certificate. Remember National Board Certification is a voluntary process, and there are many teachers who may end up in the “control group” for a study like this one who would be a Board Certified teacher if they took the time and exerted the effort required. Not accounting for the expertise of non-NBCTs also can have a huge influence on the outcome of studies like this one.
My take on these issues does not mean that policymakers and the National Board for Professional Teaching Standards itself should ignore studies such as this one (as well as the previous one conducted by Bill Sanders) that has surfaced less than positive findings. In fact, the NBPTS should be more proactive in using these findings to improve their assessment process as well as help researchers surface even more robust data (with many fewer caveats concerning their methods and results). There are ways that the National Board process can and should draw on student learning results (and not just standardized tests which can be attributed at best to a minority of our nation’s teachers) more effectively.
But most importantly, policymakers must remember that no one study should determine public policy. The majority of research conducted so far still reveals that NBCTs do make a difference for student learning and that compared to other forms of professional development and the hodge-podge of master degrees that teachers earn (and policymakers pay for in most cases), the National Board assessment process can be quite cost-effective. Teachers who go through the process routinely claim that the National Board for Professional Teaching Standards assessment process is the best professional development they have ever experienced. Perhaps, policymakers should look not only look at the research findings but also listen to the voices of the teachers themselves, considering more carefully what they say about policy and practice.
##
