The merit pay train leaves the station from the state Capitol at 2 p.m. tomorrow, when faculty members at the University of Arkansas present an evaluation of merit pay experiments in the Little Rock School District. (Noted: The UA is a major recipient of Walton Foundation money. The department doing the evaluation is a major recipient of Walton money. Walton money is promoting merit pay, charter schools and other "reform" all across America. Walton money is supporting the merit pay experiment in the Little Rock School District, with notable aid from Democrat-Gazette publisher Walter Hussman.)
The finding: No surprise here. Merit pay holds promise. More experimenting and testing are needed.
Pardon my sarcasm. Suspicion naturally arises from the multiple connections and the fact that the publicly financed UA refused our FOI request for this report since last Thursday. The report was provided to certain insiders -- supporters of the program -- and one of them was kind enough to supply a copy to us today. Secrecy about this project has been a black mark on the effort from the start.
I'll put the verbatim executive summary on the jump. The meat comes later, in test score comparisons between two schools using merit pay projects (Wakefield and Meadowcliff) and three not using them (Base Line, Chicot and Franklin). But I'm unable to post that data at the moment. There is, however, very little in the way of apples-to-apples comparison of student test scores. In the end, because of a number of variables, the only true comparison is how fourth- and fifth-grade students at one school, Wakefield, performed on a math test, compared with an average from three non-merit-pay schools. Another flaw, in my view, is that the two merit pay project schools -- with about 80 percent minority student bodies -- are compared against an average score of three schools with differing demographic makeups, including one, Franklin, a virtually all-black school with very low scores. I don't see group-to-group comparisons of individual student advancement -- among poor students at a merit pay school vs. poor at a single comparable school, black vs. black, non-black vs. non-black These schools happen to fall in an area where an interesting demographic difference may be emerging based on some recent data -- the relatively strong performance of the rising Hispanic population, particularly versus black students -- and I'm not sure any thought has been given to that.
Teacher opinion surveys produced a finding that teachers at merit pay schools didn't think they worked harder or more innovatively than other teachers, but they found a more positive atmosphere. Yes, they liked higher pay.
It is a shame that the presentation won't be rolled out in a true open forum. It should have been peer reviewed and distributed to those with differing viewpoints in time for study and informed questioning. But this is a political and public relations process at work tomorrow, with academic researchers as props.
Evaluation of Year One of the Achievement Challenge Pilot Project in the Little Rock Public School District
Joshua H. Barnett, Gary W. Ritter, Marcus A. Winters, & Jay P. Greene
Department of Education Reform
University of Arkansas
Can merit pay address Arkansas’ teacher quality dilemma? Supporters of merit pay programs in education argue that such programs would encourage teachers to be more innovative, work harder, and be more satisfied with their salary. The culmination of these advantages would result in better instruction and improved student achievement. However, opponents argue that such programs would increase negative competition, degrade the school environment, and encourage teachers to neglect low-performing students. The culmination of these disadvantages would result in poor instruction and declining student achievement. Despite the passionate arguments made by merit pay supporters and opponents alike, little is known about the actual impacts of merit pay programs on students or teachers because rigorous evaluations are rare.
The Achievement Challenge Pilot Project (ACPP) operating within the Little Rock School District provides an opportunity to conduct a rigorous evaluation to uncover the effects of a merit pay program on student achievement, as well as on the behaviors and attitudes of teachers with regard to innovation, working harder, satisfaction, competition, environment, and teaching focus. Over the last year, we analyzed data reported by the District as well as data collected from the surveys of teachers. Based on our comparison of students and teachers in the ACPP schools and students and teachers not in the ACPP schools, we note eight key findings:
1. Students in schools where the ACPP operated in 2005-06 showed an improvement of 3.5 normal curve equivalent points. For the average student, this gain represents an improvement of nearly 7 percentile points.
2. Teachers in the merit pay program reported that they were no more innovative than comparison teachers.
3. Teachers in the merit pay program reported that they were more satisfied with their salaries than comparison teachers.
4. Teachers in the merit pay program reported no more counterproductive competition than comparison teachers.
5. Teachers in the merit pay program reported that they were no more likely to work harder than comparison teachers.
6. Teachers in the merit pay program reported that their work environment became more positive than comparison teachers.
7. Teachers in the merit pay program were less likely than comparison teachers to agree that low-performing students were a burden in the classroom.
8. Teachers in the merit pay program were more likely than comparison students to report that the academic performance of their students had improved over the past year.
While the results from this first year study suggest positive impacts of the ACPP, we believe the second year study with five schools involved in the ACPP will greatly assist in expanding on and explaining the first year findings.
About the Authors
Joshua H. Barnett is a Distinguished Doctoral Fellow in the Public Policy Ph.D. program at the University of Arkansas. He has performed studies examining the effects of various education policies including merit pay, school discipline, and school finance in Arkansas, New Jersey, and Philadelphia. He works in the Office for Education Policy at the University of Arkansas. He earned his M.A. in communication studies from New Mexico State University in 2003.
Gary W. Ritter, Ph.D. is an Associate Professor of Education and Public Policy and holder of the Endowed Chair in Education Policy in the Department of Education Reform at the University of Arkansas. He is also the Associate Director of the inter-disciplinary Public Policy Ph.D. program and the Director of the Office for Education Policy at the University of Arkansas. His research interests include program evaluation, standards-based and accountability-based school reform, racial segregation in schools, the impact of pre-school care on school readiness, and school finance. He earned a Ph.D. in Education Policy in 2000 from the Graduate School of Education at the University of Pennsylvania.
Marcus A. Winters is a Doctoral Academy Fellow at the University of Arkansas. He has performed several studies on a variety of education policy issues including high-stakes testing, charter schools, and the effects of vouchers on the public school system. His
op-ed articles have appeared in numerous newspapers, including The Washington Post, USA Today, and the Chicago Sun-Times. He received his B.A. in political science with departmental honors from Ohio University in 2002.
Jay P. Greene, Ph.D., is the Endowed Chair and Head of the Department of Education Reform at the University of Arkansas. He has conducted evaluations of school choice and accountability programs in Florida, Charlotte, Milwaukee, Cleveland, and San Antonio. He has also recently published research on high school graduation rates, social promotion, and special education. His articles have appeared in policy journals, such as The Public Interest, City Journal, and Education Next, in academic journals, such as the Teachers College Record, the Georgetown Public Policy Review, and the British Journal of Political Science, as well as in major newspapers, such as the Wall Street Journal,
The Washington Post, and USA Today. His education research has been cited in
U.S. Supreme Court opinions and has appeared in scholarly and popular publications.
Dr. Greene obtained his doctorate in political science from Harvard University in 1995.