Columns » Ernest Dumas

Testing flunks

by and


Delicious irony is a trite expression but it describes perfectly the circumstance when privately financed university research unwittingly turns the tables on its sponsors.

That is the only description for the first big report of the celebrated new Department of Education Reform at the University of Arkansas, which purports to show how the state should measure the quality of teaching in Arkansas’s 1,116 public schools and how it should pass out financial rewards and penalties for good and poor teaching.

Like the corporate foundation benefactors whose $20 million endowed the school-reform department, the high-powered researchers recruited by the university last year are champions of school vouchers, private education, charter schools, scads of high-stakes tests and merit pay for teachers and also opponents of higher funding for public education — in short, the whole right-wing agenda for the public schools except mandatory prayer, Bible reading and creation science.

“The School Performance Index in Arkansas” does not help any of those causes. In fact, it demolishes just about all of them politically, particularly the notion that teachers ought to be graded on their students’ standardized test scores and paid accordingly. Gov. Huckabee, a convert to teacher pay based on test scores, will ask the legislature shortly to adopt a merit-pay regimen.

The elaborately charted report directed by Dr. Jay P. Greene, the endowed chair of the department, actually has a flawless premise, which is that you can’t measure teaching effectiveness without knowing the learning problems of the youngsters who happen to be in each classroom. But the professors’ results will outrage most lawmakers. Sunday, the biggest cheerleader of the new department and the right-wing school causes, the Arkansas Democrat Gazette, ridiculed the index. The newspaper’s editorial writer missed the whole point of the study but nevertheless sensed its import: It shot down just about every stand the paper takes on education, including consolidation.

The School Performance Index concluded that the schools that were supposed to have made a mess of education, judging by standardized test scores and financial accounting, actually were the most effective educators in the state. Far and away the best school was Altheimer, which has been on probation for its lousy performance. Others at the top were east and south Arkansas schools with abysmally low average test scores. At the bottom were some of those celebrated for their high quality.

Little Rock’s Meadowcliff Elementary School, for example. Democrat-Gazette Publisher Walter E. Hussman had secretly funded an incentive pay plan for teachers there and grandly announced bonuses because they had sharply raised scores on the standardized test that he measured them by.

Greene’s chart ranked only 32 public schools in Arkansas worse than Meadowcliff. (Right after Hussman announced his bonuses, the state put Meadowcliff on probation for its poor performance. They apparently had taught the test that Hussman valued and not the one the state Education Department used to measure educational progress.)

If the index is valid, the most effective teaching in Arkansas is going on at schools where test scores are low and where teachers have weak credentials and below-average salaries. Scores of schools like that are at the top of the rankings. They rank high because when you factor all the learning problems that children bring to the classroom — extreme poverty and poorly educated single parents — the school adds more value to their education than do teachers in schools with kids of richer, whiter, better-educated and traditional parents.

The research is based on a rational assumption to which every teacher who has spent two weeks in a classroom subscribes: It is easier to teach some kids than others.

Greene’s men tried to factor in the most commonly recognized differences: the percentage of kids in each school who are poor and minorities, the income and education levels of their parents and the percentages of single and married parents of school-aged children. Using those factors, they established an expectation for how much progress each school should make and then measured it against the progress they actually made last year on the Iowa Test of Basic Skills.

As detailed as it was, the study had some amazing flaws. Greene, a fellow at the ultraconservative Manhattan Institute, has insisted that public schools did not need more money, and the study purports to prove that money makes no difference in school quality. Its proof is that the Little Rock School District, which spends more per child than all but a few districts, ranks below average. But the researchers contrived factors dishonestly to skew the results.

The report sharply raises the expectations for Little Rock on standardized tests by using citywide figures on family income, parent-education levels and two-parent households rather than those for the public schools. Most of the children of all those white, relatively affluent college-educated parents are in private schools.

The rankings of individual schools disprove the authors’ own conclusion. Little Rock’s magnet and incentive schools, which receive extra funding, tend to produce far better results than the others. Williams Magnet ranks 19th of 1,116 and all are in the top half of Arkansas schools.

But for the Wal-Mart family and the other benefactors, the landmark study proves one thing: in academic research you don’t always get what you pay for.

Add a comment