I took a most unscientific stab at this topic in my column this week. I noted the shortcomings in a University of Arkansas ranking published in the Arkansas Democrat-Gazette that assigned a number score to every school in Arkansas based on the national Iowa Test. I took it a step further by looking at additional UA research assessing those test scores according to income level of the schools' students. Cherry picking, I found that the eStem charter schools in Little Rock and the KIPP charters in the Delta didn't stand out to the extent you might gather given the publicity they receive in the daily newspaper. eStem, particularly, was well behind the average scores of schools in its economic group.
I didn't intend to take that farther than being an example of being wary of using a one-size-fits-all number to measure charter school success against other schools.
Now comes some experts, writing in Science, from a more learned perspective but drawing some similar conclusions.
Writing in the journal Science, UC San Diego educational economist JuIian Betts and Richard Atkinson, president emeritus of the University of California and former director of the National Science Foundation, find that most studies of charter schools "use unsophisticated methods that tell us little about causal effects."
The data just isn't there to make comparisons many have made.
Most studies take a simple snapshot of achievement at a charter school, reading and math scores in the spring, say, and compare these to scores at a nearby traditional public school. A study of this sort, Betts said, is "naïve and essentially meaningless." [Little Rock charter schools and their cheerleaders do this all the time.]
The researchers find some value in studies of students in highly prized charter schools that use lotteries for admission where those students are compared with students who didn't draw into the schools. But even these studies have not been very representative, the writers say. They urge so-called "value-added" testing that can measure progression of students. This, among others, would require routine access to student data. Note, please, that Attorney General Dustin McDaniel and the state Education Department continue to fight to the death to release such data to the Little Rock School District in its federal court effort to determine the impact open enrollment charter schools have had on draining better students from the Little Rock School District.