May, 2018
We all want our kids and grandkids to have the best education possible, but it is very hard to measure the effectiveness of a school system. We can look at per-pupil spending but there’s no shortage of inefficient systems that spend a lot of money. Student-teacher ratio is easy to measure, but it often doesn’t correlate with good test results.
So we usually look at standardized test results, but these are confounded by the socioeconomic status of the students taking the test. Years of measurements have shown that children from lower socioeconomic backgrounds will, on average, score lower than children from higher socioeconomic backgrounds, no matter how good their schools.
A Stanford professor named Sean Reardon has come to the rescue. He has created what seems to me to be a much better way to judge the effectiveness of a school system.
Reardon looked at the results of three million standardized tests given in the third and eighth grades to students in 11,000 school districts nationwide between 2009 and 2015, and he measured the change that occurs in the five years between the two tests.
So, if the students in the third grade in a school scored on average what you’d expect from students in grade 2.8 and five years later those same group of students, now in the eighth grade, scored on average what you’d expect from students in grade 7.9, the school would be seen to have advanced its students 5.1 grades in the five-year period.
On the other hand, a school where the grade level went from 3.2 to 8.1 in five years, would have advanced only 4.9 years, even though its students scored higher, because only advancement is measured. This removes the effect of socioeconomic status.
Using this system, the highest scoring system was, to everyone’s surprise, Chicago — at 6.2 years advancement in the five-year period (The lowest system, Rochester, NY, scored 2.9). Chicago was a big surprise. “Here’s the third-biggest school system in the country that’s dramatically outperforming not just the other big poor districts, but almost every district in the country, at scale,” Mr. Reardon said. If we understood what was causing that, in Chicago and other disadvantaged but high-growth districts, that might help reduce educational inequality, he said.
A New York Times article describing the study and its results, including speculation as to why Chicago did so well, is here https://www.nytimes.com/interactive/2017/12/05/upshot/a-better-way-to-compare-public-schools.html?hp&action=click&pgtype=Homepage&clickSource=story-heading&module=second-column-region®ion=top-news&WT.nav=top-news
The article contains a look-up feature that allows you to look up your school system and compare its score to that of your neighbors. I did that for my town, Sandwich, MA, and several other towns on Cape Cod and the Islands. Here’s the chart.
Educational Advancement by town, 3rd to 8thgrade
|
Town |
Number Of grades advanced |
National rank percentile |
Median Family income |
Per pupil expenditure |
|
Edgartown |
5.6 |
91st |
n/a |
n/a |
|
Sandwich |
5.3 |
80th |
$96k |
$16,409 |
|
Nantucket |
5.2 |
71st |
$87k |
n/a |
|
Boston |
5.2 |
72nd |
$73k |
$24k |
|
Brewster |
4.5 |
30th |
$66k |
n/a |
|
Falmouth |
4,8 |
52nd |
$81k |
$17,480 |
|
Bourne |
4.3 |
21st |
$80k |
n/a |
|
Dennis-Yar. |
4.3 |
19th |
$60k |
n/a |
|
Mashpee |
4.0 |
10th |
$73k |
$18,087 |
|
Barnstable |
3.9 |
8th |
$74k |
$17,903 |
This research surprised me. It shows a large difference in educational efficiency from the least efficient systems – Barnstable, Mashpee, Dennis-Yarmouth and Bourne — to the most efficient – Edgartown, Sandwich and Nantucket. The lowest scoring system, Barnstable, is in the bottom 10% of school systems in the United States. That is, 91% of the 11,000 systems studied advance their students further than Barnstable between grades three and eight.
Sandwich, on the other hand, is in the top 20%. Its elementary schools advance their students more than 79% of the systems, while spending less per student than Barnstable.
What does Sandwich do that these other towns do not? If one stood at the back of the classroom and watched what was going on, could we tell? If we could figure out what factors make a school system more efficient we could train every system to use them.
Looking at Chicago is not much help. “…It’s hard to untangle what’s been most effective,” said Elaine Allensworth, who leads an education research consortium at the University of Chicago. But she is confident the results are real.
“I go into schools now and I see places that are very different from what I saw 15 years ago,” she said. “It’s much more collaborative among teachers and data-focused, and focused on students.”
Another factor in Chicago may be more autonomous school principles. Teachers are always studied, but the effect of principles on education is often underestimated.
Whatever is going on, we need to figure it out and replicate it before we can actually realize the goal of no student left behind.
