Is Swedish School Choice Disastrous – or Is the Reading of the Evidence?

Editor’s note: Some of the author’s sources link to studies that appear in Swedish.

Debate over “The Girl with the Dragon Tattoo” and which film is better—America’s version or Sweden’s—comes down to one’s opinion. The same can’t be said for disagreements over Swedish school choice. Upon investigating the available research—just as Stieg Larsson’s protagonist would—the conclusion is quite clear.

Few issues in education policy divide researchers, pundits, and the general public as much as school choice. Among those who oppose the reform, the case of Sweden’s school choice system is a particular horror story. Just look, they say, at the country’s sliding international test scores and how they coincide with the introduction of choice and privately operated, publicly funded “free schools,” or as we’ll call them, private schools. Surely Swedish school choice cannot be anything but a full-fledged flop?

Well, not necessarily. As any decent social scientist should be quick to point out, correlation isn’t causation. In the media and in politics, however, that important fact is often ignored.

As was the case in Columbia University professor Ray Fisman’s recent Slate article, which broadly argues that school choice reforms have been one reason behind Sweden’s fall in international test scores. While he cites the country’s low scores and one Swedish school choice study finding that teachers in private schools set higher marks than those in public schools, compared with external examiners, Fisman doesn’t discuss in detail the existing economic research on the direct effects of choice on outcomes in Sweden.

What that academic research on the Swedish school choice reform finds is at least some positive effects, albeit somewhat small, for student test scores.

In the most rigorous available study on the impact of private school competition in elementary school specifically, economists Anders Böhlmark and Mikael Lindahl find both short-term positive effects on test scores and grades, as well as on long-term outcomes such as marks in upper-secondary school and university credits. The results indicate that a 10-percentage-point increase in the share of private school students raised short-term and long-term outcomes by about 5 percent of one standard deviation. The impact isn’t large, but it’s there. Profit-making and non-profit private schools have the same effects. And interestingly, the largest part of the impact occurs not because private schools are better than public schools, but because public schools react positively to external competition.

Are those effects merely caused by grade inflation? As indicated by the existence of long-term effects, the answer is “no.”

How Vouchers Actually Cushioned Sweden’s International Test Score Drop

The conclusion from Böhlmark and Lindahl that there is no evidence competition causes grade inflation also supports, broadly speaking, economist Jonas Vlachos’s earlier findings, which showed a minimal effect of private school competition on grade inflation. The small discrepancy between the studies can be explained by the fact that Böhlmark and Lindahl analyzed data between 1992 and 2009, whereas Vlachos analyzed data between 1998 and 2008. Over the period assessed by the former authors, there’s no impact of private school competition on grade inflation.

Three other pieces of evidence also suggest grade inflation can’t explain the positive impact of competition.

First, in the Swedish version of their paper, Böhlmark and Lindahl refer to the 2012 grade inflation report that Fisman cites. But unlike Fisman, the authors delve a little deeper into the differences in grading standards between private schools and public schools depending on subject and level of education.

As it turns out, in ninth-grade national proficiency tests in English and mathematics—the subjects studied by Böhlmark and Lindahl—private schools did not set higher grades compared with public schools overall. In mathematics, there was no difference at all, and in English private schools actually set lower grades. As Böhlmark and Lindahl point out, if anything, this suggests they undervalue the impact of private school competition on English test scores. Similarly, in the 2013 version of the grade inflation paper, although the analyses are available only at the sub-test level, there seems to be little difference in terms of grading practices between private and public schools in ninth-grade mathematics and English overall.

Second, the authors found no evidence that higher private school enrollment increased the discrepancy between internal and external examiners’ grades overall in the municipality. If anything, they argue, “the results suggest that public schools in municipalities with a large share of students in private schools are stricter in their grading practices than public schools in other municipalities.” This further supports the argument that grade inflation doesn’t explain the positive effects of competition found by Böhlmark and Lindahl.

Third, in an intriguing exercise, those authors also turned their attention to the issue of whether the positive effects on Swedish domestic achievement and attainment measures are replicated in TIMSS, one of the main international tests. They found that competition does indeed produce higher achievement in TIMSS mathematics and science tests, and that the effects are about three times stronger than on the domestic measures. So, in fact, the evidence suggests the entrance of private schools has dampened Sweden’s fall in international test scores.

That finding is backed up by the strongest available cross-country study on the subject, by economists Martin West and Ludger Woessmann, who found that higher independent school enrollment improves countries’ performance on PISA, the most hyped international league table, and that the benefits accrue to students in public and independent schools equally. It is also supported by other research on Swedish school choice more broadly, including choice between public schools, which found a small positive effect in the short run but less evidence of any long-term effects.

Why Grade Inflation Shows Why System Design Matters

Now, although the research is positive, there are still significant flaws in the Swedish education system, and Fisman rightly points out some of them. But these have little to do with school choice and more so with overall institutional design, as my colleague Tino Sanandaji recently noted at National Review.

When Swedish politicians implemented school choice in the early 1990s, they also introduced a swathe of other, often misguided, reforms. For example, the norm-referenced grading system was abolished, giving freedom to schools to set grades however they wanted, since external moderation is basically non-existent. At the same time, high schools and universities were still forced to treat these grades equally in a highly centralized admissions process, which almost exclusively relies upon those grades. That inconsistent high-stakes grading culture is a problem because parents and pupils can exert significant pressure on schools to raise grades irrespective of choice and competition.

And, of course, if inflationary pressures have more to do with “voice” than choice, the effect of competition on domestic achievement measures may be undervalued. This is because parents and students in non-competitive areas may “compensate” for lacking competitive pressures among schools to raise achievement properly by exerting more pressure on schools to engage in grade inflation, thereby making the achievement gap with competitive areas appear smaller than it actually is. Indeed, the larger positive impact in the low-stakes and externally graded TIMSS tests, compared with domestic achievement measures, supports that story. So does the fact that public schools appear to respond to competitive pressures by making their grading practices stricter.

Regardless, the combination of decentralized grading practices and the highly centralized admissions process is just one example of problematic features in Sweden’s education system; it’s increasingly evident that Swedish politicians had no clear plan to produce a functioning education market when they introduced the choice reforms in the early 1990s. The result has been an incoherent mishmash of decentralization and centralization, which could never target quality deficiencies more than marginally. The fact the available research suggests small to moderate positive effects of Swedish choice and competition despite these problems is therefore quite remarkable.

But institutional design is not only relevant in the Swedish context. As LSE professor Julian Le Grand and I have discussed elsewhere, it is difficult to make sweeping statements regarding the efficacy or inefficacy of education markets without discussing system design.

All competition is simply not equal. This fact was one of the motivations behind my book, reviewed by Harvard professor Paul Peterson and World Bank economist Harry Patrinos, which discusses hundreds of studies on school choice and competition worldwide, while also paying attention to the incentives the different systems produce. Although the research is somewhat mixed depending on methodological approach and program studied, it is clear there is very little evidence of any negative effects of choice and competition on education quality. Most research finds small to moderate positive effects (or none at all), despite the problems of design evident everywhere. This further contradicts Fisman’s claim that school choice is a part explanation behind Sweden’s fall in international league tables. (The Swedish and international evidence on segregation and educational equality is a bit more complicated, and is more thoroughly explored in this paper.)

Ultimately, again, there’s no rigorous evidence that Sweden’s flawed school choice regime has contributed to its decline in international tests. If anything, research suggests it has cushioned that fall—a remarkable result given its many problematic features.

What Caused Sweden’s Drop in International Performance

But if school choice doesn’t explain Sweden’s lagging education performance, what does? There is no conclusive answer to this question yet, but there are strong indications related to wholesale changes in pedagogical approaches, based on progressive teaching ideas—another example of misguided, universal reforms carried out at the same time as the expansion of choice.

Indeed, the 1994 national curriculum enshrined the across-the-board changes, which were supposed to “individualize” students’ education. In practice, that meant less teacher-centric instruction, with students working more by themselves. In 2010, reviewing the research and the different explanations for Sweden’s decline in education performance, a group of Swedish researchers concluded these changes are probably the most important cause for the drop. Other suggestions—such as immigration and declining teacher quality—don’t seem to hold up.

Since that investigation, new international research supporting those authors’ story further has emerged. In Quebec, similar policies were implemented wholesale in the early 2000s, which research now shows caused strong declines in achievement in a relatively short time period. If any Swedish education reform has been a disaster, therefore, it seems to have been the move toward less-structured teaching practices in the classroom.

That does not mean all innovative teaching practices are flawed. Rather, we should not just presume such reforms work without first testing them in an experimental process—via both market mechanisms and randomized trials. That certainly has been the case with school choice in the United States.

Why Proper Research Should Talk and Ideologies Should Walk

The sad truth is that education policy in general has rarely been based on proper research, the least in Sweden. Instead, it has been the playground of educationalists with little interest in verifying whether their theories actually hold up to empirical scrutiny. As Milton Friedman said, “One of the great mistakes is to judge policies and programs by their intentions rather than their results.”

So what is the alternative? Is it hoping that politicians have the right incentives and information necessary to go through what Fisman describes as “a hard, messy, complicated process” to produce higher education quality? The history of monopolies’ poor record—epitomized in education by the fact that formal K–12 schooling has looked pretty much the same for the past 100 years—suggests not.

Of course, there are good theoretical arguments against the ideal type of education markets, some of which I can agree with. But that doesn’t mean we should throw out the baby with the bathwater. All markets have rules that determine the playing field, which in turn influences outcomes; these rules are especially important in education “quasi-markets,” characterized by public funding and user choice. At the same time, the lack of research on the ideal version, due to the fact that choice programs worldwide to date have been far from ideal, is a problem. If we haven’t properly tested a more or less free market in education, how can we know for sure that it wouldn’t work?

Indeed, finding out what works is part of the hard, messy, and complicated process of experimentation, which must be carried out in the market as well as in education policymaking. Regardless of what specific features we think are necessary, suppressing market incentives is hardly the best option. Those incentives may or may not be a panacea. But they’re quite certainly part of the solution—as we have in fact discovered in Sweden.