What You Should Know About the New IES Study of Indiana’s Voucher Program

The federal Institute for Education Sciences, through the Regional Education Laboratory Midwest, recently published a study of educational attainment in Indiana, comparing results of students who attended traditional public schools, public charter schools, private schools via the Indiana Choice Voucher Program, and private schools without a voucher.

The researchers used student-level data from the state department of education and linked data from the state higher education commission to follow the paths of 340,737 Indiana ninth graders in school from 2010–11 to 2013–14. The vast majority (317,367) attended one of 405 public high schools, while 5,820 attended one of 34 charter schools, and 2,201 and 15,529 students attended one of 84 private schools, on vouchers and on their own dime, respectively.

Here are some findings from the report, followed by some criticisms from the research community.

The Institute for Education Sciences study found both charter and voucher students were more likely than traditional public or non-voucher private school students to be:
A) a racial or ethnic minority,
B) more likely to be from low-income households and
C) particularly voucher students were more likely to be English Language learners.

It also found that after background characteristics were accounted for, voucher students were:
A) less likely  to have failed a course,
B) less likely to have been suspended,
C) more likely to have enrolled in college within one year of graduation and
D) more likely to have enrolled in a four-year college rather than a two-year college.

Voucher and charter students were more likely to come from groups defined as disadvantaged.

• While 49 percent of charter students and 30 percent of voucher students identified as Black, only 11 percent of traditional public school students and 4 percent of non-voucher private school students did so.

• While 16 percent of voucher students and 9 percent of charter school students identified as Hispanic, only 8 percent of traditional public and 5 percent of non-voucher private school students did so.

• Fifty-two percent of charter school students and 53 percent of voucher students qualified for free or reduced-price lunch, compared to only 35 percent of traditional public school students and 8 percent of non-voucher students in private schools.

• Fourteen percent of voucher students were identified as English Language Learners, twice the rate of students in traditional public schools (7 percent) and 10 points more than the rate of private, non-voucher students (4 percent).

When looking at attainment outcomes, the researchers looked at 10 indicators of post-secondary success, including if students had failed classes or been suspended, if they took AP tests or earned an honors diploma, if they enrolled in college after graduation and if so, what type of college they chose.

• Compared to students in traditional public schools (and after adjusting for background characteristics) voucher students performed better on four indicators, similarly on five indicators, and worse on one. Charter school students performed better than their peers in district schools on two indicators, similarly on four, and worse on four.

The study had several limitations.

First and most important, this was a correlational, not causal study. That means that while the researchers controlled for observed differences between students that attended different school types (including student demographics such as race and low-income status), they cannot conclude that the differences were caused by the schools they attended. Obtaining causal estimates would require randomly assigning students to the different schooling categories. There might be some unobserved differences that are driving the differences in outcomes. Second, the researchers were missing data on around 16 percent of the students in the sample, including 23 percent of students in charter schools. Finally, the college-enrollment data is limited to Indiana colleges (the data source that the researchers were able to access and link to high school data).

So what can we make of these data? I’d offer three thoughts.

First, in contrast to much of the rhetoric around the voucher program, the students who participate in the state’s voucher program are much more likely to be disadvantaged than the typical traditional public or fee-paying private school student. Almost half of voucher students identify as Black or Hispanic and more than half qualify for free or reduced-price lunch, both a much higher rate than traditional public school students. The rate of English Language Learners in the voucher program is twice that of traditional public schools. Those who want to make the argument that voucher students are somehow more advantaged need to update their talking points.

Second, the pattern of postsecondary success indicators is quite interesting. Voucher students and traditional public school students, for example, passed AP exams, took college entrance exams, and graduated in four years at about the same rate. Where the voucher students pulled away were in course failure and suspension, something that points to some differences in school practices, and college enrollment in general and four-year college enrollment in particular, which points to differences in orientation. And the only indicator where traditional public school students outperformed voucher students was on AP test-taking. Note, there was no difference in AP test passing, so that means that public schools are getting more kids to take AP tests, but they are failing them. It is not clear to me that is anything to celebrate.

Third, I’d be careful reading too much into these findings. As the authors themselves admit, their research strategy prevents them from making particularly strong causal claims. They also have data issues related both to missing data and to limited higher education data. So while these numbers are certainly interesting, it is worth taking them in the context of stronger research on the topic that can make stronger claims as to causality.

If you’d like to read more, friend of EdChoice Patrick Wolf dives into the study over at Education Next. His analysis is well worth the read.