What You Actually Need to Know About the Two New School Choice Attainment Studies

School choice research has increasingly focused on educational attainment—that is, measuring the impact on high school and college graduation, higher education enrollment, employment status and income rather than standardized test scores.

Two new high-quality studies released recently continue this trend. In this post, we’ll break down those studies, highlight the results and contextualize them within the greater school choice research world. Let’s dig in.

The Studies

In the first study, Matt Chingos of the Urban Institute investigated the attainment results of Washington, D.C.’s vouchers—the Opportunity Scholarship Program (OSP). He used National Student Clearinghouse (NSC) data to link college enrollment results with data from OSP lottery participants applying for a scholarship in 2004 or 2005, comparing enrollment data of lottery winners to lottery losers. (It’s worth noting that randomized control trials are considered the gold standard of social science, so this is a solid study.)

In the second study, researchers from the School Choice Demonstration Project led by Patrick Wolf of the University of Arkansas continued to follow a sample of Milwaukee Parental Choice Program students they have been analyzing since 2006. They also used NSC data to measure college enrollment and persistence among grade school and high school participants in the program. Rather than a randomized control trial, Wolf et al compared the results of voucher students to demographically similar non-voucher using students.

The Results

In D.C.

• Chingos found no significant difference in college enrollment between OSP lottery winners (students with vouchers) and losers (students who did not receive vouchers).
• Diving in further, lottery losers enrolled at a rate 3 percentage points higher than lottery winners (46 to 43 percent), but this difference is not considered statistically significant likely due in part to the small sample size (only about 1,600 students total).
• He further split attainment results by enrollment type, including two-year colleges, four-year public colleges and four-year private colleges, but found no significant differences between them.

In Milwaukee

• Wolf and his team found that voucher students enrolled in four-year colleges and persisted in college at higher rates than their non-voucher peers.
• However, there were no significant differences in college graduation
• Voucher students who were ninth graders in 2006 enrolled in four-year colleges at a rate of 44 percent, while non-voucher students enrolled at a rate of 38 percent. There was no difference for two-year colleges.
• The four-year graduation rate was 14 percent for voucher students and 13 percent for non-voucher students—not statistically significant.
• Third through eighth graders in 2006 enrolled in four-year colleges at a rate 5 percentage points higher than non-voucher students, but there was similarly no difference in two-year college enrollment or four-year college graduation rates.

In Context

These two studies make an important contribution to the literature on school choice. Here are our five notable takeaways:

1. The participants in the D.C. voucher study are disadvantaged, shockingly so, in fact. The median household income of voucher lottery winners was only $17,860 (in one of the most expensive cities in America). There has been debate recently as to whether voucher participants are more advantaged than their peers. In D.C., they were not. The D.C. study showed voucher lottery winners had a median household income lower than voucher lottery losers ($17,860 to $18,640).
2. We should always put benefits in perspective with costs. Chingos points out in his conclusion that the OSP voucher amount was, on average, half of what the D.C. Public School district spends on its students. Getting a statistically similar attainment result for half of the price may point toward serious efficiencies within the voucher program. This becomes even more important in Milwaukee, where researchers saw better results from a program that had an average voucher cost of $7,503, while spending $14,156 per year in the traditional public school system.
3. We need to think more about why so many students in Washington, D.C., did not choose to use the voucher that was offered to them, and what it means for future research. Chingos found that about 30 percent of OSP lottery winners didn’t end up using their voucher. Part of this pattern may be due to the choices and policies in Washington. With high-quality charter schools and political wrangling that fueled uncertainty regarding the OSP, a lot of children did not use their vouchers or used them for only a brief time. Considering other research indicates that vouchers help more the longer kids use them and that the comparison group is spread amongst several school sectors, we have to think about how the peculiarities of the program and its context affect our ability to generalize these results.
4. These studies’ results are in line with the attainment trends we see in other school choice programs. In 2013, Josh Cowen and his fellow researchers found that Milwaukee’s voucher students were more likely to graduate from high school and enroll in four-year colleges than their non-voucher peers. Just last year, Chingos’s work in Florida found positive enrollment effects of the state’s tax-credit scholarship program for two-year colleges but not for four-year schools. Check out our School Choice Research SlideShare and blog post detailing the high-quality research to-date. It includes the other studies that list attainment results of school choice participants.
5. It is great that more kids are getting into college, but we do have to think about the really high attrition rate of not just voucher students, but all students in higher education. While 44 percent of voucher students enroll in college, only 14 percent graduate. Non-voucher students in the study also followed this same trend—38 percent enrolled and only 13 percent graduated. Is it really a good thing to get kids into college only to have them drop out with a pile of debt and years of lost earnings? College enrollment and persistence are good tools to measure attainment, but we should be careful when hailing this as a win for public education. This might not have been awesome for the kids involved.

There is never a dull moment in the world of school choice research, and every time a new study launches, we see sweeping statements made about its findings. As EdChoice always preaches, advocates on either side of the issue would be wise to avoid generalizations, look deeper into what these data tell us and focus on solutions.