The recent explosion of computing capacity and speed, coupled with the rapidly-rising cost of higher education, have created a “perfect storm” in which traditional institutions are more keen to offer classes online than ever before and where demand for online higher education is high. Meanwhile, the 2006 decision by the U.S. Department of Education (ED) to lift the “50 percent rule” of the Higher Education Act of 1992, thereby allowing institutions offering more than half of all classes through distance education to distribute Title IV student aid, has permitted online only postsecondary institutions to proliferate.[i] By 2013, according to calculations from the ED’s Integrated Postsecondary Education Data System, 11 percent of all U.S. undergraduate degree-seeking students studied in online-only programs, and 27 percent of students took at least some of their classes online.[ii]
Online delivery is clearly a major part of American higher education today, so it is important to know whether online education helps or hurts the students it serves, and whether society benefits from its ubiquity. Deming and collaborators showed that prices are lower in online higher education: The higher the share of students taking all courses online, the lower the tuition and fees charged to students in both public and private-sector institutions.[iii] (Higher shares of students taking some of their courses online do not appear to be associated with lower tuition and fees.)
But is the quality of the online coursework equivalent to that which students would have experienced in face-to-face instruction? It is certainly possible to leverage online or hybrid platforms to deliver remarkably rich and deep higher education; indeed, some of my Northwestern University colleagues and others around the country are working on exactly these types of curricular innovations. However, these innovative uses of technology are by and large not what we’re talking about when we think about the proliferation of low-cost online education circa 2016. High-quality online courses are expensive to deliver—at least as expensive, if not more, to develop and staff than traditional face-to-face instruction.[iv]
The labor market seems to recognize some of the shortcomings of online education. A recent field experiment created fictitious resumes and randomly assigned degrees from for-profit online institutions and nonselective public institutions to “applicants” for real-world position vacancies in business and health fields.[v] Deming, together with a different set of collaborators, found that employers were dramatically less likely to call back applicants with a business bachelor’s degree from a for-profit online institution than those with a degree from a nonselective public institution. For jobs in healthcare, employers were substantially less likely to call back applicants with credentials from a for-profit online institution than those from a public institution—but, importantly, only in cases where the job doesn’t require an external indicator of quality such as a professional license. In cases where this external credential is necessary, the gap in the callback rate between those with for-profit online and nonselective public credentials was dramatically smaller. Taken together, these results suggest that employers are still wary about the skills of those with online higher education credentials (at least those from for-profit institutions).
The employers contacted in that field experiment might be worried about differential selection of students into online education, or they might be concerned about pedagogical shortcomings of online education, or both. There is a growing literature on the second point, and the evidence from experimental studies of online versus face-to-face delivery of the types of courses that are widespread suggests that online delivery may be inferior to face-to-face delivery of the same classes, at least for vulnerable populations.
In my own joint research with Rush and Yin, students in a large introductory microeconomics course at a major public research institution were randomly assigned to face-to-face and online-only settings.[vi] In general, those receiving online instruction fared worse than those receiving face-to-face instruction, but the differences were modest. For Hispanic students, male students, and relatively poorly-prepared students, however, the online disadvantage was considerably greater. Another study at a selective public research university by Alpert, Couch, and Harmon—this time, assigning students at the point of expressing interest in enrolling in the course rather than following enrollment—also showed that students assigned to an online-only introductory microeconomics course performed worse than those assigned to face-to-face instruction.[vii] The institution in the Alpert study also randomly assigned some students to a hybrid environment in which face-to-face instruction was halved, and in the hybrid case there appeared to be little reduction in learning outcomes relative to those in face-to-face-only instruction.
A third study of an introductory statistics course offered at six public universities by Bowen and collaborators also found that students taking a hybrid online/face-to-face course fared no worse than those taking the same course in a face-to-face environment,[viii] but another study by Joyce and collaborators investigating hybrid courses versus face-to-face courses in introductory microeconomics at a selective public institution found evidence of modest performance reductions in the hybrid case.[ix] All of these experiments concern performance in the course in question; another recent study at a selective public university demonstrates that students also perform worse in follow-on courses when their prerequisite course was taken online versus face-to-face.[x] The pattern of reduced outcomes in online-only courses versus face-to-face courses is also present in community colleges,[xi] and, as found by Hart and collaborators, there exists dramatic variation in the (generally negative) effects of online course-taking across different California community colleges.[xii] The patterns are similar in the private sector too: On average, as found by Bettinger and collaborators, DeVry University students perform worse in online settings than in face-to-face instruction.[xiii]
The weight of the evidence, therefore, presents a modestly pessimistic picture of the likely outcomes of students enrolled in online-only instruction. Online delivery, while providing opportunities for increased flexibility and personalization of education, might also exacerbate some of the behavioral barriers that many students face in navigating college. The magnitudes are not trivial: they imply that students who take only online courses would have grade point averages about ten percentage points lower in the distribution than those who took only face-to-face courses, all else equal. But there are, of course, other arguments for online education, even if the quality of this education may be at times inferior.
For instance, it’s possible that online education teaches students skills that are valued in the labor market but are not as often conferred by face-to-face instruction. Right now, we don’t know the answer to that question.
In addition, online programs may increase access to higher education for students for whom—due to spatial or time constraints—traditional face-to-face instruction may be prohibitive. It’s possible that if a very well-regarded institution provides an online-only degree program, that program might substantially increase access while at the same time conferring reputational quality signals that are highly valued in the labor market.
A recent entry into the market for graduate education in computer science provides an opportunity to investigate this possibility in part. In spring 2014, Georgia Tech’s esteemed computer science department began enrolling students in a fully-online version of its top-ranked master’s degree program. Tuition for this program is just one-sixth of what out-of-state students typically pay for the face-to-face version of the program, and the degree conferred is publicly identical to the face-to-face degree.
A just-released study by Goodman, Melkers, and Pallais investigated whether this program increased access to computer science master’s degrees, and the answer appears to be a resounding yes.[xiv] Importantly, they could find no evidence that this program crowded out enrollment elsewhere, suggesting that this program was really satisfying largely unmet demand for graduate work in computer science. They found that the program principally served mid-career Americans, and suggested that this program alone will increase the annual number of American computer science master’s degrees by seven percent. While it’s impossible to know at present whether Georgia Tech’s online-only instruction is as high-quality as its face-to-face instruction—and this is something that, given the extant literature at the undergraduate level, may still be a real concern—this study makes clear that one of the promises of online education (expansion of access to satisfy unmet demand) may well be realized if enough highly-regarded institutions enter the online education marketplace.
Another possible benefit of online education is its potential to provide competition for incumbent educational institutions. There is reason to believe that this might be the case: for instance, in prior research at the K-12 level, Hart and I found that offering school vouchers to economically disadvantaged families improved the productivity of traditional public schools.[xv]
Another just-released study considered whether postsecondary institutions became more efficient following the 2006 ED regulatory change that facilitated the rapid expansion of online-only institutions.[xvi] Deming, Lovenheim, and Patterson compared the post-2006 productivity of local postsecondary institutions with low pre-2006 competition levels to those with higher pre-2006 competition levels. They found evidence the online competition led institutions to shift resources toward instructional expenditures, at least in the public sector and four-year schools, suggesting that online institutions modestly improved the productivity of brick-and-mortar institutions. Interestingly, the authors did not find evidence that incumbent institutions compete on price—indeed, they estimated that tuition increases with online competition. The authors suggested that competition on quality may be more salient than price competition, at least at the tuition levels in question, in this market.
Online higher education is on the rise and rapidly expanding. So what is the evidence regarding this sector?
• Students tend to learn less from online courses than they do from equivalent courses with at least some face-to-face content—especially those from vulnerable populations.
• The labor market appears to value online degrees less than it does face-to-face degrees from nonselective public institutions.
• Credentials based on nationally recognized skill assessments apparently boost the value of online degrees.
• Online education appears to infuse traditional higher education with increased incentives to raise quality.
• A highly-regarded institution can substantially increase access with online offerings.
Therefore, while the overall picture regarding online education is mixed, these new papers present some cause for optimism, especially if we can figure out ways to successfully monitor and certify the quality of online education. However, as Deming and I point out in a recent essay, providing accountability for higher education institutions is easier said than done.[xvii]
— David Figlio
This post originally appeared as part of Evidence Speaks, a weekly series of reports and notes by a standing panel of researchers under the editorship of Russ Whitehurst.
The author(s) were not paid by any entity outside of Brookings to write this particular article and did not receive financial support from or serve in a leadership position with any entity whose political or financial interests could be affected by this article.
[i] David Deming, Claudia Goldin, and Lawrence Katz, “The For-Profit Postsecondary School Sector: Nimble Critters or Agile Predators?” Journal of Economic Perspectives 26(1), 139-164, 2012.
[ii] David Deming, Claudia Goldin, Lawrence Katz, and Noam Yuchtman, “Can Online Learning Bend the Higher Education Cost Curve?” American Economic Review: Papers and Proceedings 105(5), 496-501, 2015.
[iii] Deming, Goldin, Katz, and Yuchtman, ibid.
[iv] Michael McPherson and Lawrence Bacow, “Online Higher Education: Beyond the Hype Cycle,” Journal of Economic Perspectives 29(4), 135-153, 2015.
[v] David Deming, Noam Yuchtman, Amira Abulafi, Claudia Goldin, and Lawrence Katz, “The Value of Postsecondary Credentials in the Labor Market: An Experimental Study,” American Economic Review 106(3), 778-806, 2016.
[vi] David Figlio, Mark Rush, and Lu Yin, “Is it Live or is it Internet? Experimental Estimates of the Effects of Online Instruction on Student Learning,” Journal of Labor Economics 31(4), 763-784, 2013.
[vii] William Alpert, Kenneth Couch, and Oskar Harmon, “A Randomized Assessment of Online Learning,” American Economic Review: Papers and Proceedings 106(5), 378-382, 2016.
[viii] William Bowen, Matthew Chingos, Kelly Lack, and Thomas Nygren, “Interactive Learning Online at Public Universities: Evidence from a Six-Campus Randomized Trial,” Journal of Policy Analysis and Management 33(1), 94-114, 2014.
[ix] Ted Joyce, Sean Crockett, David Jaeger, Onur Altindag, Stephen O’Connell, “Does Classroom Time Matter?” Economics of Education Review 46, 64-77, 2015.
[x] John Krieg and Steven Henson, “The Educational Impact of Online Learning: How Do University Students Perform in Subsequent Courses?” Education Finance and Policy 11(4), 426-448, 2016.
[xi] Di Xu and Shanna Jaggars, “Performance Gaps Between Online and Face-to-Face Courses: Differences Across Types of Students and Academic Subject Areas,” Journal of Higher Education 85(3), 633-659, 2014; Cassandra Hart, Elizabeth Friedmann, and Michael Hill, “Online Course-Taking and Student Outcomes in California Community Colleges,” Education Finance and Policy, forthcoming.
[xii] Hart, Friedmann, and Hill, ibid.
[xiii] Eric Bettinger, Lindsay Fox, Susanna Loeb, and Eric Taylor, “Changing Distributions: How Online College Classes Alter Student and Professor Performance,” working paper, Stanford University, 2014.
[xiv] Joshua Goodman, Julia Melkers, and Amanda Pallais, “Can Online Delivery Increase Access to Education?” National Bureau of Economic Research working paper 22754, October 2016.
[xv] David Figlio and Cassandra Hart, “Competitive Effects of Means-Tested School Vouchers,” American Economic Journal: Applied Economics 6(1), 133-156, 2014.
[xvi] David Deming, Michael Lovenheim, and Richard Patterson, “The Competitive Effects of Online Education,” National Bureau of Economic Research working paper 22749, October 2016.
[xvii] David Deming and David Figlio, “Accountability in US Education: Applying Lessons from K-12 Experience to Higher Education,” Journal of Economic Perspectives 30(3), 33-55, 2016.