Luận án Angela Boatman: Tác động của Sáng kiến Thiết kế lại Khóa học Phát triển lên Thành công Học tập Sơ bộ

Evaluating Institutional Efforts to Streamline Postsecondary Remediation: The Causal Effects of the Tennessee Developmental-Course Redesign Initiative on Early Student Academic Success Angela Rae Boatman Bridget Terry Long Richard J. Willet A Thesis Presented to the Faculty of the Graduate School of Education of Harvard University in Partial Fulfillment of the Requirements for the Degree of Doctor of Education 2012 © 2012 Angela R. Boatman All Rights Reserved i Acknowledgements For someone with a near-endless enthusiasm for stories, connections, and relationships to others, an Acknowledgements section at the end of a long graduate school career is a most difficult challenge indeed. While I long to recognize every teacher, friend, colleague, student, and stranger who has aided this work along the way, I know I would still come up short.

As such, I wish to explicitly thank those to whom I owe my greatest debts, while carrying with me a deep gratitude for the support and kindness of all not listed here. This research would not have been possible without the fine people at the Tennessee Higher Education Commission and the Tennessee Board of Regents. David Wright, Chris Brewer, and Greg Schutz provided both the data and the institutional-knowledge necessary to complete this work. I especially thank Greg Schutz for his unwavering willingness to answer my calls and respond to my requests in the midst of countless other deadlines, all of which were undoubtedly more pressing than mine.

Part of this research was supported by a grant from the American Educational Research Association which receives funds for its "AERA Grants Program" from the National Science Foundation under Grant #DRL-0941014, and as such, the opinions reflect those of the author and not necessarily those of the granting agencies. My work at the Harvard Graduate School of Education was supported in every sense of the word by the members of my committee. I thank my advisor Bridget Long for her expert guidance throughout the entire six-year process, and am deeply grateful to John Willett and Richard Murnane for helping me to talk through this work and to create a polished and professional finished product. The opportunity to witness both the individual and combined expertise of these three senior scholars was an extraordinarily powerful academic experience for me, and one I will forever credit with my own development as a professional in this field.

Prior to arriving at Harvard, I received invaluable guidance from three scholars without whom I would surely not be here today. My advisor in my Master’s program at the University of Michigan, Steve DesJardins, was a most valued sounding board for my future hopes and aspirations (in large part for the fluency with which he spoke Minnesotan). Paul Lingenfelter at the State Higher Education Executive Officers (SHEEO) has generously shared his personal and professional wisdom over the years, and always made me feel as if I had someone rooting for me from the mountains of Colorado. And to Paul Courant, for whom I spent my last year in Ann Arbor working as a research assistant, who let me agonize for months over the decision of where to do my doctoral work before telling me, just as soon as I had decided, what he really thought during a conversation that remains a pivotal turning point in my life.

My colleagues at the Harvard Graduate School of Education have provided the kind of support only fellow graduate students can. I thank Scott Barge, Hanna Rodriguez-Farrar, Kevin Gee, Mike Hurwitz, John Papay, Lindsay Page, Sherry Deckman, and Adela Soliz for fully sharing in this experience with me. ii The years I spent at Eliot House at Harvard College have given me more joy and insight than I could have ever imagined. The people I met as a result of this community completed a part of me I didn’t know was missing, and I am forever changed from this experience.

In addition, Eliot House introduced me to Theresa Lund, my dear friend whose advice and perspective I value beyond words. No acknowledgements section would be complete without mention of my family and closest friends, and I only wish I could mention them all here. Amy Irene Rodquist- Kodet knows me better than I know myself and has been my single greatest support for the last 15 years. We should all be so fortunate to survive graduate school with a health and wellness coach as our very best friend.

I thank David Schofield for magically appearing in the Eliot House courtyard one year ago, and helping me to cross this finish line in the most loving and generous way imaginable. And to my mom, Nancy Boatman, who taught me, through her own example, about sacrifice and commitment, and that goodness on this earth is just as important as greatness. I dedicate my Harvard degree to her. I conclude by recognizing the four extraordinary women who have served as my mentors.

Verna Cornelia Price, June Nobbe, Stella Raudenbush, and Bridget Long have shaped my life in countless and indescribable ways. They gave me professional opportunities beyond measure, and had me attending meetings, teaching classes, and presenting at conferences long before I thought I was ready. But perhaps more importantly, they have also shared their full, rich, holistic lives with me in ways that leave me humbled, grateful, and awash in admiration. Not a day goes by that I don’t think of some small act or conversation or gesture related to one of these women.

They inspire me to do the very best I can, and to remember that life is short and worth living to the fullest. iii Table of Contents Abstract………………………………………………………………………………….1 Background and Context on Developmental Education……………………….3 Redesigning Developmental Education…….7 The Tennessee Higher-Education System and Developmental-Course Redesign Initiative…………………………………….12 Specific Research Questions…………………………………………………….20 Data………………………………………………………………………………22 Sample……………………………………………………………………………24 Measures…………………………………………………………….27 Analytic Strategy…………………………………………………………………30 Determining the Bandwidth…………………………………………………….……………………40 First-stage Results: Who Are the Students Enrolling in Developmental Math?.40 Research Question One: Estimated Effects of Redesigned Remediation Courses, Fall 2008-09 and 2009-10 Cohorts……………………………………41 Research Question Two: Estimated Effects of Redesigned Remediation Courses versus Traditional Remediation Courses at Other Two and Four-Year Institutions, Fall 2008-09 and 2009-10 Cohorts………………….…45 Research Question Three: Estimated Effects of Traditional versus Redesigned Remediation Courses at the Reform Institutions………………….47 Threats to Validity…………………………………………………………….49 Discussion and Policy Implications………………………………………………….55 Figures and Tables………………………………………………….…74 iv Evaluating Institutional Efforts to Streamline Postsecondary Remediation: The Causal Effects of the Tennessee Developmental-Course Redesign Initiative on Early Student Academic Success Abstract Large numbers of students who attend college each year are required to enroll in remedial programs aimed at enhancing their weak reading, writing, and/or mathematical skills and helping to prepare them for success in college-level courses. Recently, a host of new course innovations have surfaced that are intended to move students through remediation more efficiently and effectively. In Tennessee, the focus of this research, several colleges have redesigned the way in which they offer remedial courses, including mainstreaming students into college-level courses and making greater use of learning- technology to provide individualized modules tailored to students’ specific academic needs.

However, little research has been done to estimate the causal effects of these redesigns on student academic outcomes, and evaluate how the impact of the new courses compares to that of “traditional” remediation. Exploiting a statewide cutoff on the placement examination used to assign students to remedial courses, I employ a regression-discontinuity research design to provide causal estimates of the effects of the redesigned courses on the subsequent academic outcomes of students in remediation. Moreover, using data on student outcomes prior to the course redesign, I also test whether the redesigned remedial programs are more effective in preparing students for success in postsecondary education than were the remedial programs they replaced. The effects of enrollment in developmental mathematics are positive and statistically significant on early student persistence, as well as on the number of credits attempted but not completed in the first v semester.

However, these effects do not persist over time, as I find no statistically significant differences between groups after two years. Furthermore, students exposed to redesigned developmental math courses had more positive outcomes than their peers in non-redesign institutions during the same period and also when compared to students exposed to the previous version of traditional remediation within their institution in prior years. The results of my analysis provide insight into the extent to which the particular instruction and delivery methods of remedial courses affect subsequent student academic outcomes, thus informing administrators and policymakers as to how to best help underprepared students. 1 Evaluating Institutional Efforts to Streamline Postsecondary Remediation: The Causal Effects of the Tennessee Developmental-Course Redesign Initiative on Early Student Academic Success Introduction Increasing numbers of American students are enrolling in college unprepared for college- level work (Greene & Forster, 2003; Attewell et al., 2006; Strong American Schools, 2008).

In an effort to help these students develop the skills needed to succeed in college-level courses, postsecondary institutions offer a range of remedial and developmental courses in reading, writing, and mathematics designed to bridge the gap between high-school and college-level material. Remedial and developmental courses, which fall under the general umbrella term of remediation, are designed specifically for students with lower-level skills or those in need of material below “college-level.” While the terms “developmental” and “remedial” are frequently used interchangeably in this literature, developmental courses often refer to those courses just below college-level (e. Algebra II) while remedial courses offer material considerably below college-level (e.1 Such courses, and the costs and benefits associated with their delivery, are of growing concern to students, taxpayers, and higher-education policymakers at all levels, and for good reason: the magnitude and scope of college remediation in the United States is immense. Currently, it is estimated that nearly half of all students enrolled in postsecondary institutions are in need of at least one high-school-level course (NCES, 2004; Attewell et al., 2006; Bailey, 1 In an effort to avoid possible negative connotations associated with the term “remedial”, practitioners tend to use the term “developmental education” to describe the courses and services offered to students below college-level (Bailey, Jeong, & Cho, 2010).

In this paper I refer primarily to “developmental” mathematics courses as those courses just below college-level. Additionally, I occasionally use the term “remediation” to refer to acts or efforts designed to bring students up to college-level courses. 2 Jeong, & Cho, 2010), with some postsecondary institutions reporting that nearly six out of ten students enroll in remedial coursework during their college career (Bettinger & Long, 2009; Bailey, 2009). Statistics from Tennessee show that 73.3 percent of all recent high-school graduates enrolling in a community college for the first time in the fall of 2010 were in need of at least one remedial or developmental course (Tennessee Higher Education Commission, 2010).

Within developmental education itself, students are most likely to need help in the subject of math (Wirt et al., 2004; Bailey, Jeong, & Cho, 2010). A recent study estimated the annual cost of remediation at $1.3 billion at community colleges and another $500 million at four-year colleges, while several states cited costs of tens to hundreds of millions of dollars annually to support remedial programs (Strong American Schools, 2008, as cited in Bailey, 2009; Collins, 2010).2 Additionally, students must shoulder the tuition costs of the courses. In most postsecondary institutions, remedial and developmental courses are typically offered for credit and will count towards a student’s overall GPA, but rarely are they counted toward graduation requirements. Furthermore, the social costs of not offering remediation may be even larger than the institutional costs of the programs and the direct costs to students combined.

Unskilled individuals incur expenses including unemployment costs, government dependency, and crime (Long, forthcoming 2011). As the nation’s economy increasingly demands a more skilled workforce, educational institutions are pushed to develop more effective ways to train their workers. A growing body of research is emerging on both the scope and effectiveness of college remediation. Existing research, however, does not provide clear-cut evidence of the benefits of remediation for students.