NASP Communiqué, Vol. 34, #2
October 2005

Research Reviews

The Effects of Evidence-Based Reading Intervention on Socially Important Outcomes

By Jose M. Castillo, Larry J. Porter, Michael Curtis, NCSP & George Batsche, NCSP

Estimates indicate that approximately 20-40% of children struggle with reading and/or mental health problems in the United States (National Assessment of Educational Progress, 2003; Hoagwood & Johnson, 2003). Given the lasting detrimental consequences of getting off to a poor start in reading and developing mental health problems, as well as the limited resources available to schools, more efficient ways of preventing school-based problems are needed. Researchers have suggested that many of the reading difficulties experienced by children can be prevented by providing evidence-based reading instruction to all students in the primary grades (Torgesen, 2002). Recently, evidence has emerged suggesting that evidence-based instruction in reading leads to improvements in other socially important outcomes as well.

In this article, “socially important outcomes” are defined as those outcomes that are highly valued within the student’s environment. For elementary school students, examples of these outcomes include reading achievement and social skill development. Student progression and success within the general curriculum are also highly valued because of the importance placed on academic competence, as well as the negative life consequences associated with retention (Jimerson, 1999) and special education placement (President’s Commission on Excellence in Special Education, 2002, July). What follows is a brief sample of the literature examining the direct and indirect effects of preventing reading difficulties on socially important outcomes.

Universal Reading Instruction, the Baltimore City Schools Studies

 In a series of studies, Kellam and colleagues implemented a universal, evidence-based reading intervention known as Mastery Learning (ML) in Baltimore City Schools (Kellam, Mayer, Rebok, & Hawkins, 1998). Key elements of the ML intervention were a group-based approach to mastery and a flexible corrective process. To examine the effects of ML, Kellam and colleagues randomly assigned approximately 1,000 first grade students to ML, a universal behavioral intervention condition, or a control condition. The ML intervention was implemented during the fall of the school year and its effects were examined during the spring of the same year.

Results of the Baltimore City Schools studies indicated that ML had a significant direct effect on reading achievement over the course of first grade. Indirect effects also were reported on aggressive and depressive symptoms. Thus, the studies conducted by Kellam and colleagues suggest that universal, evidence-based reading instruction can have a positive impact on the socially important outcomes of reading achievement and social/emotional behaviors. The researchers not only found that universal reading instruction significantly improved the reading outcomes for first grade students, but it also reduced early aggressive and depressive symptoms displayed by these students. Kellam et al. (1998) suggest that the observed effects on aggressive and depressive symptoms may be due to reduced frustration experienced because of failure on the socially valued task of reading.

Response-to-Intervention, Systematically Adjusting Reading Instruction to Student Needs

The Baltimore City Schools studies provide evidence for the efficacy of universal reading instruction. O’Conner, Fulmer, and Harty (2003) examined the effectiveness of systematically providing universal, secondary and tertiary reading interventions in grades K-3 at two schools. A total of 92 students received services through a three-tiered model on an as-needed basis. Tier I services consisted of universal reading instruction and data-based decision making. Tier II (i.e., supplemental intervention) consisted of flexible, small group direct instruction that targeted areas of weakness three days per week. Finally, Tier III (i.e., intensive intervention) services consisted of flexible, individualized instruction that targeted specific areas of weakness five days per week. Students attending the two schools from previous years served as controls.

Results indicated that the students receiving tiered service delivery for reading instruction outperformed the control students from previous years. The effect sizes across tiers ranged from small to large on various measures of reading achievement. Also, students in the experimental group had reduced rates of special education identification. Thus, two socially important outcomes for students in the experimental group involved improved reading achievement and student success within the general curriculum. However, the authors noted that the lack of control schools in the study is a limitation that necessitates caution when interpreting their results.

VanDerHeyden, Witt and Gilbertson (2005) reported data on a problem-solving model of assessment called Screening to Enhance Equitable Placement (STEEP) that was implemented at the district level. This method of screening students and providing tiered services involved a series of evidence-based intervention procedures and sequentially applied decision rules at each stage for reading and math. The four sequential stages were 1) universal screening using curriculum-based measurement (CBM) reading and math probes; 2) class-wide intervention for classes with a large proportion of students performing below a functional instruction criterion; 3) brief performance/skill deficit assessment for those students who do not respond to evidence-based universal intervention; and 4) the response to a short-term, individualized intervention delivered in the classroom for those students identified with a skill deficit.

To examine the effects of STEEP, VanDerHeyden and colleagues (2005) used a multiple baseline design across the five elementary schools in the district. Across the four schools with baseline data from either one or two years prior to STEEP implementation, the number of evaluations for special education and the total number of students who qualified for services were reduced in the year(s) following STEEP implementation. Interestingly, a reversal component in which STEEP procedures were withdrawn at one of the schools during the school year indicated that both the number of evaluations and the number of students who qualified for services returned to levels above baseline following STEEP withdrawal. VanDerHeyden et al. (2005) also reported that the number of children identified as having a specific learning disability (SLD) was reduced from 6% to 3.5% following STEEP implementation in the district. Gains on high-stakes tests were observed in the school district during this time period as well (VanDerHeyden & Jimerson, 2005).

Notably, STEEP’s focus on both reading and math means that the findings in the VanderHeyden et al. study cannot be solely attributed to improvements in reading outcomes. However, since approximately 85% of students identified as SLD are referred for reading/language arts concerns (Office of Special Education Programs, 2005, August 29), improved reading achievement was a likely contributor. Nevertheless, it is clear that the provision of evidence-based interventions to all students resulted in improvements in socially important student outcomes. For example, greater numbers of students remained in the general education curriculum and setting, thereby avoiding negative consequences associated with special education placement. Improvements on high-stakes tests suggested improvements in highly-valued academic skills as well.

Although the impact of preventing reading difficulties on the social outcome of student progression was not examined in the aforementioned studies, it is reasonable to suggest that improving reading outcomes would result in a reduction in grade retentions. For example, schools in the State of Florida can automatically retain students in third grade who perform at Level 1 on the statewide reading assessment. During the 2003-2004 school year, 22% of the 206,435 third graders who took the test scored at Level 1 (Florida Department of Education, 2005, August 29). Certainly, some of these grade retentions could have been prevented with the use of evidence-based reading intervention. The improvement in statewide assessment scores reported by VanDerHeyden and Jimerson (2005) lends credibility to this argument.

The Role of School Psychologists in Preventing Reading Problems

The studies discussed above provide evidence that programs designed to prevent reading difficulties can improve various socially important outcomes. Specifically, improvements in reading, behavioral and social-emotional outcomes have been found. Reductions in the number of students who were evaluated for special education and the number of students who qualified for services, particularly for SLD, were also found. Given the benefits of preventing reading difficulties described above, school psychologists should strive to find ways to provide evidence-based reading instruction to all students.

One potential way in which school psychologists can facilitate evidenced-based reading intervention services is by making an explicit link between reading achievement and socially important outcomes. Interventions that are traditionally considered academic in nature can impact socially important outcomes beyond reading (e.g., behavioral and social-emotional problems that interfere with social skill development) as well as student progression within the general curriculum. Presenting this information to teachers, administrators and student support services personnel may provide incentive for implementing evidence-based reading intervention. Given the limited resources available to schools and the negative life outcomes associated with developing reading and mental health problems, evidence-based preventive services that efficiently improve socially important student outcomes are imperative.

References

Florida Department of Education (2005, August 29). Third Grade Reading 2004 State Profile. Retrieved August 29, 2005 from www.firn.edu/doe/commhome/pdf/3statesum.pdf

Hoagwood, K., & Johnson, J. (2003). School psychology: A public health framework I. From evidence-based practices to evidence-based policies. Journal of School Psychology, 41, 3-21.

Jimerson, S. R. (1999). On the failure of failure: Examining the association of early grade retention and late adolescent education and employment outcomes. Journal of School Psychology, 37, 243-272.

Kellam, S.G., Mayer, L.S., Rebok, G.W., & Hawkins, W.E. (1998). The effects of improving achievement on aggressive behavior and of improving aggressive behavior on achievement through two prevention interventions: An investigation of causal paths. In B. Dohrenwend (Ed.), Adversity, stress, and psychopathology (pp. 486-505). Oxford, UK: Oxford University Press.

National Assessment of Educational Progress (2003).  National Center for Educational Statistics Reading Report Card Page. Retrieved November 30, 2003, from http://nces.ed.gov/nationsreportcard/
reading/results2003/natachieve-g4.asp

O’Connor, R.E., Fulmer, D., & Harty, K. (2003, December). Tiers of intervention in kindergarten through third grade. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

Office of Special Education Programs (2005, August 29). Office of Special Education Programs Home Page. Retrieved August 29, 2005, from www.ed.gov/about/offices/list/osers/index.html?src=oc

President’s Commission on Excellence in Special Education (2002). A new era: Revitalizing special education for children and their families. Retrieved August 29, 2005, from www.ed.gov/inits/commissionsboards/
whspecialeducation/reports/images/Pres_Rep.pdf

Torgesen, J.K. (2002). The prevention of reading difficulties. Journal of School Psychology, 40, 7-26.

VanDerHeyden, A.M., & Jimerson, S.R. (2005). Using Response to Intervention to enhance outcomes for children: Screening, service delivery, and systems change. Manuscript submitted for publication.

VanDerHeyden, A.M., Witt, J.C., & Gilbertson, D. (2005). Effect of a problem-solving model (STEEP) on accurate identification of children. Manuscript submitted for publication.

© 2005, National Association of School Psychologists. Jose M. Castillo, MA and Larry J. Porter, MA, are doctoral students in School Psychology; Michael Curtis, PhD, NCSP, and George Batsche, EdD, NCSP, are Professors of School Psychology and Co-Directors of the Institute for School Reform at the University of South Florida, and Past Presidents of NASP. This article was solicited by Assistant Editor Steve Landau.