NASP Home > Publications/Store > NASP Communique > CQ Indexes > NASP CQ Vol. 34 #6 - Index > NASP CQ 34-6 - RTI Myths

NASP Communiqué, Vol. 34, #6
March 2006

Implementing RTI

RTI Myths and Misrepresentations:  Looking at Data and Experience

By Amanda M. VanDerHeyden, NCSP

For professionals working in school settings, the debate on Response to Intervention (RTI) must seem at times a little tangential and academic. The debate reflects fundamental differences in philosophies about the purpose of psychological services. Moreover, each side in the debate represents different expectations for the outcomes of our practices. It is impossible to say who is “right” and who is “wrong,” as one’s position in this argument is defined by what one primarily values as a practitioner or researcher. 

RTI: Seeking Improved Results

RTI advocates prioritize improved results for students and treat questions of diagnostic accuracy as a secondary objective. RTI is the sum of many lines of research with roots in behavior analysis, precision teaching, direct instruction, curriculum-based assessment/measurement, problem-solving assessment, and large-scale intensive reading intervention studies. Fuchs, Mock, Morgan, and Young (2003) described the two most prevalent types of RTI as “problem-solving” and “standard protocol.” If these are the two general categories, several hybrid models are evolving and more may be anticipated. Indeed, RTI as it is used in many districts has evolved to include the most promising components of early, more established models.

There are many varied approaches to RTI implementation.  They all share one similarity:  a philosophy that services should be needs-driven and their effectiveness reflected by improved direct indicators of child learning. Primary to this philosophy is defining problems and conducting assessment that identifies solutions that are likely to be effective at resolving the identified problem. This approach is in contrast with historical disability-identification-centered practices that maintain a primary focus on identifying the presence of underlying disorder for accurate diagnosis.

There is one statement that will resonate with both practitioners and scholars, on either side of the RTI fence. No model of identification or service delivery will be perfectly accurate. It is promising that many who argue against RTI as a diagnostic criterion agree that RTI should occur, but just should not be part of the diagnostic system, or more accurately, should not replace the discrepancy model of identification. Additionally, everyone seems to agree that universal screening with sensitive and reliable measures, progress monitoring, and effective intervention has merit.

Arguments about whether or not RTI can be implemented well in real systems have many valid points, but that does not mean that we should not try. Can you imagine a nurse saying, “Well, I might not place the IV correctly, so I think I better recommend that we stop using IVs.” The old adage—“nothing great was ever accomplished without risk” is apropos.  Moreover, continuing to advocate for demonstrated ineffective practices (as indicated by student learning outcomes) when viable alternatives exist is both ethically questionable and practically unwise. The key, it seems, is to evaluate the risk relative to potential gains with a focus on minimizing risk and maximizing gains. One fundamental component of this approach is the requirement of tracking the costs and the benefits of the diagnostic and intervention services that schools provide.

I am reminded of the classic Wolf (1978) article in which he described the failure of an intervention program and wrote, “We [therapists] argued, ‘what you [parents, teachers] really need is someone who knows how to give and take away points at the right time’” (p. 207). Wolf’s wisdom and insight was that a therapist or a program can be technically very good (e.g., perfect implementation of the historical special education identification system) and yet miss the mark altogether, the mark being what consumers and systems care about most (student results). The RTI approach is a system that is tied directly to what systems care about tremendously in this age of accountability, child learning. It is not that the debate of diagnostic construct development does not have academic interest or merit; it’s just that it has not yet been demonstrated to be relevant to the goal of improving child gains.  Until the diagnostic constructs used in schools demonstrate meaningfulness to improving results, they will have limited relevance in schools. 

Because the purpose of this Communiquéseries is to examine RTI implementation and what experience tells us, this next section will examine a series of RTI concerns/RTI myths that have been identified by practitioners across the country who are working on implementing RTI.  Answers to these issues are provided based both on the current literature as well as on the author’s experience implementing RTI in practice.

Myth: If we use RTI, only low-achieving students will be identified and identification of students with true disabilities will be inaccurate.

Early studies on disability identification using Curriculum-Based Measurement (CBM) found that CBM was a promising technology to accomplish accurate identification of children who were struggling in the general education curriculum (Marston, Mirkin, & Tindall, 1984; Shinn, Tindall, & Spira, 1987). Subsequently, studies indicated that trend data obtained through frequent measurement using CBM could be used to test instructional strategies and enhance individual child growth (Fuchs, Fuchs, Hamlett, & Stecker, 1991; Fuchs, Fuchs, Mathes, & Simmons, 1997). Following the ideas of Deno (1986) and others who wrote about problem-solving assessment (Reschly, Tilly, & Grimes, 1999), several models of problem-solving assessment began to appear and undergo field testing around the country. Simultaneously, several behavior analytic researchers began studying methods for testing hypothesized causes and interventions for common child learning problems (see especially Daly and colleagues, 1994, 1999).

One group of researchers examined the use of performance level and growth data over a 6-month period of regular classroom instruction to identify a group of children who were consistently lower-performing and slower-growing than their classmates in early reading performance (Case, Speece, & Molloy, 2003; Speece, Case, & Molloy, 2003). These researchers found that the performance data could be used to identify a group of children who were distinguishable from typically-achieving children and children in the same class who were considered at-risk on a variety of performance measures administered at the start of the study. The group of children identified as potentially in need of special education eligibility was about 3% of the screened total and was proportionate by race and gender.

Another group of authors has examined accuracy of identification using a model called Problem Validation Screening relative to other sources of identification, most notably teacher referral (VanDerHeyden, Witt, & Naquin, 2003). Problem validation screening included screening every child in the sample with CBMs of reading and math, conducting a follow-up assessment of the effect of incentives on child performance for an identified risk group (about 15% of the original sample), and conducting a single instructional session for an identified risk group (about 11% of the original sample). The resulting identification of a final risk group of about 5% of the screened sample was more accurate than other methods of identification, most notably teacher referral. Further examination of these data indicated that use of trend and level data during intervention identified the same children who performed in the risk range on the Iowa Test of Basic Skills and the WJ-R (VanDerHeyden, Witt, & Barnett, in press). Further, early response to intervention was predictive of ultimate response to intervention (VanDerHeyden et al., in press), a finding also reported by others (Vaughn, 2003).  This finding indicates that decisions about whether or not children should be considered for evaluation for special services can be made relatively quickly (i.e., do not delay the provision of special education services). Further, with RTI procedures low-performing children receive intensive interventions following screening. Given that identification occurs with universal screening, children are identified earlier and at lower grade levels than would have occurred without a RTI model in place.

In a district-wide implementation study, VanDerHeyden and colleagues (in submission) used a standardized set of procedures grounded in problem-solving logic to provide a series of interventions to at-risk children. Following a poor response to intervention, children were referred to the school-based decision-making team for consideration of possible referral for a full psycho-educational evaluation to determine eligibility. Prior to implementation of this model, about 50% of children evaluated in this district actually qualified for services. When the RTI procedures and criteria were implemented and resulting data indicated that children should be considered for an evaluation, the percentage of evaluated children who qualified for services increased to greater than 95% across all schools when the RTI model was introduced. Each of these authors has recognized that identification rates and accuracy in a system using relative decision-making with local norms might be affected adversely by the presence of a class-wide or school-wide instruction problem. Fuchs (2003) eloquently described the philosophical underpinnings of the decision-making process and asserted that different types of errors will be made and the errors that will be tolerated reflect the underlying purpose of the model used (see Batsche, Kavale, & Kovaleski, in press).

Current research indicates that RTI approaches to identifying students with disabilities can identify unique groups of students with specific and pervasive performance problems.  Moreover, the identification practices used yield data that can be used directly to program instructional interventions for students and can be used to inform eligibility decisions and improve resource allocation.

Myth: If we implement RTI in our district, inequitable identification will be prevalent.

The problem of equitable identification, classification, and placement has haunted special education for years. In 1982, Heller, Holtzman, and Messick noted that disproportionate identification by race and gender was a pressing problem that deserved special scrutiny in research and policy. In this document, the idea of RTI was described as a possible solution, although it wasn’t referred to as “RTI” at the time. Twenty years later, Donovan and Cross (2001) echoed these ideas, proposing that universal screening, early intervention, and data-based decision-making linked to valid progress monitoring tools offered the best solution to inequitable identification. To the degree that disproportionate identification reflects disparate opportunities to learn, it is a pressing concern for educators.

Data from field-based implementations of RTI demonstrate that RTI practices can have a positive effect on equity of identification.  For example, a program implementation study of the Minneapolis Schools problem solving model demonstrated positive effects for African-American students (Marston, Muyskens, Lau, & Canter, 2003).  That is, minority overrepresentation was reduced. Speece et al. (2003) found that gender and ethnicity were proportionately represented in their risk sample after completing a systematic screening consistent with RTI practices.

VanDerHeyden and Witt (2005) examined the accuracy of an RTI decision-making model for screening within high-achieving and low-achieving classrooms and by race and gender relative to other identification methods, most notably teacher referral. In all cases, the RTI-type method was more accurate than teacher referral. The RTI method of identification was relatively stable by race, gender, and classroom achievement level, whereas teacher referral accuracy varied dramatically. Males and African American students disproportionately appeared in the lowest performing group in the school at screening, but brief intervention had a powerful effect on risk group placement. (Prior to intervention, more than 50% of the African American children in the school appeared in the risk group. Following intervention, only 7% of the African American children in the school appeared in the risk group.) What is key about this particular finding is that universal screening alone is likely to be insufficient to improve equity in identification (Hosp & Reschly, 2004). Effectively delivered and monitored interventions linked to referral decision-making is the key antidote for reducing disproportionate identification. In a well-controlled district-wide implementation study of an RTI model for identification, VanDerHeyden and colleagues reported positive district-wide effects for proportionate identification by gender. In a district where males were over-identified (8 males:1 female were identified at baseline), use of the RTI model reduced the ratio to 2:1 (VanDerHeyden, Witt, & Gilbertson, in submission).

Hence, the available evidence suggests that the use of RTI does not increase, but instead may decrease, biases in identification and thus increase equity in special education identification.

Myth: If we implement RTI practices, we’ll break the bank. Many more children will be identified as LD.

The standard, intensive-intervention protocol studies that are characterized by the work of Torgesen and colleagues have typically reported a 3-5% failure-to-respond rate. District-wide applications of RTI models have reported reduced numbers of children receiving special education services. Studies conducted in districts in Louisiana and Arizona found that evaluations were reduced between 50% and 70% with the use of a multi-tiered problem-solving RTI model. VanDerHeyden et al. (in submission) found that the diagnostic hit rate was increased with the use of an RTI model, while the number of children identified as having SLD was decreased from 6% of children district-wide at baseline to 3.5% following one year of implementation in the highest-referral schools. This reduction maintained over the following two years as the RTI model was implemented across all remaining elementary and middle schools in the district (N = 7 schools). Tilly (2003) reported that the number of children grades K-3 who received special education services declined following the implementation of a literacy-focused RTI model.

Myth:  If we implement RTI, children who need help will not receive help and overall learning will suffer.

Reports of decreased numbers of special education children could be troubling if concurrent data were not presented indicating that learning was simultaneously improved. Multiple researchers have found improved growth and performance with correctly implemented interventions conducted as part of an RTI model in reading (Burns, Appleton, & Stehouwer, in press; Marston, Muyskens, Lau, & Canter, 2003; Speece, Case, & Molloy, 2003; Tilly, 2003) and in math (VanDerHeyden et al., 2003; VanDerHeyden & Witt, 2005; VanDerHeyden & Burns, 2005; VanDerHeyden, Witt, & Gilbertson, in preparation).

Arguably, one of the most daunting challenges confronted by psychologists working in the schools is how to best identify and assist children. Converging findings indicate that the prereferral process has not resulted in accurate identification, but has had one predictable effect: once children are referred to the school-based assessment team, they are likely to be identified and placed in special education. Because identification and placement can carry risk of negative effects and because service in special education has not been demonstrated to differentially benefit children whose academic problems fall in the high incidence category (e.g., Specific Learning Disability), prereferral identification is a process that merits further scrutiny and revision as suggested through the educational reform movements in both general and special education.

RTI is a set of practices with much empirical evidence to support its potential in schools. RTI is not a new idea. References to RTI practices have appeared in the literature for the past 30 years, though the term RTI is a neologism. Existing models have drawn from the extensive research base of CBM, behavior analysis in education including direct instruction and precision teaching, and reading intervention research. Future iterations of RTI models are inevitable and desired. Models that represent hybrids of the standard protocol and problem-solving approaches are likely. Hybridized models of technical adequacy will be necessary to evaluate the utility and effectiveness of RTI models in practice. Such technical models have the opportunity to enhance outcomes by including measurement of the independent variable as well as dependent variables over repeated trials, and including provisions that data link to specific eligibility and service decisions.

Cautions and Opportunities

Use of RTI in districts around the country can be expected to lower placement rates, improve diagnostic “hit rates” and equity, and improve child learning if correctly sequenced interventions are implemented with integrity and resulting data are linked to decision-making. If RTI becomes a checklist, all guarantees of improved outcomes are off. Whether or not RTI works on a large scale will depend upon its effect on child learning. Practitioners working in schools and school systems can proceed with risk of minimal harm so long as they have articulated the purpose they would like to serve, use validated methods, collect multiple sources of information to determine if their methods are moving them in the right direction, and are willing to respond to those data formatively to stay on track.  In my experience, RTI allows school psychologists to become instructional allies to systems, administrators, teachers, and children. It’s not less work for practitioners, but it is more gratifying, efficient, and effective work if you believe the purpose of your work is to improve child outcomes. RTI is an opportunity to reorient service delivery.  It is about helping first and diagnosing, if needed, later.  It is an opportunity for all of us to be aligned with the primary goal of schooling—enhancing achievement. 

References

Burns, M. K., Appleton, J. J., & Stehouwer, J. D. (in press). Meta-analytic review of response-to-intervention research:  Examining field-based and research-implemented models. Journal of Psychoeducational Assessment.

Case, L. P., Speece, D. L., & Molloy, D. E. (2003). The validity of a response-to-instruction paradigm to identify reading disabilities: A longitudinal analysis of individual differences and contextual factors. School Psychology Review, 32, 557-582.

Batsche, G., Kavale, K., & Kovaleski, J. K. (in press). Competing views: A dialogue on response to intervention. Assessment for Effective Intervention.

Daly, E. J., III, & Martens, B. K. (1994). A comparison of three interventions for increasing oral reading performance: Application of the instructional hierarchy. Journal of Applied Behavior Analysis, 27, 459-469.

Daly, E.J.,III, Martens, B.K., Hamler, K.R., Dool, E.J., & Eckert, T.L. (1999). A brief experimental analysis for identifying instructional components needed to improve oral reading fluency. Journal of Applied Behavior Analysis, 32, 83-94.

Deno, S. L. (1986). Formative evaluation of individual student programs: A new role for school psychologists. School Psychology Review, 15, 358-374.

Donovan, M. S. & Cross, C. T. (2002). Minority students in special and gifted education. Washington, DC: National Academy Press.

Fuchs, D., Mock, D., Morgan, P. L., & Young, C. L. (2003). Responsiveness-to-intervention: Definitions, evidence, and implications for the learning disabilities construct. Learning Disabilities Research and Practice, 18, 157-171.

Fuchs, L. (2003). Assessing intervention responsiveness: Conceptual and technical issues. Learning Disabilities Research and Practice, 18, 172-186.

Fuchs, L. S., Fuchs, D., Hamlett, C. L., & Stecker, P. M. (1991). Effects of curriculum-based measurement and consultation on teacher planning and student achievement in mathematics operations. American Educational Research Journal, 28, 617-641.

Fuchs, D., Fuchs, L. S., Mathes, P. G., Simmons, D. C. (1997). Peer-assisted learning strategies: Making classrooms more responsive to diversity. American Educational Research Journal, 34, 174-206.

Gravois & Rosenfield (2002). A multi-dimensional framework for evaluation of instructional consultation teams. Journal of Applied School Psychology, 19, 5-29.

Heller, K. A., Holtzman, W. H., & Messick, S. (1982). Placing children in special education: A strategy for equity. Washington, DC: National Academy Press.

Hosp, J. L., & Reschly, D. J. (2004). Disproportionate representation of minority students in special education: Academic, demographic, and economic predictors. Exceptional Children, 70, 185-199.

Marston, D., Tindal, G., & Deno, S. L. (1984). Eligibility for learning disability services: A direct and repeated measurement approach. Exceptional Children, 50, 554-556.

Marston, D., Muyskens, P., Lau, M., & Canter, A. (2003). Problem-solving model for decision-making with high-incidence disabilities: The Minneapolis experience. Learning Disabilities Research and Practice, 18, 187-200.

Reschly, D.J., Tilly, W.D., & Grimes, J. (Eds.)(1999). Special education in transition: Functional assessment and noncategorical programming. Longmont, CO: Sopris West.

Shinn, M. R., Tindal, G. A., & Spira, D. A. (1987). Special education referrals as an index of teacher tolerance: Are teachers imperfect tests? Exceptional Children, 54, 32-40.

Speece, D. L., Case, L. P., & Molloy, D. E. (2003). Responsiveness to general education instruction as the first gate to learning disabilities identification. Learning Disabilities Research and Practice, 18, 147-156.

Tilly, D. (2003, December). Heartland Area Education Agency’s evolution from four to three tiers: Our journey - our results. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.

Torgeson, J. K. (2000). Individual differences in response to early interventions in reading: The lingering problem of treatment resisters. Learning Disabilities Research and Practice, 15, 55-64.

VanDerHeyden, A. M., & Burns, M. K. (2005). Using curriculum-based assessment and curriculum-based measurement to guide elementary mathematics instruction: Effect on individual and group accountability scores. Assessment for Effective Intervention, 30, 15-31.

VanDerHeyden, A. M. & Witt, J. C. (2005). Quantifying context in assessment: Capturing the effect of base rates on teacher referral and problem-solving model of identification. School Psychology Review, 34, 161-183.

VanDerHeyden, A. M., Witt, J. C., & Barnett, D. A. (2005). The emergence and possible futures of response to intervention. Journal of Psychoeducational Assessment, 23, 339-361.

VanDerHeyden, A. M., Witt, J. C., & Gilbertson, D. (in submission). Effect of a problem solving model on identification of children for special education. Manuscript submitted for publication.

VanDerHeyden, A. M., Witt, J. C., & Gilbertson, D. (in preparation). Effect of a problem solving model on child and system outcomes. Manuscript in preparation.

VanDerHeyden, A. M., Witt, J. C., & Naquin, G. (2003). The development and validation of a process for screening and referrals to special education. School Psychology Review, 32, 204-227.

Vaughn, S., Linan-Thompson, S., & Hickman, P. (2003). Response to instruction as a means of identifying students with reading/learning disabilities. Exceptional Children, 69, 391-409.

Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203-214.

©2006, National Association of School Psychologists. Amanda M. VanDerHeyden, PhD, NCSP, is on the faculty of the University of California–Santa Barbara. This series (“Implementing RTI”) is coordinated and edited by W. David Tilly III, PhD.