Response to Intervention Within Tier 3: A Model for Data Teams
By Ilana Sgouros & Karen Walsh
There is increasing evidence that a significant number of children in America are not learning important basic reading skills (National Center for Education Statistics, 2005). Response to intervention (RTI) for support in reading is appropriate, but what does a school district do about students who are in Tier 3 and not making progress? What other interventions are school psychologists and special educators expected to bring to the nonresponders eligible for Individual Educational Programs (IEPs)? They can access recommendations for best practice in instruction from the National Reading Panel (2000), but how will these be applied in the special education setting?
These questions represent the core issues faced every day by teachers and psychologists at the Special School District of St. Louis County (SSD). SSD is a unique school system in that the district is an independent school district that provides special education and related services to students enrolled in 22 partner school districts serving residents of St. Louis County. These partnerships consist of more than 2,700 special education teachers and related service providers working in 265 public schools. Questions about Tier 3 are not unique to this dual system; school psychologists and educators across the nation continually consider how to close the achievement gap for students with IEPs.
Monitoring of progress has a solid history as part of practice within special education (Deno, 1985) and as a reliable and valid measurement of overall reading progress (Baker & Good, 1995; Deno, 1985; Shinn et al., 1992). Curriculum-based measurement to monitor progress is the “repeated sampling of child performance on a common task to assess growth and development in critical skills that are related to meaningful long-term outcomes” (Deno, 1985). SSD sought to utilize the proven method of curriculum-based measurement and to combine progress monitoring with the initiation of the elementary level data team model of group collaboration and evaluation of students' data within special education teams. The SSD data teams consist of groups of special educators, school psychologists, and administrators meeting on a regular basis to examine student data and solve instructional problems together. While problem solving is frequently used for students in Tier 1 or Tier 2 of RTI, the systematic data team process for students who are eligible for IEPs is not as prevalent. The district initiative was to create a more systemic and systematic process for analyzing student growth and instructional/curriculum changes based on student data.
Teachers, school psychologists, and administrators communicated a vision that all children can learn, regardless of the degree or type of impairment identified. The use of a systems-based approach to increase student achievement offers a great number of opportunities for school psychologists, whose skills include consultation, facilitation, data analysis, and expertise in research-based interventions. The data team is the equivalent of the systems approach to increasing student achievement (Reeves, 2010).
Learning occurs for all children within the context of multiple systems and is shaped by the relationships within and between those systems (Bronfenbrenner, 1979). Part of the data team model is to examine each facet of the environment, including the learner, the curriculum, the learning environment itself, and the instruction provided (e.g., direct instruction, practice, strategies). Each component is discussed by the team when a teacher is considering changes in instruction or programming, with strengths and weaknesses considered within each of these systems.
For children with IEPs, specialized instruction can occur within several contexts: whole school approaches, classroom instruction, or small group interventions that are part of their IEPs. Data Teams provide the structure for collaboration among the special education staff while also reinforcing the fidelity of progress monitoring across all the educators who serve students with IEPs. Results are shared with general education teachers and administrators, as well as with parents. Students also view the graphs that illustrate their progress, as many students participate in the data collection process in their classrooms. They are aware of their academic goals and whether they are on track to reach them.
The Genesis of Data Teams at SSD
SSD received an Elementary Achievement Grant from the Missouri Department of Elementary and Secondary Education in 2008 and continued to receive funding for this initiative through June, 2011. The grant provided seed money to purchase the primary progress monitoring tool (AIMSweb) as well as other instructional and diagnostic resources. An important grant requirement was the formation of a stakeholder group consisting of school, community, and parent participants to help guide the data team process. The goal for the data teams initiative was to increase achievement on the Missouri Assessment Program (MAP) in the area of communication arts through the introduction of a systems-change approach.
School psychologists and special educators were trained in the data team process. Frequent, formative progress monitoring began using AIMSweb as the curriculum-based measurement tool in a few selected schools. Students with reading goals on their IEPs were the initial subjects upon whom the data team focused. Measurement tools included those within AIMSweb (e.g., early literacy measures, reading curriculum- based measurement, and multiple choice cloze tasks), as well as other reading measures for students with more significant disabilities, such as Get it Got it Go! and a district-created literacy fundamentals checklist. The Get it Got it Go! measures included picture naming, rhyming, and alliteration (see the Get It, Got It, Go! website at http://ggg.umn.edu). The district also designed and provided a literacy checklist for monitoring the progress of students who were not yet able to complete the Get it Got it Go! measures or the early literacy measures of AIMSweb. Data team meetings included opportunities to share these resources, as dictated by the needs of the student.
Data Team Evolution At SSD
In the first year of implementation (in 2008) 69 teachers in 7 of the 22 districts implemented the data team initiative. In year 2, another eight districts joined with more than 250 teachers participating. By the 2010–2011 school year, data teams were operating in all 22 districts in St. Louis County, with more than 500 teachers, speech–language pathologists, and school psychologists involved. The data team approach had grown as a system-wide approach across all elementary schools and some of the middle schools and high schools served by SSD.
At data team meetings, special education staff engages in problem solving with respect to students' progress. The group uses a formal process that consists of a standing agenda, consistent norms, and standardized progress monitoring tools to review progress and make decisions about future interventions. Special education teachers collect students' progress data in reading on a weekly basis to present to the data teams, which meet monthly or bimonthly to review the data and discuss any instructional changes that may be needed.
Based on student data, teams consider three possible outcomes and reactions: (a) positive response—continue monitoring using current goal and current instruction; (b) questionable response— consider diagnostic assessments and change or modify instruction; and (c) negative response—consider change in instructional approach. The teams also examine fidelity of implementation and consider the use of diagnostic data such as informal reading inventories, teacher designed word lists, and other measures in determining specific student needs and possible intervention or instructional strategies. The model, as developed from the data team process described by the Leadership and Learning Center (Reeves, 2010), also provided training in developing district-wide processes for growth of data teams.
Due to the scale of implementation across the district, a major focus has been to develop school-based data team leaders, many of whom are school psychologists who attend cohort groups based on their school team's level of experience with progress monitoring. Cohort sessions are led by district data coaches who facilitate learning for data team leaders. Data coaches provide teams and their leaders with norms for collaboration, information about the content and use of progress monitoring tools, and summaries of recently published research in literacy and progress monitoring. Monthly cohort meetings with data coaches support the leadership and facilitation skills of the data team leaders.
Stakeholders meet four times each year to review progress, reflect upon goals and outcomes, brainstorm solutions, and provide feedback to the data coaches and data team leaders. This allows for two-way communication about the effectiveness of the data teams and the progress of students and data collection by teachers. It also serves as a vehicle to make system-wide adjustments or changes that may be needed.
Student Performance Results At SSD
The primary focus has been on increased student achievement, and the results have been very positive. One way of determining student progress included an assessment of overall progress using the benchmark tool. To evaluate growth from fall to spring, rate of improvement was calculated using three grade-level reading probes and subtracting the median score in the fall from the median score in the spring, and dividing by the number of weeks that progress was monitored across the year. Baseline data from the first year revealed average student scores measuring at the 10th percentile; SSD's modest goal was a rate of improvement (ROI) at the 25th percentile (.80). However, the average ROIs for students in grades three through five were actually at or above the 50th percentile (.84). Of note is that the sample consisted entirely of students with IEPs—not students from the general school population. Preliminary data suggested a positive correlation between data-based instructional decision-making through progress monitoring data teams and increased student performance in reading.
Data was consistent for the following 2 years of participation in data teams. Although the number of students dramatically increased in the 2010–2011 school year (n = 1,344), the ROIs were consistently higher than expected. Student data in grades three through five were analyzed. ROIs across the schools were very comparable, and 89% of districts met or exceeded the goal of gaining 0.8 words per week. Progress monitoring in communication arts was not limited to students with specific learning disabilities in reading, but also included students with language impairments, autism, health impairments, and intellectual disabilities. Rate of improvement for all areas of disability was very similar and higher than expected (.84 autism, .69 emotional disturbance, .86 other health impaired, .90 specific learning disability, .84 speech and language; .84 overall). The one exception was in the group of students identified with intellectual disabilities, who still demonstrated a rate of improvement at .55, and overall improvement over the school year. The results support the initial contention that all students can learn.
Teacher Implementation and Fidelity Results at SSD
In addition to student outcome data, district data coaches and data team leaders gathered data on teacher implementation and fidelity. One of the tools used to analyze implementation was a teacher self-analysis checklist that polled teachers about their level of independence regarding data-team skills. Results for 2011 were aggregated from 431 respondents, including teachers in their first, second, or third year of implementation. Teachers reported at a rate of 20% or less that they required support to navigate and use the AIMSweb technology, to set goals using the calculation of a rate of improvement, to administer and score probes correctly, and to use the norm tables to determine a student's current level of performance. Fewer than 15% reported that they needed help to make changes to intervention plans based on student CBM data. These results suggest a high level of teacher buy-in as the overwhelming majority of teachers reached independence in using the progress monitoring tool.
Another method for analyzing fidelity was completed in 2011 when individual AIMSweb data was reviewed for all teachers participating on data teams. On average, teachers in their third year of implementation collected data on a weekly basis at a rate of 88%, compared to 75% of teachers in their first year. Teams review their group data and set team goals at the beginning of each year. One important goal for teams is fidelity in frequency of their monitoring progress.
Administrators implement fidelity checks through the IEP process as well. Student data is reported quarterly through required progress reports to parents. For students with communication arts goals, the expectation is that annual IEP reviews include results of weekly probes, continued progress toward goals, and discussion of instructional changes as a result of trend line analysis.
Interventions Utilized to Increase Student Achievement
SSD provides data teams with the support they need for finding and selecting reading intervention resources. Among the instructional resources provided to teachers participating in data teams are sample lesson plans, IEP goal-writing tools, tools for differentiated instruction, and reading triage. Reading triage includes well-run intensive care units for the most at-risk students, specialized care for those moderately at risk, and excellent core instruction for capable readers. The purpose of this model is to ensure that teachers are equipped through professional development and follow up support to address diverse needs of all students.
The district engaged Dr. Erica Lembke, University of Missouri, to provide advanced training for district staff regarding progress monitoring and literacy strategies. Her presentation, “Connecting AIMSweb Data to Research-Based Literacy Interventions,” provided teachers and psychologists with greater understanding of the utility of strategic instructional design, and explicit, systematic teaching procedures. Among the strategies teachers learned to use were Elkonin boxes, phrase-cued reading, anticipation guides, probable passages, and “Its Says, I Say, and So.” (Lembke, 2011). The school psychologists who attended this training developed literacy strategies to use in collaboration with teachers who then implemented the strategies with students.
Literacy coaches across SSD's partner schools support the data teams through consultation and the development of two resources, the Literacy Intervention Guide, and the Literacy Quick Reference. Literacy coaches also serve data teams in the roles of interventionists and research checkers. Along with the district-specific guide and reference materials, data teams also received supplementary literacy materials to complement the standard reading and writing curricula, literacy programs, and replacement curricula. Those included the Florida Center for Reading Research Literacy Center activities (see Florida Center for Reading Research website at http://www.fcrr.org), Teaching Reading Sourcebook (Honig, Diamond, & Gutlohn, 2008), and the CORE Assessing Reading: Multiple Measures (Diamond & Torsnes, 2008), which is used by teachers for diagnostic assessment.
The Future of Data Teams at SSD
The SSD Board of Education has made an institutional commitment to the data team process. Although grant funding ended with the close of the 2010–2011 school year, district data coaches, data team leaders, and data teams continue. Review and analysis of data teams' impact continues as well. In the 2011–2012 school year, 45 middle and high schools are participating in the data team process. The district continues to look at student performance on the MAP in the area of communication arts. The district is also assessing the viability of system-wide progress monitoring in the area of mathematics for support of its mathematics curriculum and instruction. Data teams in their fourth year are expanding their reach beyond communication arts to math and behavioral measures.
A continuing question is whether data-based decision-making rules should be universally applied, or whether their application should depend on the specifics of the curriculum or instruction. Similarly, the district continues to analyze the impact of the data team initiative among the many other initiatives within the 22 partner school districts.
Standing at the Top of the Pyramid
Students with IEPs stand at the top of the RTI pyramid, but they are no longer there by themselves and without support. Special education teachers and school psychologists continue to provide frequent and formative assessments that guide instruction and intervention. With data teams, they seek to improve student achievement and to close the gap with their peers in general education. Teams are accountable to themselves, to school administration, to parents, and to the students they serve. Members of the data teams recognize the need to determine on an ongoing basis what is and is not effective instruction for special education students, to recognize what supports are needed for staff to grow and thrive as special educators, and to reflect on instructional decisions that support student achievement.
The role of the school psychologist at SSD does not end with eligibility determination or the development of the initial IEP. As data team leaders, school psychologists participate in classroom learning cycles for students with IEPs and serve as valuable consultants when reviewing student progress. The view from the top of the pyramid includes a shared vision for closing the achievement gap between students with IEPs and their peers.
Baker, S. K., & Good, R. (1995). Curriculum-based measurement of English reading with bilingual Hispanic students: A validation study with second-grade students. School Psychology Review 24, 561–578.
Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press.
Deno, S. (1985). Curriculum-based measurement: the emerging alternative. Exceptional Children, 52(3), 219–232.
Diamond, L., & Thorsnes, B. J. (2008). CORE assessing reading multiple measures, 2nd ed. Berkeley, CA: Arena Press.
Honig, B., Diamond, L., & Gutlohn, L. (2008). CORE teaching reading sourcebook, 2nd ed. Berkeley, CA: Arena Press.
Lembke, E. (2011). Connecting AIMSweb data to research-based literacy interventions. Presentation to the University of Missouri at Columbia Special School District.
National Center for Education Statistics. (2005). National assessment of educational progress. Retrieved from http://www.nces.ed.gov
National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. Washington, DC: National Institute for Literacy.
Reeves, D. (2011). Decision making for results. Englewood, CO: The Leadership and Learning Center.
Shinn, M. R., Good, R. H., Knutson, N., Tilly, W. D., & Collins, V. L. (1992). Curriculum-based measurement of oral reading fluency: A confirmatory analysis of its relation to reading. School Psychology Review, 21, 459–479.
Ilana Sgouros is a progress monitoring data coach at Special School District of St. Louis County, MO. Karen Walsh is an effective practice specialist in school psychology at Special School District of St. Louis County, MO. She is also a past president of the Missouri Association of School Psychologists.