Curious to learn more? Check out the related webinar in the Online Learning Center!

In a recent presentation, I discovered that most of the participants did not feel well trained in data analysis for MTSS, and, when they were trained, it was almost exclusively on the use of graphical trends (bar graphs) to make decisions. Participants were aware that more statistically sound methods existed (e.g., data analysis and percent of nonoverlapping data points); they just did not know how to use them. The goal of this post is to provide easy ways to integrate two statistical analyses into your daily practice.

Effect Size Reporting Analysis

First, don’t be afraid to try these methods. The word "statistics" is often intimidating, but it does not have to be. In the presentation, we discussed the use of a simple effect size calculator. See Krach (2022) for a free downloadable calculator on an Excel spreadsheet. It only requires that you enter baseline and pre–post scores. The effect size calculations are done for you! Even the interpretation standards are provided for decision making.

Percent Nonoverlapping Data (PND) Analysis

Calculating PND is even easier! Tarlow and Penland (2016) provide a simple online calculator that doesn’t need downloading. You simply answer three questions, and the PND is provided. According to Scruggs et al. (1986), a PND of 70% or more is either effective or very effective. Anything below 50% should be considered ineffective, and anything between 50% and 70% should be considered questionably effective. Using this standard, interpretation of the PND analyses results is a snap!

PND Versus Traditional Effect Size

Which one is best? It really depends on the type of data that you collected.

PND is beneficial because it can be done without having to collect postintervention data. It also requires only one data point for the baseline (Rakap, 2015). You must collect sufficient progress monitoring data for PND to be accurate, though.

Traditional effect size calculation is beneficial because it does not require data collection during the intervention (i.e., progress monitoring). It can be conducted with a minimum of three pretest scores and three posttest scores.

However, both PND and effect size results can be misleading. They are both easily influenced by the use of poor assessments, too few data points, or too short of an intervention. But these methods are still considered more accurate than graphical interpretation (Rakap, 2015).

Upcoming Related Workshop

An expanded training of these analysis will be provided at the NASP 2023 Annual Convention in Denver. We will be providing a 3-hour workshop with practice mini-groups. Participants will be analyzing social, emotional, and behavior data as well as academic scores and bilingual data. The goal of the upcoming workshop is to make these techniques accessible to everyone.

References

Krach, S. K. (2022). Effect size calculator. https://www.kathleenkrach.com/tools

Rakap, S. (2015). Effect sizes as result interpretation aids in single‐subject experimental research: description and application of four nonoverlap methods. British Journal of Special Education, 42(1), 11–33. https://doi.org/10.1111/1467-8578.12091

Scruggs, T. E., Mastropieri, M. A., Cook, S. B., & Escobar, C. (1986). Early intervention for children with conduct disorders: A quantitative synthesis of single-subject research. Behavioral Disorders, 11, 260–271. https://www.jstor.org/stable/23882207

Tarlow, K. R., & Penland, A. (2016). Percentage of Nonoverlapping Data (PND) Calculator. http://www.ktarlow.com/stats/pnd