Using Technology to Lead a Large-Scale Data Project
Volume 46 Issue 5
By Julie Shields
While working in the Social and Emotional Support Services (SESS) for Montgomery County Public Schools (MCPS), I initiated a project to collect data on our services to students with emotional needs. During the 2016–2017 school year, MCPS was the 17th largest school district in the country with nearly 160,000 students speaking 150 languages and nearly 35% participating in free and reduced meals.
During that time, I served as a school psychologist with SESS, a role I held for 13 years. The SESS team had 9 psychologists, 8 behavior support teachers, and 13 social workers who provided mental health and behavioral support to the approximately 600 students receiving our services, located in 22 comprehensive school buildings throughout the district. These students typically have educational codes of emotional disability, autism, learning disabilities, or other health impairments and have significant emotional and behavioral challenges requiring special education instruction and services.
Identifying an Opportunity to Link Data to Value
Our supervisor felt that we needed to explore improved data collection processes in order to support the enhanced focus from our new superintendent and special education leadership on data-based decision-making and outcomes. We discussed what specific benefits such a data collection could provide, including: having a deeper understanding of our services to students and meeting their unique needs, helping each SESS staff member look at patterns in our own work with our students and at our schools, sharing as appropriate with resource teachers and administrators to help them understand specific school trends and patterns, standardizing the monthly logs that the social workers and psychologists were already required to do in documenting our service delivery, and providing an empirical way to document the need for and outcomes of our services.
Building Consensus on What Data to Collect
My supervisor tapped me to lead the project because I had led similar, smaller efforts in the past. While the scale of the task seemed somewhat daunting, I felt enthusiastic about using my data-use experience as a school psychologist. Importantly, though, we knew this had to be a team effort. We formed a data committee including myself, another psychologist, and a social worker, in order to formulate what variables we wanted to collect data on and the methodology for our collection. We also reached out to the MCPS special education research and data analyst to see what information and guidance she could provide us. We sought input from the 30 SESS staff members and our supervisor, and decided on variables we wanted to look at for each of our 22 school programs on a monthly basis. These included types of key school program indicators and factors that might influence these indicators; the array of services provided and their intensity, frequency, and duration; and who was providing the services.
Key school program indicators. We examined the total number of monthly occurrences at each school program of: Risk protocols (when students make threats towards themselves or others), crisis center referrals, inpatient psychiatric hospitalizations, partial psychiatric hospitalizations, emergency evaluation petitions, physical restraints, suspensions, child protective services referrals/open cases, police contact/department of juvenile services involvement, home visits, referrals for more intensive school placements, high school eligibility status (GPA of 2.0 or higher, allowing eligibility to play sports), and graduation/drop out status.
School environment and practices factors. Elements examined within this subset of data were used to: view how school programs compare in their intensity, month by month, and eventually year by year; examine why some schools have many eligible students and what best practices promote this; look at why students are being referred for more intensive school placements, which programs are sending them (i.e., elementary, middle, or high school); and what we could do about this.
Services provided. We also wanted to collect data concerning the date, type, and amount of service time that is provided for each student by each SESS school psychologist and social worker. These included: individual counseling/problem solving, group counseling, lunch bunches, crisis intervention, parent contacts, consultations with outside provider, home visits, psychological assessments, IEP and program intake meetings, and psychological report writing, FBAs, and mental health referrals. Elements examined within this data subset included: the amount and type of service each student has received and on which days they received it; the amount and type of service each school has received and on which days they received it; the different array of services that social workers and psychologists were providing; the average amount of time spent in each service; and the total time spent in each service across individual providers, groups of providers, and the SESS unit as a whole.
Capitalizing on Technology
Because all of the SESS staff had access to an MCPS Google account and had been trained in using Google documents and Google forms, we wanted to capitalize on this technology. We created a MCPS Google website utilizing Google Forms in order to collect data from the SESS unit on the variables of interest. The information that the school psychologists, social workers, and behavior support teachers entered into Google Forms was then automatically uploaded into Google Sheets. The information was also automatically entered into Google Charts, which created attractive graphs of the data. We could also view the data in numeric form. The charts are dynamic, so that as each individual entered information, the graphs and numerical data instantly adjusted to account for the new information. The data are private and only individuals who received the invitation to view the website were able to enter and view the data.
The challenge in creating such a database and website was well beyond our skill-set. We sought out the help of a local not-for-profit technology consortium in learning how to do the basic coding that was required. In addition, using a variety of Google tutorials and videos was also beneficial. Nonetheless, creating the website and database was a very time-intensive endeavor that required significant effort above and beyond our normal workload. In addition, despite the training that staff had received in using Google Forms, there was an initial learning curve and anxiety in understanding how to enter the data. A training time was set up so that all staff would have the chance to practice entering data before the project commenced. We identified mentors within the unit for any staff who still felt uncomfortable with the new technology following the training.
Beyond Technology Challenges, Getting Staff on Board
There were a great many challenges with the data initiative, beyond the initial technological barriers. Some staff members expressed reluctance or concerns about embarking on this effort. Not everyone felt that, as a whole, collecting comprehensive data about the students and schools was relevant to our work. We knew that any sort of change often leads to some wariness, so we made a significant effort to solicit feedback from all stakeholders on an ongoing basis. We made revisions to the data collection process in order to address everyone's feedback. For example, we added an additional variable to look at the amount of inclusion time that elementary students receive each quarter based on the feedback suggesting we include more positive measures of student success. Another ongoing dilemma was getting staff to utilize and make meaning of the data to use in their work. Some staff members have utilized the data frequently while others have needed more support to see how this data could be a useful tool in their specific work with students. To help move this forward, we scheduled quarterly data chats within the unit to discuss the data and talk about how we could use it. Our supervisor began using the data at monthly meetings with staff members at each school in order to continue to promote the usage of the data website. As with so many realities in education, having all staff members consistently enter their data in a timely manner proved to be challenging because of busy schedules and the intense demands of our school programs. Our supervisor sent out both group and individual e-mail reminders to staff who were late in doing their data entry in order to try and maintain an up-to-date database. Another common concern was the amount of time it took to enter the information for the school psychologist and social worker logs. We found some ways to streamline the data entry process so that it would cut down the amount of time it would take each week.
Shining a Light on the Power of Data
Despite the many obstacles we have encountered, our efforts have not gone unrecognized. The school research and data analyst was impressed with our website and reached out to the director and associate superintendent of the Office of Special Education to share our work. We were then invited to speak about our website at a special education leadership meeting. Rather than just showing the website and its functionality, we were encouraged to share interactive stories about how using the data from the website guided us in our decision-making. We were able to present case examples that illuminated how data guided us to try more intensive interventions with a struggling student; why we thought one school program had many more academically ineligible students than the rest, and changes that we thought would help; and how one or two challenging students in any school program could consume many hours of mental health support, leading other students within the program to have less access. Later in the year, leadership reached out to us once again, asking for more information about our project to potentially share with the MCPS chief academic officer as a possible model of how other special education programs could collect and utilize data. I cannot overstate how important and encouraging it has been throughout the process to have the support and feedback from MCPS leaders, even when the process met with challenges.
The project has been a significant undertaking, but there have been many small victories along the way. I am thankful for the opportunity to showcase school psychologists as leaders in data-based decision making in our school district, use new technology to advocate for our students and school programs, and further develop as a professional.
Julie Shields, PhD, is a school psychologist in Montgomery County Public Schools, Maryland