Communiqué

Facilitators and Barriers to the Implementation of Mental Health Evidence-Based Interventions

Volume 44 Issue 1

By Taylor B. Hicks-Hoste

Schools have been implicated as the primary setting in which to provide evidence-based mental health services to our nation's youth (Rones & Hoagwood, 2000). Due to recent federal policies and initiatives, such as the School-Based Health Center Program under the Affordable Health Care Act, addressing students' mental health outcomes within school settings has become a priority. The field of school psychology has made great strides in the identification of efficacious interventions that can positively support students' emotional, behavioral, social, and academic outcomes (Forman et al., 2013). Furthermore, increased national and state-level funding has been devoted to initiatives that aim to enhance school-based mental health supports (Chambers, Ringeisen, & Hickman, 2005). However, despite these efforts, mental health evidence-based practices (EBPs) and evidence-based interventions (EBIs) are frequently underimplemented in school settings (Ennet et al., 2003; Hicks, Shahidullah, Carlson, & Palejwala, 2014). This disconnect is highly problematic, given that students cannot reap the benefits of EBPs and EBIs if they are not implemented, or implemented with fidelity, in schools.

The uptake of EBPs and EBIs in complex school settings is difficult, given that implementation is a multistage and complex process and there are multiple contextual influences that must be accounted for during this process. Multiple implementation frameworks have been proposed within the literature (Myers, Simonsen, & Sugai, 2012). Fixsen and colleagues (2005) propose one such framework that focuses on the WhatWho, and How of implementation. First, the What of the intervention includes the intervention's empirical nature, content, and procedures (Ogden & Fixsen, 2014). Second, the Who refers to the individual, or groups of individuals, who actively work to implement programs or practices, often known as implementation teams or change agents (Ogden & Fixsen, 2014). Finally, How implementation procedures are supported in practice is understood by identifying the variables that facilitate or impede this process (Ogden & Fixsen, 2014). School psychologists often serve as change agents within schools and are well equipped to navigate the What and Who stages of the implementation process (Forman et al., 2013). While school psychologists also have the skills to engage in How procedures (i.e., identifying important implementation facilitators and barriers), less is known about the ways in which school psychologists engage in this type of assessment. More specifically, the question remains: What types of implementation barriers are school psychologists identifying, and how are they able to address and navigate barriers?

The How of the implementation process is directly related to the concept of implementation science, which can be defined as the "scientific study of methods to promote the systematic uptake of research findings and evidence-based practices into professional practice and public policy" (Eccles & Mittman, 2006). Within the field of school psychology, implementation science research focuses on understanding the processes and factors related to the successful transportation of an EBP/EBI into school contexts (Forman et al., 2013). While other disciplines, such as the field of medicine, have made great advances in implementation science research, the study of implementation science is relatively new to the field of school psychology. Hence, less is known about the specific variables that serve to facilitate or impede schools' implementation efforts of mental health supports for students.

School psychologists employed in school settings should strive to collaborate with researchers and to partake in formal transportability research efforts in order to further this line of research. While continued efforts to conduct transportability research within school settings is critical, the reality is that it is becoming increasingly more difficult to conduct research within schools, considering that standards for educators are increasing, while resources (e.g., funding, time) are decreasing; hence, school personnel often have limited energy, time, and resources to participate in implementation science research (Greene, 2015). While it may not be feasible to partake in formal research processes, school psychologists must still be knowledgeable about the factors that influence implementation efforts and critically examine these factors in relation to their school's own intervention outcomes. This may be particularly important given that schools often have to adapt interventions to meet the needs of their local context (Fixsen et al., 2005). For instance, schools may have fewer resources to dedicate to implementation efforts than those suggested by EBI program developers or those studied within efficacy research. Similarly, schools may need to adapt an EBI to make it more appropriate for their target population of staff or students. These adjustments may compromise the integrity of the intervention and, as a consequence, negatively detract from intervention outcomes. Hence, school psychologists must not only be aware of these variables, but also closely monitor their relationship to intervention processes and outcomes. In doing so, school psychologists can make evidence-based decisions related to their adoption and adaption of EBPs/EBIs by accounting for those factors. The National Implementation Research Network (NIRN; http://nirn.fpg.unc.edu) is an online resource that school-based practitioners can use to help guide their understanding of, and thinking about, implementation variables. The following sections will review the NIRN's proposed framework for conceptualizing implementation facilitators and barriers.

Implementation Drivers

Based on a systematic review of the literature, the NIRN has proposed a conceptual framework of implementation drivers, in which there are three main categories of drivers: organizational drivers, leadership drivers, and competency drivers. A key component of this suggested framework is that it conceptualizes implementation as a multistage, nonlinear process in which the stages of implementation (i.e., exploration and adoption, program installation, initial implementation, full implementation) and the implementation drivers interact with one another continuously over time. Hence, school psychologists must be aware of not only the effects of each single implementation driver, but also the interplay between them.

Organizational drivers. Organizational drivers are the changes made at the systems level to ensure that work environments are well equipped and prepared to implement interventions effectively. This includes implementing best practices approaches for systems intervention, facilitative administration, and decision support data systems (Forman et al., 2013).

Systems intervention. The economic, social, political, and cultural climate of a system can serve to influence the priorities of that system, as well as the availability of personnel and financial resources needed to support the activities of that system (Fixsen et al., 2005). Within schools, the political landscape can strongly influence the everyday practices and responsibilities of school staff. For example, the No Child Left Behind Act (NCLB) of 2001 placed an increased emphasis on school accountability, in terms of targeting students' academic outcomes through increased use of evidence-based practices. Interestingly enough, developers of behavioral intervention programs have cited educational policy, such as NCLB, as a significant barrier to the successful adoption and implementation of behavioral EBIs in schools (Forman, Olin, Hoagwood, Crowe, & Saka, 2009). Such policies might also influence the priorities of school professionals, as evidenced by the finding that many school psychologists indicate their primary responsibility to be conducting assessment related to special education determination, which then limits the amount of time and energy they have to engage in EBPs related to students' mental health and behavioral supports (e.g., Hosp & Reschly, 2002; Lewis, Truscott, & Volker, 2008). However, given the recent expansion of mental health services to school-based health centers under the Affordable Health Care Act, school psychologists may find their responsibilities shifting to include a greater focus on social-emotional and behavioral supports.

Facilitative administration. Facilitative administration refers to the leadership practices that aid the implementation process, with specific attention to the ways in which administrators establish hospitable environments for staff, including the establishment of specific procedures. Within the field of education, district-level and school building-level management personnel (e.g., principals, leadership teams) play a key role in shaping the context of EBI implementation, such as developing a readiness for change among key stakeholders (Elias, Zins, Graczyk, & Weissberg, 2003). As leaders within schools, principals make key decisions related to whether and how school staff members prioritize mental health services, the selection of emotional and behavioral intervention programs, and the types of funding and training school personnel are provided in selected programs (e.g., Han & Weiss, 2005). Furthermore, the quality of principal support has been found to be related the implementation quality of EBIs (Kam, Greenberg, & Walls, 2003).

Decision-support data systems. The importance of data collection and monitoring is well documented at the individual and systems levels. Data systems are essential, in terms of monitoring EBI program effectiveness and any adaptations made to the standard implementation protocol. Data systems can be used to improve school staff members' teaching practices, including improved efficiency, response to student needs, self-evaluation, and sense of collaboration (Wayman & Stringfield, 2006). Given their level of training and expertise, school psychologists can play an important role in setting up well-organized and efficient data collection systems, as well as in analysis and interpretation of the data.

Leadership drivers. Leadership drivers are the ways in which program leaders address issues or problems related to implementation processes, such as motivational issues among staff or technical issues, such as a lack of resources (Ogden & Fixsen, 2014). Lack of time, lack of necessary resources, and financial constraints have been rated as the most serious barriers to the school-based implementation of behavioral EBIs by Nationally Certified School Psychologists in a recent national survey study (Hicks et al., 2014). Similarly, previous research on the implementation of specific EBI programs has found that practitioners rate inadequate resources, such as a lack of funding, classroom space, and qualified personnel, as significant barriers to successful implementation (Reinke, Stormont, Herman, Wang, Newcomer, & King, 2011; Thaker et al., 2008). Furthermore, adaptations to EBIs are often made to account for a lack of necessary resources (Lilienfield, Ammirati, & David, 2012). It is highly likely that school-based practitioners will encounter similar resource-related barriers during the implementation process; thus, it is important that leaders of these efforts, such as school psychologists, are able to navigate and address the barriers uniquely related to their specific context by being proactive, responsive, and flexible during the planning and implementation processes (Durlak & DuPre, 2008; Elias et al., 2003). For example, previous research has found that strong leaders of implementation must be active in addressing barriers by restructuring documentation and policies to better support staff in effective adoption and implementation of specific procedures (Torrey, Bond, McHugo, & Swain, 2011).

Competency drivers. School psychologists may be most familiar with competency drivers, given their skills in assessment and consultation. Competency drivers are personal implementer factors, such as individual attitudes toward innovation, skill proficiency, and access to implementation supports (e.g., professional development training, coaching) that influence implementation outcomes (Durlak & DuPre, 2008).

Staff selection. Practitioners' readiness to change, self-efficacy, and program acceptability can influence their willingness to try new practices (Aarons, 2004). Those who recognize a need for an intervention or believe the implementation will result in positive outcomes are more likely to adopt and implement EBIs with greater fidelity (Durlak & DuPre, 2008). For example, in a national study of middle school teachers' (n = 1905) implementation of a substance use prevention curriculum, teachers' self-reported attitudes toward the curriculum (i.e., how much they liked teaching substance use prevention lessons, how effective they believed the curriculum to be) was significantly related to the fidelity of their implementation of the program (Ringwalt et al., 2003). Given that practitioners' attitudes and beliefs may be predictive of their willingness to learn and implement behavioral EBIs, it may be important to take this into account when allotting implementation resources, such as for staff training or coaching support. This will help to ensure that resources are allotted either to (a) those who are most receptive to altering their practices and receiving training in EBIs or (b) those who are most resistant to altering behaviors and practices and thus need the most intensive level of intervention support. Unfortunately, the literature is underdeveloped in terms of examining whether or not formal assessment of staff attitudes can lead to more targeted training efforts and improved implementation outcomes. However, this may still be an important area for school psychologists to address in their school-based practice.

Professional development training. The professional development training practitioners receive in EBPs is important to provide them with the necessary skills required for implementation (Fixsen et al., 2005). Unfortunately, teachers rarely receive the preservice training, or follow-up coaching supports, needed in how to support students' mental health and behavioral needs (Koller & Bertel, 2006). Although teachers self-identify as the main implementers of classroom-based behavioral interventions, they report a general lack of experience and training in how to effectively provide these types of supports (Reinke et al., 2011). For example, in a survey study of early childhood and elementary school teachers (n = 363), only 44% of respondents indicated that they felt confident in selecting and implementing interventions for targeted students (Stormont, Reinke, & Herman, 2011). Providing professional development training is an important way in which school psychologists can address teachers' lack of knowledge and skill in the delivery of mental health supports. Previous research has identified a number of effective professional development training techniques, such as didactic instruction, modeling, role-playing, and in-class direct training methods (Slider, Noell, & Williams, 2006).

Coaching. Coaching supports serve as an important supplement to professional development training, as a way to reinforce and enhance newly taught and acquired skills (Kretlow & Bartholomew, 2010). This is especially important, given that teachers are not likely to successfully implement novel strategies following a single professional development training session. Coaching supports typically involve an expert (e.g., lead teacher, skilled peer, supervisor) providing individualized support to a novice, in a specific skill domain (Joyce & Showers, 2002). Coaching supports are especially effective when they include direct observations of teachers' implementation of strategies followed by performance feedback (Noell et al., 2005). Research has shown that coaching can be used to improve teachers' implementation of reading interventions (Nelson-Walker et al., 2013; Vernon-Feagans et al., 2012), math instruction (Kretlow, Wood, & Cooke, 2009), and classroom management strategies (Filcheck, McNeil, Greco, & Bernard, 2004; Reinke et al., 2014; Zan & Doneagan-Ritter, 2014). Again, school psychologists' knowledge and skills in the provision of consultation services easily translate to the provision of strong coaching supports.

Implications for Practice

Not only should school psychologists be aware of organizational, leadership, and competency drivers and the ways in which they might influence school-based mental health EBP/EBI efforts, but they should also be aware of how their professional skills situate them to be strong leaders and advocates in identifying and navigating these potential barriers. For example, with regard to organizational drivers, school psychologists are highly trained in systematically collecting and analyzing data to inform decisions related to policies and practices. Hence, they are able to provide a best-practices approach in how to establish and utilize decision-support data systems to monitor intervention outcomes at both the systems and individual levels. As school-based consultants, school psychologists are also well positioned to formally or informally assess school personnel's level of buy-in related to new intervention approaches; when low levels of buy-in are related to feelings of low self-efficacy or lack of training support, school psychologists are often able to equip school staff with the training or coaching supports needed to improve acceptability, which has significant effects on implementation fidelity and outcomes.

As the implementation science literature advances within the fields of education and school psychology, new insights will be gained with regard to the ways in which implementation drivers uniquely relate to school-based mental health intervention efforts, as well as effective strategies for addressing those factors. In the meantime, school psychologists should use their knowledge of, and skills in, assessment, consultation, and intervention to systematically evaluate the influence of these drivers within their local contexts and make evidence-based decisions in how to best navigate them so that schools can successfully provide students with effective mental health supports.

To aid school psychologists and other implementation leaders within schools, the NIRN offers many valuable resources. In particular, the NIRN has developed an assessment tool, referred to as "Implementation Drivers: Assessing Best Practices." This assessment tool can be used by implementation teams to evaluate each of the implementation drivers previously described (i.e., leadership drivers, organization drivers, competency drivers) in relation to EBI implementation. Furthermore, this assessment tool can be used throughout the implementation process to assess how implementation teams are accounting for implementation drivers at the exploration, installation, and implementation stages. When using this tool, implementation teams are asked to answer the question, "To what extent are best practices being used?" for each implementation driver. For example, is professional development training provided before the practitioner attempts to, or is required to, use a new program or practice? Are coaches fluent in the intervention? What types of accountability measures are in place? Rating various components of each implementation driver yields a "Best Practices Score," which allows implementation teams to make evidence-based decisions about whether or not best practices are being employed. This, then, can be used to help evaluate and interpret intervention outcomes.

This tool can also help implementation teams assign implementation responsibilities to specific team members and establish a timeline for which implementation drivers should be prioritized and addressed first. However, it is recommended that this assessment tool be completed by a group of overseeing implementation team members who are responsible for developing, monitoring, and improving intervention implementation; hence, practitioners who are responsible for the actual implementation procedures should not be included in the evaluation process, as this tool is not meant to be a self-report assessment measure.

In sum, school psychologists can play an important role in the identification, adoption, implementation, and sustainability of important mental health EBPs/EBIs. Having knowledge of implementation drivers that either serve to facilitate or impede implementation processes in the unique contexts of schools, as well as effective methods of assessing and addressing these factors, can help to promote successful implementation, which will likely result in important and beneficial outcomes for both school staff and students.

References

Aarons, G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6, 61-74. Retrieved fromhttp://www.springer.com/public+health/journal/11020

Chambers, D. A., Ringeisen, H., & Hickman, E. E. (2005). Federal, state, and foundation initiatives around evidence-based practices for child and adolescent mental health. Child and Adolescent Psychiatric Clinics of North America, 14(2), 307-327. doi:10.1016/j.chc.2004.04.006

Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327-350. doi:10.1007/s10464-008-9165-0

Eccles, M. P., & Mittman, B. S. (2006). Welcome to implementation science. Implementation Science, 1, 1. doi:10.1186/1748-5908-1-1

Elias, M. J., Zins, J. E., Graczyk, P. A., & Weissberg, R. P. (2003). Implementation sustainability, and scaling up of social-emotional and academic innovations in public schools. School Psychology Review, 32, 303-319. Retrieved fromhttp://www.nasponline.org/publications/spr/index.aspx?vol=32&issue=3

Ennett, S. T., Ringwalt, C. L., Thorne, J., Rohrbach, L. A., Vincus, A., Simons-Rudolph, A., & Jones, S. (2003). A comparison of current practice in school-based substance use prevention programs with meta-analysis findings. Prevention Science, 4, 1-14. doi:10.1023/A:1021777109369

Filcheck, H. A., Mcneil, C. B., Greco, L. A., & Bernard, R. S. (2004). Using a whole-class token economy and coaching of teacher skills in a preschool classroom to manage disruptive behavior. Psychology in the Schools, 41, 351-361. doi:10.1002/pits.10168

Fixsen, D. L., Naoom, S. F., Blasé, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication @231).

Forman, S. G., Olin, S., Hoagwood, K., Crowe, M., & Saka, N. (2009). Evidence-based intervention in schools: Developers' views of implementation barriers and facilitators. School Mental Health: A Multidisciplinary Research and Practice Journal, 1, 26-36. doi:10.1007/s12310-008-9002-5

Forman, S. G., Shapiro, E. S., Codding, R. S., Gonzalez, J. E., Reddy, L. A., Rosenfield, S. A., ... Stoiber, K. C. (2013). Implementation science and school psychology. School Psychology Quarterly, 28, 77-100. doi:10.1037/spq0000019

Greene, J. A. (2015). Serious challenges require serious scholarship: Integrating implementation science into the scholarly discourse. Contemporary Educational Psychology, 40, 112-120. doi:10.1016/j.cedpsych.2014.10.007

Han, S. S., & Weiss, B. (2005). Sustainability of teacher implementation of school-based mental health programs. Journal of Abnormal Child Psychology, 33, 665-679. doi:10.1007/s10802-005-7646-2

Hicks, T. B., Shahidullah, J. D., Carlson, J. S., & Palejwala, M. H. (2014). Nationally certified school psychologists' use and reported barriers to using evidence-based interventions in schools: The influence of graduate program training and education. School Psychology Quarterly. Advance Online Publication. doi:10.1037/spq0000059

Hosp, J. L., & Reschly, D. J. (2002). Regional differences in school psychology practice. School Psychology Review, 31, 11-29. Retrieved from http://www.nasponline.org/publications/spr/sprmain.aspx

Joyce, B., & Showers, B. (2002). Student achievement through staff development (3rd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.Kam, C. M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4, 55-63. doi:10/1023/A:1021786811186

Koller, J. R., & Bertel, J. M. (2006). Responding to today's mental health needs of children, families, and schools: Revisiting the preservice training and preparation of school-based personnel. Education and Treatment of Children, 29, 197-217. Retrieved from http://www.educationandtreatmentofchildren.net

Kretlow, A. G., & Bartholomew, C. C. (2010). Using coaching to improve the fidelity of evidence-based practices: A review of studies. Teacher Education and Special Education, 33, 279-299. doi:10.1177/0888406410371643

Kretlow, A. G., Wood, C. L., & Cooke, N. L. (2011). Using in-service and coaching to increase kindergarten teachers' accurate delivery of instructional units. The Journal of Special Education, 46, 234-246. doi:10.1177/0022466909341333

Lewis, M. F., Truscott, S. D., & Volker, M. A. (2008). Demographics and professional practices of school psychologists: A comparison of NASP members and non-NASP school psychologists by telephone survey. Psychology in the Schools, 45, 467-482. doi:10.1002/pits.20317

Lilienfield, S. O., Ammirati, R., & David, M. (2012). Distinguishing science from pseudoscience in school psychology: Science and scientific thinking as safeguards against human error. Journal of School Psychology, 50, 7-36. doi:10.1016/j.jsp.2011.09.006

Myers, D. M., Simonsen, B., & Sugai, G. (2012). Increasing teachers' use of praise with a response-to-intervention approach.Education and Treatment of Children, 34, 35-59. Retrieved from http://wvupressonline.com/journals/etc

Nelson-Walker, N. J., Fien, H., Kosty, D. B., Smolkowski, K., Smith, J. L. M., & Baker, S. K. (2013). Evaluating the effects of a systematic intervention on first-grade teachers' explicit reading instruction. Learning Disability Quarterly, 36, 215-230. doi:10.1177/0731948712472186

Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Gatti, S. L., Williams, K. L., ... Resetar, J. L. (2005). Treatment implementation following behavioral consultation in schools: A comparison of three follow-up strategies. School Psychology Review, 34, 87-106. Retrieved from http://www.nasponline.org/publications/spr/index.aspx?vol=43&issue=2

Ogden, T., & Fixsen, D. L. (2014). Implementation science: A brief overview and a look ahead. Zeitschrift fur Psychologie, 222, 4-11. doi:10.1027/2151-2604/a000160

Reinke, W. M., Stormont, M., Herman, K. C., Wang, Z., Newcomer, L., & King, K. (2014). Use of coaching and behavior support planning for students with disruptive behavior within a universal classroom management program. Journal of Emotional and Behavioral Disorders, 22, 74-82. doi:10.1177/1063426613519820

Reinke, W. M., Stormont, M., Herman, K. C., Puri, R., & Goel, N. (2011). Supporting children's mental health in schools:Teacher perceptions of needs, roles, and barriers. School Psychology Quarterly, 26, 1-13. doi:10.1037/a0022714

Ringwalt, C. L., Ennett, S., Johnson, R., Rohrbach, L. A., Simons-Ruldoph, A., Vincus, A., & Thorne, J. (2003). Factors associated with fidelity to substance use prevention curriculum guides in the nation's middle schools. Health Education & Behavior, 30, 375-391. doi:10.1177/1090198103253627

Rones, M., & Hoagwood, K. (2000). School-based mental health services: A research review. Clinical Child and Family Psychology Review, 3, 223-241. Retrieved from http://link.springer.com/journal/10567

Slider, N. J., Noell, G. H., & Williams, K. L. (2006). Providing practicing teachers classroom management professional development in a brief self-study format. Journal of Behavioral Education, 15, 215-228. doi:10.1007/s10864-006-9033-7

Stormont, M., Reinke, W., & Herman, K. (2011). Teachers' characteristics and ratings for evidence-based behavioral interventions. Behavioral Disorders, 37, 19-29. Retrieved from http://www.ccbd.net/publications/behavioraldisorders

Stormont, M., Smith, S. C., & Lewis, T. J. (2007). Teacher implementation of precorrection and praise statements in Head Start classrooms as a component of a program-wide system of positive behavior support. Journal of Behavioral Education, 16, 280-290. doi:10.1007/s10864-007-9040-3

Thaker, S., Steckler, A., Sanchez, V., Khatapoush, S., Rose, J., & Hallfors, D. D. (2008). Program characteristics and organizational factors affecting the implementation of a school-based indicated prevention program. Health Education Research, 23, 238-248. doi:10.1093/her/cym025

Torrey, W. C., Bond, G. R., McHugo, G. J., & Swain, K. (2011). Evidence-based practice implementation in community mental health settings: The relative importance of key domains of implementation activity. Administration and Policy in Mental Health and Mental Health Services Research, 39, 353-364. doi:10.1007/s1488-011-0357-9

Vernon-Feagans, L., Kainz, K., Amendum, S., Ginsberg, M., Wood, T., & Bock, A. (2012). Targeted reading intervention: A coaching model to help classroom teachers with struggling readers. Learning Disability Quarterly, 35, 102-114. doi:10.1177/0731948711434048

Wayman, J. C., & Stringfield, S. (2006). Technology-supported involvement of entire faculties in examination of student data for instructional improvement. American Journal of Education, 112, 549-571. doi:10.1086/505059

Zan, B., & Donegan-Ritter, M. (2014). Reflecting, coaching, and mentoring to enhance teacher child interactions in Head Start classrooms. Early Childhood Education Journal, 42, 93-104. doi:10.1007/s10643-013-0592-7


Taylor Hicks-Hoste is a fifth year school psychology doctoral student at Michigan State University