Populations Students Early Career Families Educators View My Account
Skip Navigation LinksNASP Home Publications Communiqué Volume 39, Issue 6 Should Student Achievement Data Be Considered in the Evaluation of School Psychologists?

Advocacy in action

Should Student Achievement Data Be Considered in the Evaluation of School Psychologists?

By Anastasia Kalamaros Skalski

A growing number of states are pursuing teacher evaluation policies and practices that factor student achievement data into retention and promotion decisions. This is largely in response to the Race to the Top competition, a federal grant program that has awarded millions of dollars to states that were willing to prioritize President Obama’s school improvement agenda within their school reform efforts. The Race to the Top requirements asked states to address teacher quality as a priority, and in scoring applications for the awards, gave a large number of points to applications that included teacher evaluation procedures that assessed student growth as a significant factor. Within the application, student growth was defined as “the change in student achievement … for an individual student between two or more points in time.” Most educators consider including an assessment of student growth in response to instruction (or intervention) as informative and a reasonable consideration as a piece of a person’s professional evaluation. The debate arises over how much weight should be given to student achievement and what measures or indicators best represent student growth. The Race to the Top application rules favor utilizing students’ test scores on state assessments as evidence for this growth. This emphasis on the use of high stakes test data has raised many concerns and resulted in significant questions about the validity and reliability of such practices.

This increasing emphasis on student achievement in policy has also led to more use of a method called value added modeling or value added assessment (VAA). This technique utilizes complex formulas that incorporate student test scores in an effort to determine the difference between a student’s predicted scores and their actual scores. The amount of difference typically results in some assignment of credit or blame for the result. Perhaps the most well known application of this approach occurred in the past year when the Los Angeles Times published an article on August 14, 2010, utilizing a VAA method for classifying Los Angeles Unified School District teachers into categories of effectiveness ranging from “least effective” to “most effective” (Feich, 2010). Many policy makers publicly lauded this article and the techniques used. It is interesting to note that the findings published by the Los Angeles Times have since been called into question by researchers at the University of Colorado who were unable to replicate the findings and questioned the reliability and validity of their application (Briggs & Domingue, 2010).

A National Education Policy Center report summarizes the collective caution of researchers addressing this new trend by noting that

A steady stream of authoritative statements from the nation’s foremost researchers has cautioned against the use of VAA to make high-stakes decisions, both because of remaining methodological challenges and because an overly narrow focus on standardized test scores as the most important—and in some cases, only—student outcome measure is not aligned with what the field agrees an effective teacher does (Hinchey, 2010, p. 5).

Despite these cautions, a growing number of states are passing legislation that prioritizes the use of student test scores in the evaluation of teacher effectiveness. A study by the RAND Corporation (Steele, Hamilton, & Stecher, 2010) examined five different systems that were currently being utilized including two at the state level (Tennessee and Delaware) and three at the district level (Denver, Colorado; Hillsborough County, Florida; and Washington, DC). This report addressed a variety of validity and reliability concerns that impact the use of VAA and examined how state and local educational agencies are trying to weave together the research into policy and practice. It also mentioned that these methodologies would likely be applied to nonteaching personnel.

And in fact, this type of legislation is quickly spilling over into school psychology, and NASP members and leaders are becoming increasingly concerned that our association needs to release related policy and practice guidance. Not only is NASP concerned, but the leadership of other specialized instructional support personnel (SISP, also known as pupil or related services professionals) such as school counselors, speech and language therapists, and school nurses are engaging in discussions about how to respond to these emerging policies and their application to their professionals. Critical questions are being raised like “How can the services of a school nurse be fairly evaluated by a student’s performance on a high stakes test?” and “What sources of student achievement data exist that are sensitive to the effectiveness of interventions implemented by the school psychologist?” In response to the concerns of these professionals, the National Alliance of Pupil Services Organizations (NAPSO) has crafted a set of policy guidelines to assist states in considering how to best apply student achievement outcomes to educator evaluation systems. At the time of the writing of this article, a final draft was under consideration for approval by the members of NAPSO and included the following excerpted recommendations.

In an effort to promote fair and appropriate performance assessments of SISP, NAPSO recommends policy makers recognize and support meaningful evaluation systems that provide school professionals with relevant, supportive, and instructive feedback. NAPSO recommends these systems include the following:

SISP involvement in the creation of evaluation systems. Professionals with specific expertise in each SISP discipline must be involved in the creation of evaluation or performance appraisal systems designed to determine their professional competence for these purposes. Systems should be research-based and consider evaluation and supervision models supported by the national professional organizations responsible for advancing research and practice for each distinct profession.

  • Support continued research in developing evaluation systems for SISP. The Institute for Education Sciences, through research and demonstration grants, should examine the appropriateness of professional appraisal systems for each SISP discipline. These grant programs should focus on evaluating professional performance as it relates to student performance outcomes.
  • Evaluator expertise in SISP roles, responsibilities, and job functions. It is critical that any evaluation system include evaluators with expertise (e.g., specialized training, related professional development, etc.) specific to the SISP position they are evaluating. Evaluators must understand the unique practices and foundational knowledge of SISP and the specific demands, needs, and requirements of each position, so that useful and meaningful feedback is provided.
  • Utilize multiple measures in determining professional performance. Research supports the idea that performance assessment systems are most reliable when evaluators utilize multiple measures for the evaluation of professional performance as opposed to single indicators such as student standardized test scores. Other measures—such as visual observation; examples of student work; and surveys of interactions with families, community, peers, and staff— contribute to a more reliable measure for professional performance.

The complete set of NAPSO policy recommendations for Utilizing Multiple Measures in Determining Professional Performance of Specialized Instructional Support Personnel can be reviewed at www.napso.org.

At the NASP convention in San Francisco, our Executive Council and Delegate Assembly reviewed these NAPSO recommendations and began to chart a course for how school psychologists might best respond based upon existing research and best practices. As with every public policy initiative, the policies and accompanying practices that are ultimately proposed at the state and local levels will be largely due to your leadership and “advocacy in action.” Are you ready to demonstrate how your services provide value in this era of unprecedented accountability and educational reform? NASP needs you to know, understand, and be able to communicate how your work matters for kids. Being able and willing to regularly gather data informing this issue and contributing your expertise to the policy dialogues going on are the building blocks for effective professional practice and educational reform.

References

Briggs, D., & Domingue, B. (2010, February). Due diligence and the evaluation of teachers: A review of the value added analysis underlying the effectiveness rankings of Los Angeles Unified School District teachers by the Los Angeles Times. A National Education Policy Center Report. Boulder, CO: University of Colorado.

Felch, J. (2010, August 14). Who’s teaching L.A.’s kids? Los Angeles Times. Retrieved from: http://www.latimes.com/news/local/la-meteachers- value-20100815,0,2695044.story

Hinchey, P. H. (2010, December). Getting teacher assessment right: What policy makers can learn from research. A National Education Policy Center Report. Boulder, CO: University of Colorado.

Steele, J. S., Hamilton, L. S., & Stecher, B. M. (2010). Incorporating student performance measures into teacher evaluation systems. Santa Monica, CA: The RAND Corporation.

Anastasia Kalamaros Skalski, PhD, is NASP Director of Public Policy. Advocacy in Action is a regular column dedicated to providing state associations and their school psychologist members with ideas on how they can become involved in legislative advocacy efforts. If you have a good idea you would like to share for this column, e-mail Stacy Skalski, Director of Public Policy, at sskalski@naspweb.org.