Preventing Disproportionality: A Framework for Culturally Responsive Assessment
By Amanda L. Sullivan
For as long as there has been special education, there has been racially based disproportionality in identification and placement coupled with the concern that some students may be inappropriately identified as disabled (Artiles, 1998). This is especially true for Black students in the categories of emotional disability (ED) and mild mental retardation (MR), where they have long been two to three times more likely to be identified than White students (Donovan & Cross, 2002). While the roots of disproportionality are far-reaching and varied, there has been much concern regarding the influence of cultural dissonance in referral and assessment practices on disparities.
In its 2002 report on disproportionality, the National Research Council recognized that the overrepresentation of Black children as ED may suggest the need to reexamine the procedures used for screening and identifying students (Donovan & Cross, 2002). This perspective is not unique; within the disproportionality literature, it is frequently suggested that professional practices in referral and identification are one cause for disparate rates of disability across racial groups (Artiles & Trent, 1994). Several authors (e.g., Artiles & Trent, 1994; Osher, Woodruff & Sims, 2002) have challenged the appropriateness of diagnoses of children of color as ED or MR in particular, contesting the reliance on professional judgment by practitioners who may be unintentionally influenced by cultural misconceptions. They suggest that racial bias and/or lack of cultural competence may be the basis of many educational disability diagnoses rather than any cognitive, psychological, physical, or affective deficits intrinsic to individual students. Indeed, there is much concern that practitioners confuse difference—particularly in terms of behavior, interaction styles, and funds of knowledge—with disability, thus inappropriately identifying racial minority students as disabled in schools.
School psychologists can contribute to the reduction of this ongoing phenomenon by ensuring that their own practices are sound. Given the increasing diversity of our nation’s schools, it is inevitable that practitioners will encounter students and families with backgrounds and experiences drastically different from their own. As such, the cultivation of knowledge, skills, and dispositions conducive to effectively serving diverse populations is essential to ensuring that our professional practices remain relevant and beneficial to the communities we serve. A broad approach to reducing educational inequities that is increasingly advocated for is culturally responsive practice (Klingner et al., 2005). Such an approach foregrounds considerations of culture in direct service, thus reducing the likelihood that mere (cultural) difference will be misinterpreted as disability when school psychologists and educators work with students from culturally and linguistically diverse backgrounds. This is particularly relevant to the discussion of ensuring the appropriateness of the educational diagnoses for culturally and linguistically diverse (CLD) children and youth. This article seeks to provide actionable steps for practitioners seeking to engage in culturally responsive assessment of CLD students for special education eligibility. While we acknowledge the need for a culturally responsive approach to educational and school psychological practices generally, this article emphasizes assessment as one dimension of such practice.
Cultural responsiveness moves beyond most notions of cultural competency and multiculturalism in its explicit emphasis on making our practices responsive to the differences we encounter. Thus, while critical awareness, pluralism, and knowledge of diversity are necessary, they are not enough. Culturally responsive professionals use their understanding of students’ cultural knowledge, experiences, and performance styles to bolster the students’ educational experiences. This necessitates careful analysis of the ways in which learning and performance are conceptualized and how they may need to be reconceptualized within the diverse settings in which we work to ensure students from minority backgrounds are not disadvantaged by certain mainstream assumptions and attitudes. A culturally responsive approach is based on the belief that all students have the potential to be successful in their academic endeavors when they are provided access to quality programs, services, and supports and their culture, language, heritage, and experiences are acknowledged, valued, and used to facilitate their learning and development (Klingner et al., 2005). Moreover, cultural responsiveness is also underpinned by an ethos of respect, care, responsibility, and substantive transformation of discriminatory systems. The key components of cultural responsiveness include (a) affirming diversity, (b) developing sociocultural consciousness, (c) engaging in critical reflection, (d) examining the cultures that shape schools, (e) promoting change, and (f) seeking professional learning.
The most basic component of culturally responsive assessment is respecting the cultural differences of students, families, and colleagues. Cultures differ in what constitutes desirable behavior, temperament, and traits, and those behaviors that deviate substantially from the prevailing norms of a given setting risk being pathologized. A culturally responsive individual recognizes that such deviations often represent groupbased differences in values and learned behaviors, instead of assuming they represent some kind of dysfunction. Thus, rather than thinking of certain differences as something that disadvantage a student, a culturally responsive practitioner adjusts expectations to allow for natural human diversity, tries to understand the possible value of those differences, and considers how students’ differences can be used to facilitate their success. This involves a willingness to recognize and accept that there are multiple legitimate ways of behaving in any given context. Thus, a culturally responsive school psychologist acknowledges the diversity of potential ways of knowing, learning, and interacting in educational settings.
Moreover, when working with diverse families, it is especially important to understand that there are a variety of ways in which individuals approach school involvement. While it is essential to establish a cooperative partnership with students’ families during the evaluation process, we should not expect that all parents will engage in the process in the same way, and we should be respectful of variations in families’ school involvement and rapport with education professionals. Recognizing the importance of avoiding stereotypes, it is appropriate to consult with cultural brokers (i.e., individuals who can provide information about a particular culture) when encountering individuals from cultural backgrounds with which you are unfamiliar.
Failure to affirm diversity can lead to a number of unintended and unwanted outcomes, including lowered expectations for minority students, failure to develop rapport with students and families from diverse backgrounds, inappropriate educational planning and service provision, and reliance on ineffective practices. As previously noted, there is much concern that disproportionality in special education, particularly overidentification of racially diverse students as ED or MR, may be the result of a widespread failure to acknowledge and value differences among students from racial and culturally diverse backgrounds. This includes failure to acknowledge that students come to schools with different funds of knowledge and learned behaviors, that “inappropriate” behaviors may be adaptive in home contexts, that individuals’ values and norms may not match what is expected within schools, and that these differences may not be indicative of a learning or emotional disability. Thus, engaging in culturally responsive assessment requires knowing and valuing the cultural differences of the students we serve—seeking to understand the cultural background and experiences of the individual and the implications thereof for learning and behavior in school. It also means suspending judgment regarding deficits and disability until we have ruled out the possibility that the problems are due to difference.
Develop Sociocultural Consciousness
A related component is development of sociocultural consciousness, or awareness of the social nature of learning and development. This perspective, heavily influenced by Vygotsky’s work, emphasizes that both learning and development occur through social interaction and are largely dependent on the environment in which the child lives. As such, understanding the contexts and relationships in which individuals develop are critical to understanding their behaviors. Within the assessment process, this includes acknowledging that educators’ conceptualization of what is consideredproblematic academic or social–emotional functioning is socially based, and that students’ behaviors, both social–emotional and academic, can be heavily influenced by the classroom environment. Consequently, practitioners must engage in critical reflection of how the culture of students, families, and educators may impact students’ educational experiences.
The sociocultural basis of disability is particularly relevant when working with students for whom diagnoses of learning disabilities, emotional disabilities, or mild cognitive impairments are considered. These disorders are generally regarded as being less biologically determined and more the result of the reciprocal interaction of the child and ecological factors (D’Amato, Crepeau-Hobson, Huang, & Geil, 2005). Thus, there is the potential that the child’s difficulties are heavily impacted by the educational environment, necessitating thorough consideration of educational opportunity and experiences. School psychologists should engage in ecological assessment (Hosp & Ardoin, 2008) before considering intrinsic explanations (i.e., disability). This will require examining how the educational environment—curriculum, instruction, behavior management, school climate—is structured in ways that support or hinder academic and/or social–emotional development. If the school curriculum is not relevant for all students, instruction is not provided in a culturally and linguistically appropriate manner, or certain groups are marginalized within the classroom or school, this creates a context that is not conducive to learning and cannot be ruled out as a determining factor in students’ academic or behavioral difficulties. Before undertaking an evaluation, we should always consider whether it is warranted given the quality of the student’s learning experiences. Ecological and cultural considerations should be foregrounded throughout the assessment process to ensure that external factors are ruled out as causes for the students’ academic difficulties. For example, in the Los Angeles Unified School District, multidisciplinary teams strive to conduct evaluations that account for students’ cultural differences via multimodal assessment and emphasize consideration of prereferral intervention; thus reducing ED identification for all students (Hernandez, Ramanathan, Harr, & Socias, 2008).
Critically Reflect on Perspectives and Practices
Engaging in culturally responsive assessment also requires reflecting on the ways our professional attitudes, norms, and values—and the ways we view and interact with children, families, and colleagues—are shaped by our own cultural backgrounds and educational experiences. Whenever we take on a new case, we should examine the potential biases that may be triggered by the characteristics of those with whom we work and the implications for our capacity to effectively serve those individuals. When initiating a case we should ask ourselves a series of questions:
- What is my knowledge base about this particular student?
- How might cultural differences affect my own perceptions of the student and his behaviors?
- What stereotypes might be activated by what I know of the student?
- To what extent does the student’s cultural background, level of acculturation, and language proficiency inform my selection of assessment procedures?
Throughout the process of engaging in any case, whether in assessment, consultation, or intervention, we should check our expectations for bias and self-fulfilling prophecies, as well as consider the biasing effect of the educational setting in which we work. That is, some sites develop characteristic responses to certain academic or social behaviors such that a given behavior almost always leads to a particular diagnosis regardless of the circumstances of a specific case.
Particularly essential to assessment is remembering the importance of the interpretations we apply to tests and to all of the factors that influence the interpretations we make day-to-day. Rigid approaches to interpretation risk pathologizing diversity by failing to account for cultural differences and ecological influences on individual’s responses within the assessment process. We must also acknowledge the limitations of normative comparisons, keeping in mind that when utilizing a standardized instrument, we assume that the examinee is similar to the standardization sample in terms of language, history, opportunity to learn, acculturation, and so on. To the extent that this is not the case, we must be cautious in making interpretations and predictions based on the observed scores.
We should be careful to adopt a problem-solving approach that begins with critical reflection on our own biases and suspend judgment. Adopting a disconfirmatory approach, that is, one in which we look for evidence that disproves potential diagnoses, can serve us well. Utilizing a grounded-theory approach can be particularly valuable because it avoids prespecified hypotheses and instead endeavors to develop a theory of the client and problem throughout the assessment process. The basic steps to this process are as follows:
- Collect data on past and current functioning (e.g., cognitive, adaptive, academic, behavioral), including teacher and family perspectives.
- Look for patterns and convergence across methods, sources, and time (e.g., records reviews, medical background, developmental history, parent and teacher interview data, self-reports, norm-referenced instruments).
- Summarize and formulate a theory of the problem.
- Test the theory against all available data.
- Collect additional data as needed.
- Consider potential diagnoses’ fit and added value.
- Consult with stakeholders regarding appropriateness.
Another area of reflection that can enhance our interactions with minority families is the cultural bases of our field and the potential implications for practicing in culturally responsive ways. As an example, consider the roots of cognitive assessment and special education and how this might impact our own attitudes towards assessment and diagnosis compared to those of students and families from marginalized backgrounds. These individuals may have negative attitudes toward our roles because of histories of discrimination related to testing and institutionalization (Murdock, 2007). If we seek to understand the histories of our practices, we can better navigate the resistance and suspicion we may encounter when engaging with those who come from racial or cultural groups that historically were negatively impacted by certain practices. For instance, the “six-hour retarded child” (President’s Commission on Mental Retardation, 1969) is not something we frequently discuss today, but if we consider its legacy, we will see it may continue to influence our interactions with others.
A culturally responsive approach also necessitates matching our assessment practices to the characteristics of the students and utilizing a variety of formal and informal methods to adequately tap the strengths and weaknesses of the student. Such methods include curriculum based measurement, portfolio assessment, criterion-referenced assessment, test–teach–retest, observation, and dynamic assessment. Assessment should always be driven by questions regarding learner needs, not the search for pathology and the quest to secure scores that fit disability criteria. Considerations of culture and individual differences should inform the assessment process and be accounted for throughout instruction and intervention planning. As Cartledge, Kea, and Simmons-Reed (2002) suggest, it is best “neither to make culture account for everything nor to discount its impact altogether” (p. 114). Rather, cultural considerations should be made throughout the process to ensure that incorrect interpretations and decisions are avoided.
Examine the Cultures That Shape Schools
In addition to reflecting on our own culture, it is essential to examine the multiple cultures that shape students’ experiences in schools and the ways in which they may influence achievement and behavior. This requires acknowledging that in any given classroom, there are numerous cultures at play: (a) the cultures from which each individual student and teacher come, (b) the culture of the classroom that is created by the individuals’ interactions within it, and (c) the culture of the school produced by the professional norms, attitudes, and values of those in authority. Thus, it is necessary to acknowledge the cultural nature of schooling and the norms and values that guide the professional behavior of teachers, administrators, staff, and school psychologists. When students’ behaviors violate these norms or values, they are likely to create friction within classrooms and may lead to efforts to “fix” the child’s problem, including referral for intervention or special education consideration. Further, we have to consider how the status quo may contribute to the disabling of certain children because they don’t fit in with what is expected, not because they have a true disability.
The school culture, and its corresponding values and norms, is often based on mainstream White culture, which can inadvertently disadvantage students from diverse backgrounds when they do not come to school with the knowledge, attitudes, or behaviors that are valued within the system. As noted by the National Research Council (Donovan & Cross, 2002):
Every institution, including the school, has practices (or a culture) that are taken for granted. Children whose households are imbued with the very same culture as that of the school are likely to have an advantage once they enter school. This advantage is likely to be maintained over time because the very taken-for-granted nature of many school practices reduces the likelihood that school personnel will attempt to explicitly instruct disadvantaged students as to the cultural norms of the school. Indeed, school personnel may be unaware of the particularistic nature of their unspoken, taken-for-granted assumptions and the actions that flow from them. (p. 183)
This issue can be particularly salient with the special education referral and evaluation process. School psychologists should give careful consideration to those “taken-for-granted” practices and the ways in which they may contribute to diagnoses by default—that is, those practices that are more heavily influenced by the habits of the setting than the needs of the student.
Promote Change Where Necessary
School psychologists should also consider how the school culture could be changed so that all students are supported and differences are not problematized by virtue of difference alone. Some school systems have assessment policies and practices in place that are not appropriate for many of the individuals or groups within the school community (e.g., standard assessment batteries that do not allow for test selection based on student characteristics, prereferral or assessment processes that do not allow for parent input). As school psychologists, we have an ethical responsibility to speak up against practices that are potentially biased, not based on research, and/or that contribute to undesirable outcomes. This will require being familiar with the laws and best practice guidelines, and acknowledging the fallibility of memory and clinical judgment—particularly for subjective diagnoses such as mild MR, ED, and LD. When evaluating practices or policies, do not just ask what works, but what works under what circumstances and for whom. We should promote critical evaluation of policies and practices, as well as promoting change where necessary. This may also include helping administrators to construct more appropriate policies and assisting teachers to create more effective practices so that the need for assessment is reduced.
Seek Professional Learning
Finally, the breadth of our professional roles and the dynamic nature of the field necessitate engaging in professional learning to deepen our understanding about emerging evidence for instruction, consultation, intervention, and assessment practices that are promising, recognizing that some are better for certain groups than others, and may or may not be applicable nor effective in one’s context. It is also essential to appreciate that one-size does not fit all; therefore, we must have a broad and ever-expanding conceptual toolbox to effectively serve our school communities.
Artiles, A. J. (1998). The dilemma of difference: Enriching the disproportionality discourse with theory and context. Journal of Special Education, 32, 32–36.
Artiles, A. J., & Trent, S. C. (1994). Overrepresentation of minority students in special education: A continuing debate. Journal of Special Education, 27, 410–437.
Cartledge, G., Kea, C., & Simmons-Reed, E. (2002). Serving culturally diverse children with serious emotional disturbance and their families. Journal of Child and Family Studies, 11, 113–126.
D’Amato, R. C., Crepeau-Hobson, F., Huang, L. V., & Geil, M. (2005). Ecological neuropsychology: An alternative to the deficit model for conceptualizing and serving students with learning disabilities. Neuropsychology Review, 15, 97–103.
Donovan, M. S., & Cross, C. T. (Eds.). (2002). Minority students in special and gifted education. Washington, DC: National Academies Press.
Hernandez, J. E., Ramanathan, A. K., Harr, J., & Socias, M. (2008). Study of the effect of an intervention to reduce the disproportionate identification in the category of emotional disturbance in Los Angeles Unified School District. Journal of Special Education Leadership, 27, 64–73.
Hosp, J. L., & Ardoin, S. P. (2008). Assessment for instructional planning. Assessment for Effective Intervention, 33, 69–77.
Klingner, J. K., Artiles, A. J., Kozleski, E., Harry, B., Zion, S., Tate, W., et al. (2005). Addressing the disproportionate representation of culturally and linguistically diverse students in special education through culturally responsive educational systems. Education Policy Analysis Archives, 13(38). Retrieved from http://epaa.asu.edu/ojs/article/view/143
Murdoch, S. (2007). IQ: A smart history of a failed idea. Hoboken, NJ: John Wiley & Sons.
Osher, D., Woodruff, D., & Sims, A. E. (2002). Schools make a difference: The overrepresentation of African American youth in special education and the juvenile justices system. In D. J. Losen and G. Orfield (Eds.), Racial inequity in special education (pp. 93–116). Cambridge, MA: Harvard Education Press.
President’s Committee on Mental Retardation. (1969). The six-hour retarded child. Washington, DC: Bureau of Education for the Handicapped, Office of Education, U.S. Department of Health, Education, and Welfare.
Henning-Stout, M. (1994). Responsive assessment: A new way of thinking about learning. San Francisco: Jossey-Bass.
Amanda L. Sullivan, PhD, is an assistant professor of school psychology at Arizona State University and a certified school psychologist.
The author would like to thank Elizabeth Rose A’Vant, Daphne R. Chandler, NCSP, D’Andrea Jacobs, and Tremaine Sayles, PsyD, for their contributions to an earlier version of this article.