Printer Friendly Version (pdf, 132K)

Professional Development on Assessment Systems

What measurement professionals need to know

Measurement professionals working with state assessment systems have expertise in and in-depth knowledge of the Standards for Educational and Psychological Testing (American Educational Research Association et al., 1999). However, they may not be familiar with the need to apply these standards to the entire range of assessment methods available for students with disabilities because they will likely have had limited exposure to students across the full range of disabilities and, therefore, may not fully appreciate why different modes of application are needed and how to apply these different modes according to different types of disabilities . Their major need, therefore, is to understand students with disabilities and the implications of their disabilities for participation in and performance on statewide accountability assessments. Professional development might begin with specific information about special education and students who have IEPs. State measurement professionals should be able to use current terminology in referring to individuals with disabilities and should have some understanding of the variation in students' response modes and support needs. These professionals may need to learn the current categories of disabilities in IDEA 2004, including both definitions and terminology; current state categories and definitions also would be relevant. In this context, it may be useful to explain why IDEA 2004 refers to this population as "individuals with disabilities" rather than as "handicapped children" and to emphasize the importance of using language that is respectful to the persons to whom one is referring.

State measurement professionals also must understand that neither IEPs in general nor assessment participation decisions in particular are based on disability categories. Once a student is eligible for special education services, his or her needs must be considered in planning a free, appropriate public education and in selecting an assessment method. In this process, it is essential to consider instruction designed for students with disabilities, to note specific student needs, and to help organize appropriate student services.

Measurement professionals need to understand that students with disabilities may demonstrate learning in a wide variety of response modes. As they design and develop assessment methods, measurement professionals must allow for this individualization of response and participation, and they must understand the inferences that can be made from scores derived from different testing formats. They need to apply principles of universal design to provide accessibility for many students who were previously unable to participate in assessments or demonstrate their knowledge. In this process, then, assessments are "designed and developed from the beginning to allow participation of the widest possible range of students, and to result in valid inferences about performance for all students who participate in the assessment. Universally designed assessments add a dimension of fairness to the testing process" (Thompson, Johnstone, & Thurlow, 2002). Measurement experts must consider universal design, which is part of IDEA and of Title I of the Elementary and Secondary Education Act (ESEA), Title I regulations in particular. Finally, they must appreciate the importance of response variations in considering appropriate accommodations.

State measurement professionals must be familiar with current federal regulations concerning the participation of students with disabilities in statewide assessments and accountability systems. Providing them with copies of these policies, along with state guidelines, is important. Detailed knowledge of these regulations is particularly critical for measurement professionals who develop alternate assessments or state-level policies for participation in assessment.

Measurement professionals need working knowledge of the concept of access to the general curriculum and should know that this is an expectation for all students with disabilities. In this context, it may be useful to clarify that some terms used by measurement specialists can be confusing when applied to the education of students with disabilities, their access to the general curriculum, and their achievement of academic content standards. For example, in measurement circles, the term "developmental" is used to describe a type of derived or transformed score, such as developmental age or developmental quotient. Early childhood educators use "developmentally appropriate" to refer to an educational curriculum that is appropriate to students' current abilities. Among those involved in the education of students with severe disabilities, "developmental" refers to the outmoded practice of applying an early childhood curriculum throughout the student's lifespan by planning an education program based on his or her mental age¹. In the same way, "off grade level" may have different meanings for the measurement and special education communities. For measurement specialists, the term may refer to the scaling of content. To special educators, it may describe the selection of appropriate teaching materials from a lower grade level to help a student access the general education grade-level content (for example, teaching a fifth-grade curriculum using a book written at a second-grade readability level).

Several resources are available to help measurement professionals understand access to the general curriculum. Technical assistance centers that focus on general curriculum access (e.g., http://www.k8accesscenter.org) provide summary documents that define general curriculum access and provide examples. Applications of this content to students with significant cognitive disabilities are available on the National Alternate Assessment Center Web site (http://www.naacpartners.org).

Professional development for measurement specialists also may need to include specific references to the application of measurement principles in the development and validation of alternate assessment formats. Few states have created or circulated technical reports on their alternate assessments. Although specific technical issues, such as item discrimination and differential item functioning, may need to be reconsidered when applied to alternate assessment formats, proper documentation and dissemination of technical adequacy are required for these areas just as for the general assessment.

If validity arguments for each testing format used by a state are to be adequately addressed, measurement professionals in the state may need increased knowledge of approved accommodations, the state's inclusion criteria for participation in assessment methods, and standardized reporting of student performance. In particular, measurement professionals need to address construct misrepresentation and construct-irrelevant variance as they relate to assessments for students with disabilities. Further, scoring constructed-response assessments requires careful planning and analysis. Because subjective scoring may introduce bias into the judgments of student performance, scorers must be adequately trained and made aware of potential biases.

With greater knowledge of students with disabilities, test accommodations, and technical considerations, measurement professionals can make more substantive contributions to the annual review of technical data related to all the assessment approaches used within a state for students with disabilities. In this way, they can help to strengthen or increase the inferences made from those assessments. Table 1 summarizes content that might be used in professional development for measurement specialists responsible for including students with disabilities in the statewide assessment and accountability systems.

Table 1

Summary: What Measurement Professionals Need to Know about Including Special Education Students in Statewide Assessment and Accountability Systems

¹ The current focus in educating students with severe disabilities is to use chronologically age-appropriate educational activities and those that provide opportunities to participate in the learning of grade-level content.