Professional Development on Assessment Systems
Large-scale assessment systems need to be technically adequate so that the public has confidence in the accountability decisions being made from them. Technical adequacy is established when the process for developing and implementing state assessments is explicit and well documented, and the state provides evidence that its assessments are reliable and support the decisions that are being made (i.e., validates the claims or inferences). To develop and implement assessment systems with this kind of technical adequacy, states must invest considerable resources in professional development. The investment should focus on four groups of professionals: (a) measurement experts, who need to know more about students with disabilities and the assessments that are appropriate for them; (b) special education professionals, who must become more proficient in understanding measurement principles in general and their applicability to assessments designed for students with disabilities; (c) education leaders and administrators, including principals, who oversee the participation of all students in large-scale assessments; and (d) individuals likely to serve on Individualized Education Program (IEP) teams, which are responsible for recommending a particular assessment method for each individual student with disabilities. This paper provides a brief overview of some professional development principles, defines what each constituent group needs to know, and lists topics and resources for each group.
Professional Development Principles
Context, process, and content are the three key elements in designing effective professional development (National Staff Development Council, 2001). The context for professional development must be conducive to learning, which may be achieved through the creation of learning communities under the guidance of effective leaders who can appropriately deploy critical resources. The process of training should focus on data use and learning outcomes, including an evaluation of training effectiveness. The content of training should be scientifically based, current, and responsive to the needs of stakeholders (i.e., test coordinators, teachers, related service providers, and administrators). The National Staff Development Council (2001) provides standards for each of these elements to guide staff development personnel in designing professional development experiences. As an example of a state that explicitly addresses these key elements, the Maryland Department of Education (2001), in its guide on professional development, notes that professional development is most effective when it is designed to take place in vibrant professional learning communities (context), be data-driven, utilizing rigorous analysis of data (process), and lead to knowledge, skills, and dispositions that apply research to decision making (content).
Effective professional development also must be a continuing process that succeeds in creating lasting changes in behavior and practice (Jones & Lowe, 1990). Effective professional development cannot be a one-time event in which information is presented with little or no follow-through. To have any chance of its effects being sustained over time, professional development must be continuous and results-oriented. Ideally, professional development takes advantage of both external and internal expertise and actively involves the educators who sign up for the training. Further, professional development must occur frequently enough to provide timely and accurate information. To be responsive to changes in policy and new research on assessing students with disabilities, states must develop systems for ongoing delivery of information. One indicator of the effectiveness of this type of system is whether professional practice becomes more consistent with new policy and research over time.
Professional development needs to be aimed at improving the technical adequacy of assessment practices for students with disabilities, including the technical adequacy of the procedures for implementation of the assessment as well as the technical adequacy of the outcomes for decision making. For example, teachers need professional development aimed at ensuring that they follow proper procedures for gathering student work samples for alternate assessments, such as portfolio assessments or performance tasks, and that they carefully score the portfolios or performance tasks to ensure the dependability and credibility of the work samples for making accountability decisions. However, for at least two reasons, the training may not necessarily lead directly to immediate improvements in student learning (Sparks & Hirsh, 2000). First, student performance data obtained from an improved assessment system may not be comparable to previously reported data. Second, professional development on participation guidelines for students with disabilities may not, in itself, be expected to improve student performance. Instead, educators need information on how to improve the quality of curriculum and instruction and how to use research-based procedures like understanding key measurement, assessment, and inclusion principles (Elliott, Braden & White, 2001) and applying progress monitoring to improve student outcomes (visit http://www.studentprogress.org for more information about progress monitoring). However, as school personnel become more informed about the appropriate assessment of what students are taught and as they use technically adequate approaches and methodologies, the impact on achievement may be quite noticeable.
Therefore, professional development must address curricular and instructional components as well as focus on results and outcomes. Only then will it be possible to provide a complete validity argument that includes a claim (or inference) supported by both reliability and validity evidence focusing on procedural and statistical components of the entire process. That is, professional development is not only about the technical adequacy of the outcomes (reaching grade-level content standards) but also about providing students systematic access to instruction focused on the standards and access to high-quality implementation of assessments aligned with those standards. Claims of performance and proficiency are much stronger (validated) when teaching and learning are systematically related.
Content for Professional Development
The following sections assume that the design of professional development incorporates appropriate consideration of context and process. This section focuses specifically on content.
Primary and additional resources for developing the content of professional development
Two primary resources should form the basis of professional development on the participation of students with disabilities in large-scale assessments. The first is the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999). The second set of resources is the federal regulations that specify this participation, such as the Individuals with Disabilities Education Improvement Act of 2004 (IDEA) (PL 108-446), the No Child Left Behind Act of 2002 (NCLB), the regulations on alternate achievement standards (Federal Register, Dec. 9, 2003), and the most current policy statements on the U.S. Department of Education's Web site, which recognize a need for modified achievement standards (http://www.ed.gov/policy/elsec/guid/raising/alt-assess.html). States also may want to include their state guidelines for the participation of students with disabilities in state assessments.
Technical assistance centers offer additional resources that may be useful to professional development planners. For example, the IEP team may benefit from the information on linking assessments to academic content standards for students with significant cognitive disabilities—available through the National Alternate Assessment Center (http://www.naacpartners.org). The National Center on Educational Outcomes offers numerous reports on the participation of students with disabilities in large-scale assessments (http://education.umn.edu/nceo). The Access Center provides direct assistance, networking, and Web-based resources to assist states in building the capacity of all students to access the general curriculum (http://www.k8accesscenter.org). The National Center on Accessing the General Curriculum provides professional development on accessing the general curriculum and universal design (http://www.cast.org/ncac). The Disabilities Studies and Services Center has a complete list of technical assistance centers at http://www.federalresourcecenter.org/frc/oseptad.htm. The online course, Assessing One and All: Educational Accountability for Students With Disabilities, on the Council for Exceptional Children Web site (http://www.cec.sped.org/pd/webcourses/cec112.html) uses a case-focused approach to understanding key measurement, assessment, and inclusion practices. The site also links to every state's testing guidelines and Adequate Yearly Progress (AYP) reports.