Printer Friendly Version (pdf, 4.1M)

Designing from the Ground Floor: Alternate Assessment on Alternate Achievement Standards

Part VII: Measurement Perspectives for "Alignment"

Purpose of Part VII

The purpose of Part VII is to define "alignment" terminology from a measurement perspective, consider a variety of alignment procedures, and apply strategies for linking to grade-level content instruction.

Trainer's Note: This presentation is designed to stand alone. Therefore, you will find elements of Part I: Overview, Terminology, Theory, and Research in this presentation.

This section deals approximately equally between both the observation and cognition vertices. It may connect slightly heavier in terms of the observation vertex as it deals with measuring student performance(s) but also strongly connects with the cognition vertex by identifying just what it is that students should know and be able to do.

The Importance of Alignment

We sometimes assume that instructional systems are driven by academic content standards, which are translated into assessment, curriculum materials, instruction, and professional development. Research has shown that teachers may understand what content is wanted and believe they are teaching that content, when in fact they are not (Cohen, 1990; Porter, 2002). Improvements in student learning will depend on how well assessment, curriculum, and instruction are aligned and reinforce a common set of learning goals, and on whether instruction shifts in response to the information gained from assessments (National Research Council, 2001).

Accurate inferences about student achievement and growth over time can only be made when there is alignment between the standards (expectations) and assessments; from this perspective, alignment has both content and consequential validity implications (Bhola, Impara, & Buckendahl, 2003).

Alignment has great potential to improve the education of all students. Students with significant disabilities who receive instruction that is aligned to state content standards may have an opportunity to demonstrate learning academic content. Students who are assessed with items that are aligned to state content standards can demonstrate measurable levels of growth. Finally, students who receive instruction that is aligned to the assessment have a greater chance of demonstrating proficiency.

Standards-based educational reform aims to focus educational resources, efforts, and energy towards students' achievement. Improvements in student learning will depend on how well assessment, curriculum, and instruction are aligned and reinforce a common set of learning goals, and on whether instruction shifts in response to the information gained from assessments (National Research Council, 1999).

There is great potential for teachers to provide access to the general curriculum when they align their instruction to the content standards. Earlier modules described how teachers can best foster this access and opportunities. If instruction and assessments are aligned, the data provided from the assessments can inform teaching and support student growth.

Examining the alignment between standards and assessments provides one avenue for examining evidence for score interpretation. Evidence of content representativeness of an assessment provides test users with information about the inferences that can be made from an assessment. It would be a disservice to students and schools to make judgments about achievement of academic expectations based on assessments that were not aligned to content standards.

No Child Left Behind has mandated that states demonstrate alignment between the state academic standards and all assessments, including alternate assessments. Because alignment between the standards and assessments must be clearly described and documented, teachers need to track the links between the content, instruction, and the assessment. For example, teachers of students with significant disabilities have to identify each student's needs based on the student's grade level content standards. From these and potentially other needs, the student's IEP is written and instruction is designed. Students are then assessed based on the instruction they receive in the selected grade level standards in an alternate assessment. Assessments are then scored and student performance is then judged against alternate achievement standards. It is critical that alignment occurs between the three areas.

Standards-Based System

The ultimate goal of standards-based reform is to improve student learning and teacher instruction. For this to occur, there needs to be a high degree of alignment between the academic content standards, assessments, and ongoing classroom instruction.

Alignment Procedures

Procedures for evaluating the alignment of standards, instruction, and assessments and research into the effectiveness of these alignment procedures are still emerging, and four of the more promising alignment methods can be found at the Council of Chief State School Officers website (http://www.ccsso.org/). We will briefly present t wo of the four alignment methods today, one by Norman Webb and the second developed by Andrew Porter, John Smithson, and other researchers called Surveys of Enacted Curriculum (SEC). Webb's alignment procedure examines the alignment between standards and assessments and SEC examines the alignment among standards, assessments, and instruction.

Webb's Alignment Method

First we will start with Webb's alignment method. Norman Webb provides a systematic procedure for quantifying the degree of alignment of content standards and assessments. Qualitative expert judgments and quantified coding produce a set of statistics that examine different dimensions of alignment. Webb (1997) recommends several criteria for examining alignment between content standards and assessments, which not only examines the link between the two components, but provides information about the breadth and depth of the alignment.

Categorical Concurrence

Categorical concurrence is the consistency of categories of content in the standards and assessments. The criterion of categorical concurrence between standards and assessment is met if the same or consistent categories of content appear in both the assessment and the standards. For example, if a content standard is measurement (2 nd level down on the pyramid) in mathematics (1 st level of the pyramid) does the assessment have items that target measurement? It is possible for an assessment item to align to more than one content standard. For example, if an assessment item requires students to calculate surface area, which is aligned to the content standard of measurement, to successfully answer the question the student needs to be able to multiply numbers, which is aligned to the content standard of operations. In this case the item is aligned to both content standards.

To produce an acceptable level of reliability for assessment scores, Webb recommends at least six items per content standard. In other words, there should be at least six assessment items related to the topic of measurement. Most states have multiple content standards or topics that are defined in their academic standards. If a state included five content standards under mathematics for 3 rd graders (e.g., measurement, operations, etc.), there should be at least six items that align to each content standard. The more content standards expected by educational agencies, the more assessment items will be needed to align to those standards.

Range of Knowledge

While categorical concurrence is the most obvious alignment criteria, additional alignment dimensions are needed to fully capture the complex knowledge and skills that are often emphasized in academic standards. For example, all the assessment items could be aligned to only a few of the many academic content standards. Examining the range of standards an assessment covers and the balance of assessment items across the standards provides additional evidence about how well the assessment is capturing the breadth of the standards.

Range-of-knowledge correspondence criterion examines the alignment of assessment items to the multiple objectives (3 rd level of the pyramid) within the content standards. Range-of-knowledge correspondence is used to judge whether a comparable span of knowledge expected of students by a standard is the same as, or corresponds to, the span of knowledge that students need in order to correctly answer assessment items. The range-of-knowledge numeric value is the percentage of content standards (2 nd level of the pyramid) with at least 50% of the objectives (3 rd level of the pyramid) having one or more hits. For example, if there are five objectives (e.g., length, area, volume, telling time, and mass) included in the content standard of measurement, a minimum expectation is at least one assessment item is related to at least three of the objectives.

Previous  |  Next