|
Designing from the Ground Floor: Alternate Assessment on Alternate Achievement Standards
Part I: Overview, Terminology, Theory, and Research
Purpose of Part I
The purpose of Part I is to articulate principles of high quality assessment design, define a common terminology, examine theoretical principles upon which sound assessment systems can be built, and apply them to alternate assessments for students with the most significant cognitive disabilities.
Outcomes for Part I: Overview, Terminology, Theory, and Research
- Articulate policy guidance for alternate assessments on alternate achievement standards
- Define assessment terminology
- Articulate a theory of assessment design
- Define who the students are
- Define a theory of learning
- Define what we know about teaching academic content to students with the most significant cognitive disabilities
|
Alternate Assessment - Alternate Achievement Standards Development Site Map
- Articulate policy guidance
- Define assessment effective practice
- Define population to be assessed
- Define a theory of learning for assessed population
- Review and articulate academic content standards
- Use tools to evaluate content
- Produce a content linking chart
- Consider alignment procedures
|
Assessment Effective Practice
The Purpose of All Educational Assessments
- to assist learning or student acquisition of skills and concepts
- to measure individual achievement
- to evaluate programs
(Pellegrino, Chudowsky, Glaser, 2001) |
Assessment By Nature Imprecise
- "Assessment result is an estimate of achievement based on samples of knowledge and performance from the much larger universe of what a person knows and can do" (Pellegrino, Chudowsky, Glaser, 2001, p. 36)
- "Assessment is a process of reasoning from evidence… Using less than direct methods to make judgments about what students know" (Pellegrino, Chudowsky, Glaser, 2001, p. 36)
|
As Pellegrino, Chudowski, and Glaser (2001) remind us, educational assessment for all students may have the following purposes:
- assist learning or student acquisition of skills and concepts;
- measure individual achievement; and/or
- evaluate educational programs.
The purpose, however, of the No Child Left Behind Act of 2002 (NCLB) is the third bullet: the evaluation of educational programs. Unfortunately, when an assessment tries to accomplish multiple purposes, it generally doesn't provide high quality information for all three.
In addition, we want participants to remember that assessment by nature is imprecise, as it is at best an estimate of achievement based on samples of knowledge and performance and is a process of reasoning from evidence, using less than direct measures.
How Students with Disabilities Participate in Assessment
| |
General |
AA-GLAS |
AA-AAS |
| Content Standards taught and assessed (access and alignment targets) |
Grade-level |
Grade-level |
Grade-level linkage to content standards |
| Achievement Standards |
Grade-level |
Grade-level |
Alternate level |
| Participating Students |
Most students, including those with disabilities (with or w/o accommodations) |
Students with disabilities who need alternate way(s) to show what they know |
Students with the most significant cognitive disabilities |
|
Assessment Fairness/Accessibility
- Items/tasks provide an equal opportunity for all students to fully demonstrate knowledge and skills
- Assessments are administered fairly
- Results are reported fairly
- Results are interpreted fairly
(Peer Review Guidance, April 2004, p. 34) |
Participation & Accessibility
Students with disabilities participate in assessment and accountability systems in three ways. Most students with disabilities participate in the general assessment with or without accommodations that are aligned to grade-level content and achievement standards. Some students with disabilities may participate in assessment through an alternate assessment that is also aligned to grade-level content and achievement standards. Finally, a few students with the most significant cognitive disabilities will participate in an alternate assessment that is linked to the grade-level content standards and has different definitions of proficiency (NAAC, 2004).
We also want to make sure that ALL assessments adhere to the fairness/accessibility standards by providing opportunities to demonstrate knowledge and skills, administer assessments fairly, and to ensure results are reported and interpreted fairly.
Universal Design for Learning
- General assessments that are valid and accessible for the widest array of possible users:
- Reduce the need for accommodations
- Reduce the need for multiple alternate assessments
(CAST, 2002)
|
Universal Design for Learning: Avoid Retrofitting
- Design assessments from the start based on the Principles of Universal Design for Learning
- As with any retrofitted solutions, accommodations in assessment can result in:
- Limitations in efficacy
- Compromises to validity
|
Universal Design for Learning for AA-AAS
- Multiple means of expression
- Students must be able to show what they know and can do
- Multiple means of representation
- Students must be able to access the content of the assessment
- Multiple means of engagement
- Students may need more time, meaningful activities, and contextual orientation
(CAST, 2002)
|
Universal Design for Learning: Application to Assessment
- By considering student diversity during item construction, we should be able to minimize assumptions about student abilities which might interfere with the measurement of intended constructs.
|
In addition, the principles of Universal Design for Learning (UDL) also apply to alternate assessments of alternate achievement standards in that general assessments are valid and accessible for the widest array of possible users. Adherence to these principles could both reduce the need for accommodations and reduce the need for multiple alternate assessments. Universal Design as applied to alternate assessment means that consideration should be given to multiple means of expression, multiple means of representation, and multiple means of engagement.
Just as in architecture, when the design from the beginning contains requirements to meet the needs of all users, thoughtful, functional, elegant design is the usual result. However, when forced to retrofit (make changes after completion of the design), the product is often less efficient, less effective, and frequently not to standard. In assessment, retrofitting solutions to accommodate students with disabilities may result in assessments that are no longer technically adequate. The validity and reliability of the measures may be compromised in retrofitting alterations. The end result, we may not be measuring what is needed, the standards, or student knowledge.
In adopting the principles of UDL when building assessments, the National Alternate Assessment Center (NAAC) will be considering student diversity from the start. In this way, those issues that interfere with measuring the intended constructs will be minimized.
UDL Principles:
- Provide alternative formats for presenting information (multiple or transformable accessible media). Recognition
- Provide alternative means for action and expression (write, draw, speak, switch, graphic organizer, etc.). Strategic
- Provide alternative means for engagement (background knowledge, options, challenge, and support). Affective
Checkpoint
Checkpoint
- In what way is your state documenting the accessibility of their assessments?
|
Notes
Defining Measurement Terminology
Defining Terms
- Academic Content Standards
- Academic Achievement Standards
- Alignment
- Alternate Academic Achievement Standards
- Appropriate Challenge
- Technical Quality
- Universal Design
|
Academic Content Standards
- Define what students are expected to know and be able to do
- Contain coherent and rigorous content
- Encourage teaching of higher order skills
- Must be grade-specific or may cover more than one grade if grade-level content expectations are provided for each of grades 3-8.
(Peer Review Guidance, April 2004, p. 2) |
Academic Content Standards: Examples
- Detail facts
- Sequence events
- Use context cues
- Identify the purpose for a reading activity
- Communicate prior knowledge regarding a topic
- Ask questions about a literacy topic
- Identify similarities and differences across texts
- Categorize connections across texts
- List details about a topic
- Organize information while reading
|
Seven key terms are discussed: academic content standards, academic achievement standards, alignment, alternate academic achievement standards, appropriate challenge, technical adequacy, and universal design. These terms are often confused and it is important to clarify the differences. Academic content standards define what students should know and be able to do and are often grade or grade/band specific for grades 3-8. An example of a content standard from mathematics is: Students will solve equations. Examples of content standards from language arts are provided. The purpose of this workshop is to assist states in defining and linking their content standards in reading and mathematics for alternate assessments on alternate achievement standards for students with the most significant cognitive disabilities to appropriate grade-level content.
Academic Content Standards: Examples
- Detail facts
- Sequence events
- Use context cues
- Identify the purpose for a reading activity
- Communicate prior knowledge regarding a topic
- Ask questions about a literacy topic
- Identify similarities and differences across texts
- Categorize connections across texts
- List details about a topic
- Organize information while reading
|
Academic Achievement Standards
- Answer the question "How good is good enough?"
- Must be aligned with grade-level academic content standards
- Description of achievement levels (e.g., basic, proficient, advanced)
- Description of rationale and procedure used to determine levels (standard setting)
- Academic Achievement Standards have:
- Performance Levels – labels of achievement
- Performance Descriptors – describe each level of performance
- Exemplars – samples of student work at each performance level
- Cut Scores – scores that separate the performance levels
|
Example of Proficience Description of 7th Grade Reading
PROFICIENT
- Student demonstrates overall knowledge of the text, including some inferential as well as literal information.
- Explains the literal and some inferential meaning of a passage taken from text appropriate for middle level students.
- Effectively uses word attacks skills such as applying meanings of common prefixes and suffixes, knowledge of synonyms, antonyms, and homonyms, and multiple word meanings to aid in comprehending text.
|
Academic achievement standards are summary descriptions of how well a student should demonstrate proficiency in a content domain and is often described in at least three levels (e.g., Basic, Proficient, or Advanced). Alternate achievement standards also include descriptors of what student work reflecting the achievement looks like at basic, proficient, or advanced levels.
Alignment of Content and Achievement Standards
- Academic Achievement Standards must be aligned with the Academic Content Standards in that they:
- Capture the range of content
- Measure content and process
- Degree and pattern of emphasis
- Reflect the full range of cognitive complexity
- Representative of achievement levels as defined by the challenging, coherent, rigorous content standards.
(Peer Review Guidance, April 2004, pp. 14, 41)
- A-AAS linkage to Content and Achievement should:
- Represent similar achievement levels
- (basic, proficient, advanced)
- Represent a similar pattern of emphasis
- Represent both content and process
- Represent an increasing range of cognitive complexity
|
General Assessment versus Alternate Assessment on Alternate Achievement Standards
- For students participating in alternate assessment on alternate achievement standards, that while required to link to grade-level content standards the alternate assessment will not be required to meet the same grade-level achievement standards (i.e., breadth, depth, and complexity).
|
Alternate Academic Achievement Standards
- Must be linked to grade-level content standards
- Promote access to the general curriculum
- Reflect professional judgment of the highest learning standards possible
- Defined by a documented and validated standard setting process
- Grade-level content may be reduced in complexity
- For each grade level, one or more alternate achievement standards may be defined
- Should be defined in a way that supports individual growth because of their linkage to different content across grades
|
Appropriate Challenge
- Rely on the judgment of experienced special educators [and general educators], administrators, higher education representatives, and parents of students with disabilities.
|
Alternate achievement standards must be linked to grade-level content in order to promote access to the general curriculum as required by IDEA 97. The key is to achieve an appropriate level of challenge as judged by experienced professionals and stakeholders who understand the learning characteristics and theory of learning around the population of learners with the most significant cognitive disabilities. The achievement standard must be defined through a documented, validated standard setting process. This may result in grade-level content that is reduced in complexity, depth, and breadth. There may be one or more alternate achievement standards. Alternate achievement standards should be linked and defined in such a way that supports individual growth across grade-levels. This workshop does NOT address the development of alternate achievement standards, except for developing summary descriptions of the selected content. It is important to remember for students participating in alternate assessment on alternate achievement standards, that while required to link to grade-level content standards the alternate assessment on alternate achievement standards will not be required to meet the same grade-level achievement standards in regard to breadth, depth, and complexity.
Alignment
Alignment in the measurement world commonly refers to the extent to which the academic content standards are aligned to academic achievement standards in the following five characteristics: 1) range of content, 2) measurement of content and process, 3) the degree and pattern of emphasis, 4) the range of cognitive complexity and 5) representative achievement levels. Alternate assessments on alternate achievement standards for students with the most significant cognitive disabilities must be linked to content and achievement standards in that the same achievement levels must be represented, the pattern of emphasis at each grade is similar, measurement includes both content and process, and represent an increasing range of complexity. The Peer Review Guidance (USDOE, April 28, 2004) suggests that an appropriate level of challenge can be determined by relying on the judgment of a diverse stakeholder group that includes special educators, administrators, higher education representatives, and families of students with disabilities. Effective practice would emphasize the importance of including general education and content specialists in the work group.
Technical Quality
- Content validity
- Relationship of assessment to other variables
- Consistency of student responses
- Internal structure
- Statistical techniques used to verify reliability and validity
- Reliability
(Peer Review Guidance, April 2004, pp. 32, 33)
|
Technical quality encompasses at least five elements: content validity, the relationship of the assessment to other variables, consistency of student response, internal structure, and reliability. The first step in defining technical quality of alternate assessments on alternate achievement standards is to define content validity. The purpose of this workshop is to assist states in determining the appropriate academic content for alternate assessments on alternate achievement standards for students with the most significant cognitive disabilities.
Purpose of this workshop
- Is:
- To assist states in designing alternate assessments based on alternate achievement standards for students with the most significant cognitive disabilities beginning with content.
|
Check Point
Checkpoint
- What terms were familiar to you?
- What terms were new?
- What terms need further definition or discussion?
|
Notes
|
Information About PDF
