|
Standards and Assessment Approaches for Students With Disabilities Using a Validity Argument
Massachusetts Content Standards and Alternate Assessments With Portfolios
We confine our analysis to the content standards for grades three and four that focus on number sense (which has seven objectives) and operations (which has three objectives), both critical areas for understanding mathematics. These standards have a certain breadth and depth and, as we will see in comparison to Oregon’s standards, represent a very reasonable alignment with a number of mathematics constructs that focus on number sense and operations, fitting well with the standards from the National Council of Teachers of Mathematics (2000).
Table 4
Massachusetts Standards for Grades Three and Four
No. |
Number Sense Standards |
Essence |
| 4.N.1 |
Exhibit an understanding of the base ten number system by reading, modeling, writing, and interpreting whole numbers to at least 10,000; demonstrate an understanding of the values of the digits; compare and order the numbers. |
- Manipulate numbers at a higher level by counting, writing, grouping, sorting, comparing and ordering.
- Use a variety of numerical forms/classes.
- Recognize and use decimals.
- Understand and compare equivalent forms of decimals and fractions.
|
| 4.N.2 |
Represent, order and compare large numbers (to at least 100,000) using various forms, including expanded notation, e.g. 853 = (8 X 100) + (5 X 10) + 3. |
| 4.N.3 |
Demonstrate an understanding of fractions as parts of unit wholes, as parts of a collection, and as locations on the number line. |
| 4.N.4 |
Select, use and explain models to relate common fractions and mixed numbers (1/2, 1/3, 1/4, 1/5, 1/6, 1/8, 1/10, 1/12, and 1 1/2); find equivalent fractions, mixed numbers, and decimals; and order fractions. |
| 4.N.5 |
Identify and generate equivalent forms of common decimals and fractions less than one whole (halves, quarters, fifths, and tenths). |
| 4.N.6 |
Exhibit an understanding of the base ten number system by reading, naming, and writing decimals between 0 and 1 up to the hundredths. |
| 4.N.7 |
Recognize classes (in particular odds and evens, factors or multiples of a given number, and squares) to which a number may belong, and identify the numbers in those classes. Use these in the solution of problems. |
| No. |
Operations Standards |
Essence |
| 4.N.8 |
Select, use and explain various meanings and models of multiplication and division of whole numbers. Understand and use the inverse relationship between the two operations. |
- Understand the meaning of multiplication and division.
- Represent multiplication and division problems concretely.
- Use all operations to solve problem situations related to money.
- Understand commutative properties of addition and multiplication (order can be reversed).
|
| 4.N.9 |
Select, use and explain the commutative, associative and identity properties of operations on whole numbers in problem situations, e.g. 37 X 46 = 46 X 37, (5 X 7) X 2 = 5 X (7 X 2). |
| 4.N.10 |
Select and use appropriate operations (addition, subtraction, multiplication and division) to solve problems, including those involving money. |
To provide a functional equivalence across standards, an "essence" of the standard is distilled in Massachusetts, where standards are translated into some minimal specifications (see Table 4 for examples of standards distilled into essences or minimal specifications) that eventually are used in guiding the development of alternate assessments based on alternate achievement standards. (In contrast, Oregon’s content standards are affixed to common curricular goals across grades and are applied directly to the alternate assessments based on alternate achievement standards.) These grade-level standard "essences" are then used in Massachusetts to fully articulate the alternate assessment system to ensure alignment with it and to help structure the assessment approach (in this state, portfolios). For each grade-level standard, the state has illustrations posted on its assessment Web site (retrieved on May 30, 2005); see Table 5 for structuring activities related to the standard (left column) and documentation or end product portfolio entry (right column) that eventually is judged as proficient (or not).
Table 5
Application of Massachusetts Learning Standards and Assessment Strategies
How can all students participate in this assessment activity? |
| Addressing Learning Standard(s) as Written for This Grade Level |
Possible Assessment Strategies and Portfolio Products |
| Ricardo participates in a cooperative group activity with classmates to solve open-ended mathematical problems involving money. They make multiple purchases and compare selections, estimates, total cost and change received. |
- Ricardo’s flyer/catalog and work samples of items bought, estimations made, amount spent and change received
- Chart of Ricardo’s grades/scores on quizzes and tests related to mathematical problem solving
- One copy of a quiz/test chosen by Ricardo for his portfolio
- Journal entry in which Ricardo reflects on his work samples and performance on the quiz/test
|
| Addressing Learning Standard(s) at Lower Levels of Complexity ("Entry Points") |
Possible Assessment Strategies and Portfolio Products |
| Dominique participates by making purchases with her classmates. She selects items for purchase and indicates the amount needed by identifying the "next highest dollar" from the price given. A vertical number line provides Dominique with support so she can participate independently in this activity. |
- Dominique’s vertical number line
- Work products in which she selected her purchases and indicated the number of dollars she needs
- Dominique’s graph, created with teacher assistance, demonstrating her accuracy in identifying the "next highest dollar" for purchases of $10 or less
- Photographs of Dominique making purchases in a variety of settings (classroom, cafeteria, school store, drug store, etc.) with her number line using the "next highest dollar" method
|
| Addressing Access Skill(s) (skills embedded in academic instruction) |
Possible Assessment Strategies and Portfolio Products |
| Alice participates in this activity by assembling money envelopes paired with pictures. Alice works with a classmate who counts the money needed for each item and helps Alice place the correct amount into its corresponding envelope. Alice exchanges these envelopes when making a purchase. |
- Teacher note describing the work accomplished by Alice and her classmate
- Data collected on Alice’s ability to assemble money envelopes and exchange correct envelopes when making a purchase
- Videotape of Alice making a purchase
- Alice ’s choice of money envelopes selected for her portfolio
|
The assessment activities provide highly connected portfolio products aligned with the "essence" of the standards. The primary issue we address here is the need for making a validity claim and then collecting both procedural and empirical evidence to evaluate the claim.
Procedural evidence arises from the processes used by teachers while they assess the student:
- Was the test developed in a way that is consistent with testing standards and are the scoring procedures credible?
- What is the quality of the portfolio entries and are they formatted in a way that is understandable and accessible?
- How are work samples assembled and organized into a total portfolio?
- How well conducted are the test administration and scoring procedures?
- Do the various assessment activities of the alternate assessment represent the "essence" of the standard?
- Does the actual evidence described in the possible assessment strategy fully reflect this construct?
- Are all representations in the portfolio well displayed so they can eventually be scored?
- Does the student independently complete work that is displayed in the portfolio or is the teacher part of this process and, if so, to what extent does the teacher assist the student?
Score reporting and analyses should address questions such as how standards were established and what kinds of statistical procedures were used to analyze the outcomes. Technical documentation should be available to determine how clear and consistent the results reported to the public are.
Empirical evidence also needs to be established by investigating the dependability and credibility of work samples as reflections of the construct:
- How carefully do teachers collect evidence for the portfolios?
- Are judges who score the portfolio adequately trained?
- Is there agreement among their scores?
- Are there appropriate forms of reliability estimation?
Content coverage needs to be documented:
- What is the alignment of the portfolios with the standards?
Internal structures and item functioning need to be addressed:
- How are different dimensions like generalization, independence, and achievement scored?
- How well do work samples "hang together" to reflect a standard?
- Which work samples in the portfolio are related to each other and to what extent do those samples reflect a consistent structure?
Response processes should be considered as part of the validation evidence:
- Are there patterns in how students respond that reflect systematic variance, for example, whereby all students assessed with real money are judged successful and those solving "artificial" word story problems all fail?
Finally, evidence needs to consider relations with other variables:
- Do students with all disabilities reach proficiencies in equal proportions and what other demographics are related to outcomes?
- How is performance on the portfolio related to any other performances?
All of these are examples of empirical evidence: reliability-related evidence, content-related evidence, internal structures, response processes, and relations with other variables.
We offer this system and these questions to highlight the essential issues that all states must eventually face as they develop a fully articulated assessment system for students with disabilities. The assessment system must be sensitive to the needs of students, instructionally relevant to teachers, and related to grade-level content standards.
 Previous | Next 
|
Information About PDF