 |
 |
|
 |
||||||||||
|
||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
At the University of North Carolina at Charlotte, we have been conducting research on alternate assessment and general curriculum access for the last five years. In this segment, we will briefly review our research findings on alignment and the evidence for teaching academic content to students with significant cognitive disabilities from our current review of the literature. Alignment Research: What curricular domains are present in states' alternate assessments?
We first asked, What curricular domains are present in states' alternate assessments? When we began our work, there was some confusion about whether to align alternate assessments with the general curriculum or a separate functional curriculum. Our method was to collect information on alternate assessments from as many states as possible. We obtained information from 42 states; 31 of these had enough information in their materials to be coded for patterns. Our first finding was that even prior to No Child Left Behind, most states included academic content domains. Please note that while this study is in press for 2005, it was submitted in 2002 and is now a bit dated. Alignment Research: Do the performance indicators in states' alternate assessments align with reading and mathematics standards?
Next, we asked, do the performance indicators in states' alternate assessment align with reading and math standards?We used the term "performance indicators" to refer to the many variations states used to describe what the alternate assessments targeted. Some states provided sample assessment items; others sampled teaching tasks or curricular frameworks for their standards; others used the term "extended standards" for specific performance tasks like using a daily schedule. We wanted to learn if the performance indicators used in alternate assessments were really reading and math. I called this our "Emporer's New Clothes" study based on the children's fable in which the king's court pretends to see clothes that are not visible. We wanted to be sure the items were really reading and math; that they were visibly related to academic content when viewed by math and reading researchers from general education. Our findings, the new Emporer does indeed have clothes- sometimes. These content experts found that some states' performance indicators had clear alignment with reading and math, some states reflected weak links, and others had both strong and weak links. We listed some of examples from the states' performance indicators in the slide above to illustrate the difference. Alignment Research: What type of curriculum is reflected in states' alternate assessments?
We decided to contrast the states with strong and weak alignment to ask, What type of curriculum is reflected in states' alternate assessments? To answer this question, we coded each of the indicators for the type of curricular focus reflected in both its task and context. Our findings revealed a significant difference between states the curriculum experts had identified as having strong and weak links. While many indicators from both types of states included a blend of functional and academic tasks and contexts, the states with clearly aligned indicators used many more academic ones Alignment Research: To what extent do alternate assessments align with grade level content standards?
This led to evaluating states with clear alignment to academic content in more depth. We wanted to know to what extent do alternate assessments align with grade level content standards? We asked researchers in alternate assessment to nominate states who had alternate assessments with strong alignment to general curriculum. Three states were identified and all agreed to supply copies of their alternate assessments and their states' academic content standards for this research. Two of these states used portfolio models and one used a performance assessment. We applied Norman Webb's criteria for alignment which looks at categorical concurrence, depth-of-knowledge consistency, range of knowledge correspondence, and balance of representation for each state using the state's grade level academic content standards. Our findingsconfirmed that these were states with overall strong alignment. Most of the assessment items could be directly linked to grade level academic content standards. In contrast, there was some narrowing of the breadth and depth of standards addressed in the alternate assessments. Fewer objectives were sampled; there were fewer items per standards; and less balance across objectives. The states' standards each had a normal distribution for depth of knowledge- that is most standards were geared to the middle level and fewer sampled more basic or complex levels. In contrast, the alternate assessments were negatively skewed for depth of knowledge with more items at the basic level. In a follow up conversation with the states, all emphasized the importance of sampling the entire range of depth of knowledge which all three states had done. For example, we found items that required more complex demonstrations of knowledge like synthesis and evaluation. They also emphasized the importance of stretching towards broader and deeper alignment with their state standards.
Notes
ReferencesAchieve, Inc. (2001). Measuring up—a commissioned report on education assessment for Minnesota. Washington, DC: Author. Bhola, D. S., Impara, J. C., & Buckendahl, C. W. (2003). Aligning tests with states' content standards: Methods and issues. Educational Measurement: Issues and Practice, Fall, 21-29. Browder, D. M., Ahlgrim-Delzell, L., Flowers, C., Karvonen, M., Spooner, F., & Algozzine, R. (2005). How states define alternate assessments for students with disabilities and recommendations for national policy. Journal of Disability Policy Studies, 15, 209-220. Browder, D. M., Flowers, C., Ahlgrim-Delzell, L., Karvonen, M., Spooner, F., & Algozzine, R. (2004). The alignment of alternate assessment content to academic and functional curricula. Journal of Special Education, 37, 211-224. Browder, D. M., Spooner, F., Ahlgrim-Delzell, L., Flowers, C., Algozzine, B., & Karvonen, M. (2003). A content analysis of the curricular philosophies reflected in states' alternate assessment performance indicators. Research & Practice with Persons with Severe Disabilities, 28, 165-181. Cohen, D. K. (1990). A revolution in one classroom: The case of Mrs. Oublier. Educational Evaluation and Policy Analysis, 12, 311-330. Cook, L., Eignor, D., & Cahalan, C. (2004, June). Alternate assessments: Key issues and research implications. Paper presented at the CCSSO Large-Scale Assessment Conference, Boston, MA. Council of Chief State School Officers (n.d.). Alignment Models. Retrieved April 29, 2005, from http://www.ccsso.org/Projects/alignment_analysis/models/418.cfm. Courtade-Little, G., & Browder, D.M. (2005). Aligning IEPs with academic standards for students with moderate and severe disabilities. Verona, WI: Attainment Company. English, F. W., & Steffy, B .E. (2001). Deep curriculum alignment: Creating a level playing field for all children on high-stakes tests of educational accountability. Lanham, MD: The Scarecrow Press, Inc. Fisher, D., & Frey, N. (2001). Access to the core curriculum: Critical ingredients for student success. Remedial and Special Education, 22(3), 148-157. Flowers, C., Browder, D. M., & Ahlgrim-Delzell, L. (in press). An analysis of three states' alignment between language arts and mathematics standards and alternate assessment. Exceptional Children. Glaser, R., Linn, R., & Bohrnstedt, G. (1997). Assessment in transtion: Monitoring the nation's educational progress. New York: National Academy of Education. Haertel, E. H (1999). Performance assessment and education reform. Phi Delta Kappn, 80(9), 662-666. Herman, J. L., Webb, N. & Zuniga, S. (2002). Alignment and college admissions: The match of expectations, assessments, and educator perspectives. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Kleinert, H. L., & Kearns, J. F. (2001). Alternate assessment: Measuring outcomes and supports for students with disabilities. Baltimore: P.H. Brookes. Linn, R. (2000). Assessments and accountability. Educational Researcher, 29(2), 4-16. National Research Council (1999). High stakes: Testing for tracking, promotion, and graduation. Committee on Appropriate Test Use. J. P. Heubert & R. M. Hauser (Eds.) . Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press. Nolet, V., & McLaughlin, M. J. (2000). Accessing the general curriculum . Thousand Oaks, CA: Corwin Press. Porter, A. C. (2002). Measuring the content of instruction: Uses in research and practice. Educational Researcher, 31(7), 3-14. Porter, A. C. & Smithson, J. L. (2001). Defining, developing, and using curriculum indicators (CPRE Research Report Series RR-048). University of Pennsylvania: Consortium for Policy Research in Education. Project 2061. (2002). American Association for the Advancement of Science (2000). Middle Grades Mathematics Textbooks A Benchmarks-Based Evaluation. Retrieved July 30, 2004, from http://www.project2061.org/tools/textbook/matheval/default Roach, A. T., Elliott, S. N., & Webb, N. L. (2005). Alignment of an alternate assessment with state academic standards: Evidence for the content validity of the Wisconsin alternate assessment. The Journal of Special Education, 38 (4), 218-231. Thompson, S. J., Quenemoen, R. F., Thurlow, M. L., & Ysseldyke, J. E. (2001). Alternate assessments for students with disabilities. Thousand Oaks, CA: Corwin Press. Thompson, S. J., & Thurlow, M. L. (2001). 2001 State special education outcomes: A report on state activities at the beginning of a new decade. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved September 5, 2001, from http://education.umn.edu/NCEO/OnlinePubs/onlinedefault.html Sands, D. J., Adams, L., & Stout, D. M. ( 1995). A statewide exploration of the nature and use of curriculum in special education. Exceptional Children, 62, 68-83. Spooner, F., & Browder, D. M. (2005, February). Current alignment of national science standards and performance indicators: What this means for students with severe disabilities. Invited address presented at the annual meeting of the North Carolina Association for Behavior Analysis, Wrightsville Beach, NC. Thompson, S. J., Thurlow, M., Esler, A., & Whetstone, P. J. (2001). Addressing standards and assessments on the IEP. Assessment for Effective Intervention, 26(2), 77-84. Thurlow, M. L., Elliott, J. L., & Ysseldyke, J. E. (1998). Testing students with disabilities: Practical strategies for complying with district and state requirements. Thousand Oaks, CA: Corwin Press. Webb, N. L. (1997). Research Monograph No. 6: Criteria for alignment of expectations and assessments in mathematics and science education. Washington, DC: Council of Chief State School Officers. Webb, N.L. (1999). Alignment of science and mathematics standards and assessments in four states. (NISE Research Monograph No. 18). Madison: University of Wisconsin-Madison, National Institute for Science Education. Washington, DC: Council of Chief State School Officers. Webb, N.L. (2002). Alignment study in language arts, mathematics, science, and social studies of state standards and assessments for four states. Washington, DC: Council of Chief State School Officers. Webb, N.L., Horton, M., & O'Neal, S. (2002, April). An analysis of the alignment between language arts standards and assessments for four states. Paper presented at the meeting of the American Educational Research Association, New Orleans. Wehmeyer, M. L., Sands, D. J., Knowlton, H. E., & Kozleski, E. B. (2002). Teaching students with mental retardation: Providing access to the general curriculum. Baltimore: Brookes. References and Annotated Bibliographies for Part VII: Measurement Perspectives for "Alignment"Browder, D., Ahgrim-Delzel, L., Flowers, C., Karvonen, M., Spooner, F., & Algozzine, R. (2005). How states define alternate assessments. Research and Policy in Developmental Disabilities, 15(4), 209-220. Browder, D., Flowers, C., Ahlgrim-Delzell, L. Karvonen, M., Spooner, F., & Algozzine, R. (2004). The alignment of alternate assessment content to academic and functional curricula. Journal of Special Education, 37, 211-224. Browder, D., Spooner, F., Ahlgrim-Delzell, L. Flowers, C., Karvonen, M. & Algozzine, R. (2004). A content analysis of curricular philosophies in states' alternate assessment performance indicators. Research and Practice in Severe Disabilities, 28, 165-181. Flowers, C., Browder, D., & Ahlgrim-Delzell, L (in press). An analysis of three states' alignment between language arts and mathematics standards and alternate assessments. Exceptional Children. Council of Chief State School Officers (n.d.). Alignment Models. Retrieved April 29, 2005, from http://www.ccsso.org/Projects/alignment_analysis/models/418.cfm. Porter, A. C., Smithson, J. L. (2001). Defining, Developing and using curriculum Indicators. (CPRE Research Report Series RR-048). University of Pennsylvania: Consortium for Policy Research and Education. Webb, N. L. (1997). Research Monograph No. 6: Criteria for alignment of expectations and assessments in mathematics and science education. Washington, DC: Council of Chief State School Officers. Ref. Type: Journal article Notes: Journal article Title: A content analysis of the curricular philosophies reflected in states' alternate assessment performance indicators. Authors: Browder, D., Spooner, F., Ahlgrim-Delzell, L., Flowers, C., Algozzine, & Karvonen, M. Pub. Date: 2004 Source: Research & Practice with Severe Disabilities Vol, Issue: 28, 4 Publisher: Page #: 165-181 Keywords: alternate assessment, performance indicators Abstract: Participants: Six selected states from a previous study that were identified as having strong, weak, or mixed links to reading and math were included in the study. Test Design: The purpose of the study was to examine five curricular philosophies (developmental, functional, social inclusion, self-determination, and academic) that states use in their alternate assessment guidelines for students with severe disabilities, and to evaluate how these philosophies were reflected in the performance indicators of states selected from an earlier study on alignment. The content analysis used both qualitative and quantitative procedures. Using qualitative methods in an earlier study, states were identified as having strong, weak, or mixed links to reading and math. These states' performance indicators were classified by current philosophy and analyzed using non-parametric statistical procedures in the current study. Findings: Results revealed that clear link states used predominantly academic tasks in their performance indicators for math and reading. Overall clear link states used more academic contexts than the weak link or the mixed link states. Ref. Type: Journal Notes: Journal article Title: The alignment of alternate assessment content to academic and functional curricula Authors: Browder, D., Flowers, C., Ahlgrim-Delzell, L., Karvonen, M., Spooner, F., & Algozzine, R. Pub. Date: 2004 Source: The Journal of Special Education Vol, Issue: 37, 4 Publisher: Page #: 211-223 Keywords: alternate assessment, academic curricula Abstract: Participants: 31 states participated in the study. Test Design: This study investigated the curricular focus of alternate assessments using performance indicators in math, language arts, and functional skills from 31 states. Professionals in math education, language arts and severe disabilities together with a group of stakeholders evaluated the performance indicators relative to their alignment to national standards and curricula. States that had alternate assessment performance indicators that were clearly aligned to math or language arts and those that did not were identified. The functionality of the indicators was also considered. Features of the performance indicators that exemplified alignment with general or functional curricula were identified through a series of discussions. Findings: Results indicate that while some states have created lists of indicators that are accurate representatives of math and language arts, other states even within the most experienced states have missed the mark. Overall, the findings indicate that alternate assessments have a strong focus on academic skills, but they also reflect additive curricula approach which links academic and functional skills. Ref. Type: Journal Notes: Journal article Title: How states define alternate assessments. Authors: Browder, D., Ahlgrim-Delzell, L., Flowers, C., Karvonen, M., Spooner, F., & Algozzine, R. Pub. Date: 2005 Source: Research and Policy in Developmental Disabilities, Vol, Issue: 15, 4 Publisher: Page #: 209-220 Keywords: alternate assessment, developmental disabilities Abstract: Participants: 42 states participated in this study. Test Design: After IDEA 1997 mandated inclusion of students with disabilities in schools accountability systems, states began to develop alternate assessments to focus on students' performance on state standards. The purpose of this study was to examine how alternate assessments linked to state standards and how the assessments were scored. Researchers obtained and reviewed alternate assessment materials from 42 states. The information collection period ranged from June 2001 to November 2001. This information that consisted mainly of manuals and guidelines was examined to identify the skills and knowledge being measured by alternate assessments. Findings: Findings from this study indicated that states employed a wide variety of implementation and scoring methods. Generally in most states, assessments were either linked back to state standards or standards were extended to alternate assessments. Most states measured academic domains; however, some states measured only functional skills. Regarding scoring, most states used some form of rubric to score the alternate assessments mainly measuring mastery, progress, or level of independence. Ref. Type: Report Notes: Technical Report Title: Defining, Developing and using Curriculum Indicators Authors: Porter, A. C., & Smithson, J. L. Pub. Date: 2001 Source: CPRE Reports Series PR-048 Vol, Issue: Publisher: Page #: Keywords: curriculum indicators, defining, developing Abstract: Participants: Test Design: Findings: The purpose of this report was to describe the progress made to develop valid and efficient measures of instructional content and its relationships to assessments and standards. The authors paid particular attention to mathematics and science. First they discussed the Reform Up Close Study, a Consortium for Policy Research in Education (CPRE) project. Then they highlighted the pertinent issues involved in defining and measuring curriculum indicators while taking note of how the approach has developed over the past ten years. In addition, they also provided information on using curriculum indicators in school improvement, program evaluation, and informing policy decisions. More focus was given to new methods for determining alignment among instruction, assessment, and standards. Suggestions for the next steps are also provided. Ref. Type: Report Notes: Monograph Title: Criteria for alignment of Expectations and Assessments in Mathematics and Science Education. Research monograph no 6 Authors: Webb, N. L. Pub. Date: 1997 Source: Research Monograph no. 6 Vol, Issue: Publisher: Washington, DC: Council of Chief State School Officers Page #: Keywords: Expectation, criteria, educational assessment, academic achievement, educational change, standards, Abstract: Participants: Test Design: Findings: The monograph presents criteria for judging the alignments between expectations of student achievement and assessment. Specifically, 12 criteria for judging alignment are discussed and grouped into five categories namely: 1) content focus; 2) articulation across grades and ages; 3) equity and fairness; 4) pedagogical implications; and 5) System applicability. Along with the criteria, examples and levels of agreement are also provided. An expert panel formed as a cooperative effort of the Council of Chief State School Officers and National Institute for Science Education developed the above criteria.
This publication was developed under a grant for the Office of Special Education Programs, U.S. Department of Education. Opinions expressed herein are those of the authors and do not necessarily reflect the position of the U.S. Department of Education, and such endorsements should not be inferred.
|
||||||||||||||||||||||||||||||
Information About PDF
