Overheads for Unit 8--Chapter 11 (Performance-Based Assessment)

1. Alternative assessment—connotes what it is not (not paper and pencil)
2. Authentic assessment—connotes realism and practical application
3. Performance-based assessment—more useful, according to your text, because:
• More descriptive
• Not pretentious
• Realism is a matter of degree anyway

1. Poses a task to be performed (conduct and report an experiment, sing a song, recite a poem, repair an auto, etc.)
2. Many possible modes of response possible, but essays common
3. Same basic continuum as essay questions in terms of freedom of response, with restricted-response tasks at one end and extended-response tasks at the other.
4. No single correct answer
5. Expert observers must judge quality of performance or product

1. High freedom of response, therefore:
1. Can measure complex, important learning outcomes that other forms cannot
2. Not efficient for measuring factual knowledge
3. Scoring is time-consuming
4. Scoring is unreliable and prone to bias unless great care is taken
2. Applicable to many subjects
3. Can be good instructional activities
4. Sample little of the content domain unless cumulated over a long period

1. Essay is the more familiar and established form of non-objective assessment
2. Performance-based assessment became the hot new rage in the 1990’s
3. Performance-based assessments emphasize doing (process) as well as knowing (product)
4. Performance assessments include broader/longer activity than normal essay (planning, gathering information, doing drafts, getting critiques, etc.)
5. Unlike essay questions, performance-based assessments do not require a written product, so can assess more outcomes (speaking skills, auto repair, etc.)
6. Performance-based assessments of process have to be rated on the spot

• They parallel essay questions in getting at highly complex performances (analysis, synthesis, evaluation)
• Can go beyond essay questions to assess non-writing performances and products
• As with essays, must carefully specify learning outcomes and construct tasks that actually call them forth

• They parallel the two forms of essay questions
1. Restricted-Response Tasks
   1. Intended performances more narrowly defined than on extended-response tasks
   2. (Sometimes) question may begin like a multiple-choice or short-answer stem, but then asks for an explanation, justification, etc.
   3. (Sometimes) may have introductory material like an interpretive exercise, but then asks for an explanation of the answer, not just the answer itself
   
   
2. Extended-Response Tasks
1. Activities for single assessment may be multiple and varied (gather data or information, analyze it, and write a report)
2. Activities may extend over a period of time (a series of drafts and revisions)
3. Products from different students may be different in focus (different forms of music, different research topics, etc.)

Focus: Process vs. Product--Note: More here than in your textbook

• One or both may be assessed, depending on the learning outcome and stage of instruction
1. Good procedure is emphasized when:
1. There is no product
2. The procedure is orderly and directly observable
3. Correct procedure is crucial to later success
4. Analysis of procedural steps can aid in improving product
5. Learning is at an early stage


2. Good product is emphasized when:
1. Different procedures result in an equally good product
2. Procedure is not available for observation
3. Procedural steps have been mastered
4. Product has qualities that can be identified and judged

Degrees of authenticity--Note: More here than in your textbook

• Realism of tasks can range widely
• But all are performed under controlled conditions
• From least realistic
  1. Paper-and-pencil exercise (plan a garden on paper)
  2. Observe and identify real objects and processes (tools or product flaws)
  3. Perform an isolated procedure (adjust a microscope, weld a joint)
  4. Simulated performance of part or whole real-world activity (mock interview, flight simulator)
  5. Work sample (drive a car, repair an engine, give a concert)
• To most realistic

 

OH 5
Performance-Based Assessments: Advantages and Limitations

Advantages

1. Can assess complex learning outcomes not measured by other means
2. Can assess process as well as product
3. Can clearly communicate instructional goals that relate to real-world skills
4. Can constitute good instruction, not just assessment
5. Implement new "constructivist" approaches to learning and self-evaluation
6. Engaging (more active and realistic)

Limitations

1. Scoring often unreliable
2. Time-consuming
3. May be costly (if authentic)
4. Provide only narrow sampling of learning outcomes

 

OH 6
Performance-Based Assessments: Suggestions for Constructing Tasks

Suggestions

1. Focus on learning outcomes that require performance-based assessments
2. Make sure that skills called forth apply to relevant content
3. Minimize dependence on irrelevant skills (that is, irrelevant difficulty)
4. Provide necessary scaffolding to understand task and expectations
5. Give task directions that make students’ task clear (providing them freedom is no excuse for your vagueness!)
6. Clearly communicate performance expectations with scoring rubrics because it:
1. clarifies the task
2. provides guidance on proper focus in responding
3. conveys learning priorities

 

OH 7
Performance-Based Assessments: The Performance Criteria

Performance Criteria are Absolutely Crucial

• Some experts say that clear and appropriate performance criteria are the best way to assure valid performance-based assessment
• Like essays, performance-based assessments require scoring rubrics (analytical or holistic)
• Unlike essays, they may also require rating scales to assess live performances
• Performance criteria should be created before assessment is given

 

OH 8
Performance-Based Assessments: Types of Rating Scales

Checklists

1. A list of key qualities of a product or process
2. Rater checks each as present or absent, correct or not, completed or not, etc.

Numerical Rating Scales

1. A series of numbers (e.g., 1-5) used to rate some characteristic (thesis, explanation, etc.) by quality (poor to excellent)
2. Useful when number of rating levels is limited (3-7)
3. Useful when rating levels are clearly defined and agreed-upon (usually not)
4. Meaning clearest when rating levels are "behaviorally anchored" (have descriptions for each level)

Graphic Rating Scales

1. Horizontal line with rating levels (never, sometimes,…; inappropriate, somewhat appropriate, ….) ranged across it
2. Rater can check anywhere along the line
3. Most useful when rating categories are limited and "behaviorally anchored" (have descriptions for each level)

 

OH 9
Performance-Based Assessments: Common Rating Errors

"Personal Bias Errors"—Didn’t use the whole scale

1. Generosity error (***)
2. Central tendency error (**)
3. Severity error (*)
Problem?
• Scores may reflect rater as much as student
• Little variance in the scores

"Halo Effect"—A general impression colors all achievement ratings for a student

• Obscures student’s strengths and weaknesses
• Is a form of prejudice

"Logical Error"—Preconception that two characteristics (e.g., good behavior and high intelligence) do--or don't--go together—

• Won’t get separate, valid ratings of different strengths and weaknesses

 

OH 10
Performance-Based Assessments: Principles of Effective Rating
• Can reduce rating errors if:
1. Characteristics to be rated (in your rubrics) represent specified learning outcomes
2. Each rating scale describes the level of learning desired for an outcome
3. Each characteristic is directly observable
4. Characteristics and points on rating scale are clearly defined
5. Type of scoring rubric chosen (analytical vs. holistic) is the most appropriate for task and purpose
6. Use 3-7 rating positions
7. Rate all students on one task before rating the next task
8. When possible, avoid knowing student’s name
9. When consequences are important, use several (diverse) raters and average their scores
• Can instruct and motivate students if they help design performance criteria and use them to rate their own progress