Overheads for Unit 7--Chapter 10 (Essay Questions)


OH 1
Essay Questions: Forms

They represent a continuum in how much freedom of response is allowed, ranging from restricted-response essays on one end to extended-response essays on the other.

  1. Restricted-response essay
    1. limits content and response to be given
    2. can limit via how narrowly question is phrased (e.g., as specific as a short-answer question)
    3. can limit via scope of the problem posed (e.g., with introduction like that of an interpretive exercise)
    4. therefore, can approach the objectivity of short-answer and interpretive exercises
  2. Extended-response essay
    1. great freedom so that allows problem formulation, organization, originality
    2. therefore, shares similar scoring difficulties with performance-based tasks


OH 2
Essay Questions: Uses

Represent a continuum in complexity and breadth of learning outcomes assessed, with interpretive exercises on the left end, restricted-response essays in the middle, and extended-response essays at the right end.

  1. Restricted-response essays
    1. For learning outcomes not readily assessed objectively
    2. Compared to extended-response questions, they target narrower learning outcomes, such as more specific mental processes (e.g., draws valid conclusions)
  2. Extended-response essays
    1. For learning outcomes not readily assessed objectively or with restricted response essays
    2. Compared to restricted-response questions, they assess broader learning outcomes, such as integrating a set of mental processes (e.g., integrates evidence to evaluate a scientific theory)
    3. Compared to interpretive exercises, both kinds of essays can assess more complex learning outcomes
  3. See Table 10.1 on page 240


OH 3
Essay Questions: Advantages and Limitations


  1. Measure complex learning outcomes not measured by other means
    1. Restricted-response essays: (i) require students to supply, not just identify, the answer and (ii) can target specific mental skills
    2. Extended-response essays: emphasize integration and application of high-level skills
  2. Can measure writing skills in addition to (or instead of) knowledge and understanding
  3. Easy to construct—but only if you don’t care what you actually measure and how reliably you do so!
  4. Contribute to student learning, directly and indirectly


  1. Unreliability of scoring (unless clear learning outcomes, good scoring rubrics, practice in scoring)
  2. Time-consuming to score—especially if follow guidelines. Can be impossible if conscientious in scoring, give good feedback, and have many students
  3. Limited sampling of content domain


OH 4
Essay Questions: Suggestions for Writing


Suggestions for writing essay questions

  1. Restrict use to learning outcomes that cannot be measured well by objective means (e.g., organization, originality)
  2. Write questions that can call forth the intended mental processes
    1. Easiest to do with restricted-response
    2. See sample stems on pp. 243-244
    3. For extended-response items, helps to state evaluation criteria in the question
    4. Make sure they do not target what has not been taught
  3. Phrase the question so that student’s task is clear and comparable for all
    1. Easiest with restricted response
    2. For extended-response, don’t define the task so tightly that its purpose is spoiled
    3. Rather, give explicit instructions on type of answer desired (e.g., "Your answer should be confined to 100-150 words. It will be evaluated in terms of the appropriateness of the facts and examples presented and the skill with which it is written.")
  4. Indicate approximate time limit for each question
    1. Give plenty of time (should be a power test not a speed test)
    2. Do not create overconcern about time
  5. Avoid optional questions
    1. Giving choices means students taking different tests
    2. They will not study the entire domain
  6. Review checklist on p. 248


OH 5
Essay Questions: Scoring Criteria

The Scoring Problem

Tackle it early--before you give the test

  1. Carefully specify your scoring criteria before you finalize the exam
    1. May cause you to rethink or modify the question and its accompanying performance criteria
    2. That, in turn, enhances likelihood of calling forth the intended responses
  2. Do an initial review of answers to a question to find exemplars or anchors for your scoring levels
  3. Make sure you can describe the kinds of performance (e.g., "lists two of the four key points") that qualify for each scoring level ("satisfactory," 2 points, etc.)



OH 6
Essay Questions: Scoring Criteria

You need rubrics!

Rubrics for restricted-response questions

  1. Write exemplar answer(s)
  2. Decide how to give points for each part expected for a full answer
  3. Decide the level of explanation necessary for full vs. partial credit

Analytical rubrics for extended-response questions

  1. Specify the separate characteristics or dimensions you want to score (focus, elaboration, mechanics, etc. for an expository essay)
  2. Assign a series of levels to each characteristic (1-7, poor to excellent, etc.).
  3. Summarize the performance corresponding to each level ("main idea present but may not maintain consistent focus" for "adequate achievement" or 4 points for "focus"—example from table on p. 251)
  4. Result is a matrix against which to judge the elements of each essay
  5. Good for giving feedback to students
  6. See websites for examples (e.g., www.nwrel.org/eval/toolkit/traits/index.html )

Holistic rubrics for extended-response questions

  1. Provides single overall score, no separate dimensions
  2. Decide how many levels.
  3. Summarize the performance corresponding to each level
  4. Easier to construct and apply than analytical rubrics
  5. May correspond better to grading needs
  6. But provides less feedback to students about strengths and weaknesses


OH 7
Essay Questions: Summary of Suggestions for Scoring


  1. Ease your burden by writing good questions in the first place (OH 4)
  2. Prepare outlines of expected answers in advance
  3. Use most appropriate scoring rubric, which depends on purpose
  4. Decide in advance how to handle factors irrelevant to learning outcomes
  5. Evaluate all responses to one question at a time. Reorder the papers before grading the next question.
  6. When possible, evaluate without seeing students’ names
  7. Watch out for bluffing
  8. If important decisions rest on the results, use several raters