Recommended Courseware Evaluation Criteria
Print this to rate software on a scale from 1 to 5
  1. Required Instructional Design and Pedagogy: Does it teach?
This covers a wide range of possible problems, from little or no interactivity to insufficient examples for concept development. One program had no graphics at all, even though it was a mathematics package intended for very young children. Learners at early stages of development are known to need concrete examples rather than text only. One particularly blatant error of this type was in a courseware package intended to teach young children about how the human body works. It depicted the human heart as a square box. Another, a math program, displayed a number of objects based on what the student answered, but never bothered to change the number of objects if it was a wrong answer. Thus, the student could be seeing the corrected numeral but the wrong number of objects. Programs must be sensitive to student's feelings, even if comments are intended humorously. One program based on a well-known cartoon cat with an acerbic personality belittled the student's name, saying "What kind of name is that for a worthy opponent?" It also commented on the student's "lack of mental ability" when a wrong answer was supplied. Although this was in keeping with the cat's persona, it was still inappropriate. Although this may apply to any use of language in any program, it is particularly applicable to tutorials, which may require many explanations. For example, one tutorial for second-grade math skills had explanations at a fourth-grade reading level. This would probably not be an appropriate expectation for students who have trouble with this level of math. Pictures and animation are considered motivational to students, but this is not always true. For example. animated feedback may be charming the first ten times the students see it, but may achieve just the opposite effect after that. Also, some courseware attracts students' attention by flashing text or objects on the screen. This can be distracting when one is trying to focus on other screen text. Early courseware used a device called "scrolling" which had text moving up the screen as the student tried to read it, but this was quickly identified as a distracting mechanism and is rarely seen now.
  1. Required for Content: Is it correct?
Even though a program may be on a nonlanguage topic, it should reflect accurate language since students learn more than just the intended skills from instructional materials. One early release on punctuation skills misspelled the word "punctuation" three different ways in the program! Many people do not associate errors such as these with courseware material; they seem to trust content presented on a computer, as if the computer would correct the text itself if it spotted a problem! Content inaccuracies have been observed in a number of packages. For example, one program referred to blood as a "red substance," which, of course, is not always true. Instructional materials in social studies should be carefully screened for inaccurate reflections of country names, which are changing rapidly. Look for diversity in names and examples used. Are they all for "Dick and Jane" and are they always in the suburbs? Also review examples for gender stereotypes. Are all doctors men? Are all homemakers women? Does courseware exhibit a sensitive treatment of moral and/or social issues? For example, do games and simulations avoid unnecessary violence?
  1. Required for User Flexibility: Is it "user friendly"?
Depending on the purpose of the program, the students should normally be able to go from screen to screen and read each screen at a desired rate. They should also have exit options available at any time. Since courseware may be used in classrooms, the teacher should have the ability to make the courseware quiet so it will not disturb others.
  1. Required Technical Soundness: Does it work correctly?
A common problem in early courseware, problems of this kind are not seen very often now. Again, this was a more common problem in early courseware. Programs should be designed to expect any possible answer, not just the correct or most obvious ones. When unexpected answers are entered, they should give an appropriate response to get the student back on track. If the screen indicates the student should be able to exit or go to another part of the program, this capability should be allowed as stated.

 

 Optional criteria. Teachers reviewing courseware may consider a great many other criteria depending on their needs, the program's purpose, and the intended audience. These are detailed in Roblyer (1983), Lockard et al., 1990) and Merrill et al., (1992). Many of these criteria, which are listed below, are subjective in nature; it is up to the teacher to decide whether or not the courseware meets them.
Optional Instructional Design Criteria
Does the courseware state its objectives? ____ Prerequisite skills

Are skills specified that students will need to do the courseware activities?

Do instructional units follow a logical sequence based on skill hierarchies? Do tests match stated skills and are they good measures of the skill? Are stated skills "educationally significant" (e.g., in the curriculum)? Does courseware make good use of the medium? Is there evidence the courseware has been field-tested with students and revised based on this feedback before its release?
Optional Student Use Criteria
Is the program easy to use for the intended students? Does it require physical dexterity to answer items the students may not have even though they know the correct answers? Is a lot of typing required? Are the keys required to input answers easy to remember (e.g., pressing "B"for going "back")? Are alternate input devices allowed to make courseware more usable for special populations? Are there on-screen directions on how to use it? Are there print support materials to support on-screen activities? Is a "HELP" feature available if the student runs into difficulty? Can students skip directions, if they desire, and go straight to the activities? Do materials foster creativity rather than just rote learning? Are students given an on-screen summary of performance when they finish working?
Optional Teacher Use Criteria
Can teachers figure out, with minimum effort, how to work the program? Does courseware contain adequate recordkeeping and management capabilities? Are clear, nontechnical teachers manuals available with the courseware? Are courseware materials designed to integrate easily into other activities the teacher is doing? Does courseware improve the teacher's ability to teach the subject? Can teachers adapt the courseware for their needs by changing content (e.g., spelling words) or format (e.g., animated versus written feedback)?
Optional Presentation Criteria
Are graphics, animation, and color used for instructional purposes rather than flashiness? Are screens so "busy" or cluttered that they interfere with reading? Is speech of adequate quality so students can understand it easily? Does the program require peripherals the schools are likely to have (e. g., light pens, speech synthesizers)?
Optional Technical Criteria
Does the response judging allow for ALL possible correct answers and disallow ALL possible incorrect ones? Does the program present itself quickly so displays and responses are accomplished without noticeable delays? Can teachers transfer the courseware from one machine to another? Does courseware run on more than one platform? Do teacher or user manuals contain technical documentation on program operation and any technical features or options?