EDUC
865: Spring 2001
Instructor:
Ratna Nandakumar, Ph.D.Lectures:
T 4:00-7:00 p.m.
Office:
213 Willard HallRoom:
MDH 220
Phone:831-1635
Office
Hrs:
Mon: 2:30 - 4 p.m. and by appointment
Email:
nandakum@udel.edu
Textbooks:
1.Allen,
M. & Yen, W. (1979).Introduction
to Measurement Theory.California:Brooks/Cole
Publishing Co.(AY)
2.Traub,
R.E. (1994).Reliability for the
Social Sciences (Vol. 3).California:Sage
Publications, Inc. (T)
3.Camilli,
G. & Shepard, L. A. (1994). Methods for Identifying Biased Items
(Vol. 4). California:Sage Publications,
Inc. (CS)
Determination
of Grades:
Class
Participation20%
Midterm
Exam40%
Course
Project40%
Overview
of the Course:
This
course deals with the psychometric issues concerning the construction of
an educational/psychological instrument or questionnaire on a domain of
interest. The course provides the technical knowledge necessary for test
construction and responsible test use. The topics to be covered in this
course include principles of test construction, classical true-score theory,
reliability, validity, scaling, and procedures to identify biased items.
The
main emphasis of the course will be on the conceptual understanding of
the theoretical concepts and the logic that underlie standard test construction
and their applications to real data. In this regard, each of these conceptual
concerns of high quality of assessment and measurement will be applied
and explored in the context of real data collection situations of interest
to the students. As part of the course, students will construct a test,
refine it, collect data, analyze and interpret the results.
Prerequisites:
EDST 665 or equivalent
Reading
Assignments and Lectures:
We
will have a lecture/discussion format in the class.It
shall be assumed that students have read the assigned readings before the
class session.Each of you is expected
to contribute to class discussions.
COURSE
PROJECT:
You
are required to conduct and present a project in class.There
are several possible types of projects that can be conducted.
(1)
Test construction, validation--development of a paper-and-pencil test designed
to measure one or more attributes or traits (e.g., quantitative ability,
anxiety, attitude towards war, quality of instruction etc.).The
instrument would be given to a sample of subjects and appropriate reliability
and validity questions investigated.
(2)
Theoretical paper--an in-depth exploration of some concept or technique
in test or measurement theory [e.g., (a) a review and critical discussion
of factor analysis, (b) a detailed discussion of multiple regression or
validity].
(3)
Literature review of the research and methodology associated with some
instrument [e.g., (a) an extended review and critical discussion of recent
research on the validity and reliability of the MMPI, (b) a review and
synthesis of recent research on racial differences in ability, including
discussion of the statistical aspects of such research].
(4)
Computer project--writing a Fortran or PL-1 program to implement one or
more statistical procedures of interest in psychometrics, including an
application to real, or artificial data [e.g., program to perform detailed
item analysis].Prior programming
skills would be necessary to do a project of this type.
If
there is some other type of project that you would like to do that does
not fall into above categories, see me for possible approval.Project
proposals of types (2)-(4) should be very specific and well-formulated.
Project
Proposal:
A proposal outlining the rationale and methodology of your project is due
by March 20.Specially, proposals
for test construction should include:
(1)
Specification of the attribute, trait, or stimulus domain you want to measure.
(2)
Brief review of other research on the attribute or domain (include references).
(3)
As much detail as possible on the instrument--instructions, item format(s),
etc.
(4)
Number and type(s) of subjects to be employed; sampling procedures to be
used.
(5)
Brief discussion of how reliability and validity will be assessed.
Project
Report:
Your project should be written and formatted in APA publication style.
For example, a test construction project should include the following sections:
(1)
Abstract.
(2)
Introduction--indicating the nature and scope of problem investigated and
citing other pertinent research; statement of hypotheses, if any.
(3)
Method--description of (a) subject sample; (b) instrument, scales, etc.;
(c) administration procedures.
(4)
Analysis--description of statistical measures used, specification of item
analyses, how reliability and validity were assessed.
(5)
Results/Discussion (may be combined)--presentation of results of study;
use tables and figures where possible.Discussion
section should go into the meaning and significance of major results and
relate these to previous pertinent work; discussion of implications.
(***If
your study "bombed" (i.e., results not statistically "large" or significant
and/or uninterpretable), include in your discussion educated guesses about
what went wrong and what improvements in methodology would be made if the
study were to be repeated.)
(6)
References/Appendices.
Please
turn in typewritten copy of your project report in APA style.Include
as appendices correlation matrices, the instrument used, programs, etc.
Grievance
Policy:Any
reasonable request to regrade a particular item on an assignment or examinations
will be considered.However, the
instructor reserves the right to regrade the entire assignment or paper.
SYLLABUS
WeekSuggested
ReadingsTopics
______________________________________________________________________
Feb.
6AY: Chap. 2Review
of basic statistical concepts
Feb.
13AY: Chap. 6Principles
of test construction
(Delete
Sections 6.4-6.6)
Feb.
20AY: Chaps. 3&4True-score
theory and Reliabiltiy
T:
Chaps. 2-3
Feb.
27AY: Chaps. 4&5; Sec. 6.4Reliability
(contd.)
T:
Chaps. 4-6
March
6AY: Chap. 5Validity
T:
Chaps. 7-8
March
13AY: Chap. 5Validity
March
20AY: Chap 9Prediction
and Selection
March
27No ClassSpring
Break
April
3MidtermTurn
in the project proposal
April
10No ClassAERA/NCME
conference
April
17AY: Chap. 7Transforming
and equating test scores
April
24assigned readingsScaling
May
1 CS: Chaps. 1&4DIF
May
8SIBTESTDIF
May
15Class Presentations
______________________________________________________________________
NOTE THAT THE SCHEDULE IS ONLY APPROXIMATE.SOME ADJUSTMENTS MAY BE DESIRABLE AS A FUNCTION OF STUDENT BACKGROUND AND INTEREST TO EXPAND OR REDUCE COVERAGE OF SOME OF THE ABOVE TOPICS.