Using the rasch measurement model for standard setting of the English language placement test at the IIUM
Loading...
Date
2007-03
Authors
Abu Kassim, Noor Lide
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
With the use of cutscores and standards for making high-stakes educational
decisions, efforts should be made to search for more defensible standard setting
methods. This study represents an effort to this end. The main intent of this study is
to investigate the efficacy of the Objective Standard Setting Method (OSS), which is
based on the Rasch Measurement Model, in constructing multiple cutscores that are
valid and defensible on tests utilizing diverse item types. The OSS, which was
developed by Stone (1996) to set a single cutscore on tests utilizing selectedresponse
items, has been demonstrated to yield valid results. However, its efficacy in
handling other item types and the construction of multiple cutscores have yet to be
empirically established. As the quality of the tests used in the standard setting
endeavour influences the validity of derived cutscores as well as the validity of
examinee classification, assessment-related issues are also of major concern.
Measurement theory is one other aspect that requ"ires serious consideration. The
need for a measurement model that transforms counts correct into interval linear
measures that are neither sample-bound nor test-bound, and at the same time
references an examinee's performance (on the test) and status (based on the
standards set) directly to the measured construct cannot be underrated. The same
applies to the capacity to resolve important measurement and standard setting
issues. In this study the efficacy of the OSS was examined in the context of the
English Language Placement Test conducted at the IIUM. It was found that with the
use of the OSS, multiple cutscores on diverse item types can be easily set without
compromising the validity of the derived cutscores or standards. Additionally, with the
use of the 055, the desired level of attainment can be directly translated onto the
measured construct and, thus, allowing the standards set to have real meaning and
not just proportions of correct answers. The Rasch measurement model has also
proved to be useful in resolving fundamental issues in measurement and standard
setting. However, one cautionary word is necessary. Regardless of how sound a
standard setting method is, the results of a standard setting study are bound to be
impacted by test quality, judge competency, performance level descriptions and other
variables in the standard setting process. This has been demonstrated very clearly in
this study. Steps must, therefore, be taken to address all these issues to ensure that
the reliability and validity of derived ·cutscores and standards are not compromised.
Description
Keywords
Measurement theory is one other aspect , that requ"ires serious consideration.