Using the rasch measurement model for standard setting of the English language placement test at the IIUM
Loading...
Date
2007
Authors
Abu Kassim, Noor Lide
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
With the use of cutscores and standards for making high-stakes educational
decisions, efforts should be made to search for more defensible standard setting
methods. This study represents an effort to this end. The main intent of this study is to
investigate the efficacy of the Objective Standard Setting Method (OSS), which is based
on the Rasch Measurement Model, in constructing multiple cutscores that are valid and
defensible on tests utilizing diverse item types. The OSS, which was developed by Stone
(1996) to set a single cutscore on tests utilizing selected-response items, has been
demonstrated to yield valid results. However, its efficacy in handling other item types
and the construction of multiple cutscores has yet to be empirically established. As the
quality of the tests used in the standard setting endeavour influences the validity of
derived cutscores as well as the validity of examinee classification, assessment-related
issues are also of major concern. Measurement theory is one other aspect that requires
serious consideration. The need for a measurement model that transforms counts
correct into interval linear measures that are neither sample-bound nor test-bound, and
at the same time references an examinee’s performance (on the test) and status (based
on the standards set) directly to the measured construct cannot be underrated. The
same applies to the capacity to resolve important measurement and standard setting
issues. In this study the efficacy of the OSS was examined in the context of the English
Language Placement Test conducted at the IIUM. It was found that with the use of the
OSS, multiple cutscores on diverse item types can be easily set without compromising
the validity of the derived cutscores or standards. Additionally, with the use of the OSS,
the desired level of attainment can be directly translated onto the measured construct
and, thus, allowing the standards set to have real meaning and not just proportions of
correct answers. The Rasch measurement model has also proved to be useful in
resolving fundamental issues in measurement and standard setting. However, one
cautionary word is necessary. Regardless of how sound a standard setting method is,
the results of a standard setting study are bound to be impacted by test quality, judge
competency, performance level descriptions and other variables in the standard setting
process. This has been demonstrated very clearly in this study. Steps must, therefore,
be taken to address all these issues to ensure that the reliability and validity of derived
cutscores and standards are not compromised.
Description
PhD
Keywords
Education , rasch measurement model , English language