Kaz's SAS, HLM, and Rasch Model
A Bootstrap Approach to Rating Scale Optimization
Large National Data Sets
Kaz Library
SAS manuals
What is "error"?
Rasch Model
Factor Analysis
Reading output via SAS
Excel functions for Statistical Analysis
Essays on learning statistics
Statistics in Japanese
My profile
My SAS questions and SAS's responses
My work tool box

SAS programs used for this paper
  • Optcat sample 1 (3 rating category data, Liking Science data set)
  • Optcat sample 2 (4 rating category data, No data inserted in this version.  See other samples to learn how the data can be inserted into this synatx.)
  • Optcat sample 3 (5 rating category data, Death & Dying data set)

A Bootstrap Approach to Rating Scale Optimization

Eric Van Lente, University of Chicago

George Karabatsos, University of Illinois at Chicago

Kazuaki Uekawa, American Institutes for Research

Submitted to a book project, Access to the Foundations of Measurement: Professional Identity in the Career of Benjamin D. Wright.
Whenever  rating scales are administered to test respondents, for several possible reasons, the respondents may not use the rating categories as intended by the test constructor. This can lead to a psychometrically disordered rating scale, which in turn, causes inconsistencies in the measurement of respondents. Many different methods have been proposed to diagnose rating scale inconsistencies, which in turn help the analyst decide the “optimal rating scale”, i.e., the particular recategorization of the rating scale that eliminates inconsistencies. However, there is no clear consensus as to which method is best, and furthermore, it could be argued that all of the available methods are sample dependent because none of the algorithms conventionally used attempt to maximize model generalizabilty. This study introduces a sample-free method of rating scale optimization, based on the bootstrap, which addresses the issues just mentioned. The bootstrap is a general statistical procedure that simulates the population distribution by resampling from the original (sample) data set with replacement. A given Rasch rating scale model, employing a particular rating categorization, is used to analyze the resampled data sets. The fit of that model averaged over these data sets indicates its generalization error, defined as the model’s ability to predict data over the population of respondents. With the same resampled data sets, generalization error can be computed for each of a number of different Rasch models employing different rating categorizations. In this way the optimal rating scale is identified as the one that minimizes generalization error. The bootstrap method with be demonstrated with two real data sets arising from rating scale questionnaires. Following these illustrations we conclude by discussing other situations in which the bootstrap method may be useful. For example, the method easily handles the task of rating scale optimization of visual analog scales. Furthermore, for any given data set, the method naturally handles the task of selecting among the many Rasch models for polytomous response data.


Enter supporting content here

Copyright 2005 KU
For information inquiry (AT) estat.us