Estimating Scale Values of Response Anchors in Likert-type Scales
Social science and psychological research often uses questionnaires or scales to collect data. Likert-type scales are the most commonly used scales in questionnaires or scales when collecting data. The psychological scale values of the scale labels/descriptors/anchors have important implications in the analysis and interpretation of data. Different methods and formats have been proposed for estimating the scale values of a Likert-type scale. Previous studies have used successive interval and simplified successive interval estimation methods. Studies have also used an open-response format and category-level assigning format to study the scale values. However, there is no consensus and still a lack of studies comparing different estimation methods. In this study, three estimation methods of label scale values are investigated: the paired-comparison method, direct rating method, and simultaneous ranking evaluation method. Three common types of scale labels are investigated in this study: agreement-, conformity-, and frequency-type. A review on the usages of Likert scale labels in literature was conducted, 24 agreement-type labels, 24 conformity-type labels, and 20 frequency-type labels were included in this study. There were 97 subjects (27 males and 70 females) who participated in this study. The mean age of the subjects is 20.68 years old with a SD of 1.54. The subjects took part in lab experiments with three tasks. In the direct rating task, subjects used a mouse to drag a marker on a horizontal bar to indicate the rating of a given label stimulus. In the pair-comparison task, subjects used the mouse to drag a marker on a horizontal bar to indicate the degree of difference of a given pair of labels. In the simultaneous ranking evaluation task, subjects were presented with all the labels on the top panel of the screen and the subjects’ task was to drag these labels in any order they preferred to any location on the horizontal bar located in the center of the screen. Subjects could adjust the location of any label on the horizontal bar before they clicked the “confirm” button to submit their answer. Subjects’ responses to the three tasks were recorded on a 0-1000 scale for further analysis. The collected data were analyzed and compared the psychological scale values estimated by the three estimation methods. The results suggested the equal space assumption between points on the Likert scale is in general not hold based on the estimated scale values of this study. The results also suggested substantial differences among the three estimation methods. Moreover, the estimated scale values of the selected labels were also not evenly distributed on the evaluation scale; more labels were loaded toward the extreme portion of the scale. The possible explanations for this observation are discussed in the discussion section. Since it is difficult to locate appropriate labels based on the estimated psychological scale values on a Likert-type scale, we suggested using only the “end-label” on a Likert scale in practice, for example, “completely disagree- completely agree” and “never-always” as the labels on a Likert-type scale. The limitations of the present study were presented in the discussion section as well as references for further reading on these topics. Other psychometric methods such as item-response theory (IRT) and Fuzzy Linguistic/numbers approach for dealing with Likert-type scale labels were also discussed in the concluding remarks.