"Don't Know" Answers,Guessing Effect and Knowledge Levels:The Evidence from Genetics Knowledge Scale
Knowledge level has been conventionally measured by the number of correct answers, where "don't know (DK)" response was considered is incorrect answer. Different perspectives of educational measurement and political science have not reached agreement on the encouragement of the DK answers. Some of the previous studies have suggested that the encouragement of DK answers led to guessing response sets. This paper explores guessing response sets and their effect on the respondents' knowledge levels by using genetics knowledge scale as an example. In particular, the changes of guessing effect were examined by considering the characteristics of respondent and interviewer.
Data were drawn from Taiwan Genomic Survey collected in 2003. A group of adults aged 18-64 were selected using three-stage stratified sampling scheme. A total of 1090 interviews were completed. The results showed that regardless of sample weighting, the average number of correct answers among the ten-item genetics knowledge scale was 6.15(with a standard deviation of 2.10), which was similar to those found in the United States, Canada, Japan, and European countries. Without controlling for other factors, the knowledge scores of the respondents who did not guess at all were significantly lower than their counterparts who fully guessed, but were significantly higher than those who partially guessed. After controlling for the characteristics of respondent and interviewer, the knowledge scores of the respondents who partially guessed and fully guessed were lower than those who did not guess.
Guessing effect turned insignificant after controlling for other factors. For the respondents with a guessing rate between 0.01 and 0.99, whether their genetics knowledge increases with guessing rate requires further investigation. One of the reasons why guessing effect turned insignificant may be the discouragement of DK in the original survey design. It remains suggested to encurage more rigid experimental studies in the future.