Monitoring Interview Duration for Survey Quality Control
To collect data in social sciences, it is common to use questionnaire survey methods. Data quality is essentially a function of the magnitude of error in data. If data have sufficient accuracy, it is said that the data quality is high. On the other hand, if the magnitude of error in data is large, the data are said to have poor quality. These criteria for data quality can be applied when using data to estimate population parameters. There are two forms of error in estimating parameters. The first, known as sampling error, is the error as a result of drawing a probability sample rather than conducting a complete enumeration. The second, known as nonsampling error, refers to all other forms of error that can occur during data collection and processing procedures. Therefore, efforts to improve data quality are directed at nonsampling error. Biemer and Lyberg (2003) provide an extensive discussion of methods for data quality improvement. They suggest focusing on steps to improve the data collection process. Often a good-quality process implies good quality of both the data sampled and the resulting output.
Using paradata and time stamps to manage survey data collection can lead to a more efficient operation that produces higher quality data at lower cost. In this study, we explore the effect of interview duration on data quality in the Taiwan Social Change Survey (TSCS). The TSCS, the first nationally representative survey in Taiwan, was established in 1985 and is conducted through face-to-face interviews. In it, interviewers are instructed to perform their tasks according to the key principles of standardized interviewing. If the interviewer complies with standardized procedures, an abnormal interview duration is less likely. Conversely, if the interviewer deviates from the standardized procedures, there is higher probability of an abnormal interview duration. Interview duration is very important for the interviewer with regard to planning fieldwork activities, and is a factor which influences his or her cost and benefit analysis. Short and well-paid interviews are financially and organizationally more attractive. Japec (2005) suggested monitoring interview duration, since it can tell us something about data quality.
This paper addresses the problem of quality control in a survey based on interview duration data, and applies control charts to the monitoring process. We conducted this process in three parts: (1) classification of interviewers based on interview duration data using principal components analysis. The aim was to improve the efficiency of monitoring the data collection process. (2) detection of the optimal rate among different interviewers' classifications, and avoidance of continually collecting data by using non-standard interview procedures which affect the quality of survey data. (3) use of a median control chart to monitor the potential causes of variation between individual interviewers to find out why some interviewers' interview times exceed the control limit. To assess the applicability of the proposed method, we used the 2015 Work and Life module of the TSCS to illustrate the proposed process for survey quality control. The research goal of this article was not simply to monitor, but also to improve the quality of the process over time. Based on the proposed process, it is notable that the inspection operation cost can be more effectively controlled.