Catch Them When You Can: Speeders and Their Role in Online Data Quality
Abstract:
The role of panelist engagement on survey data quality in online panels has been a subject of discussion and commendation by industry and academic leaders (see Baker et. al. 2010). While there are many indicators of data quality in online surveys such as item nonresponse (Rao and Gravelle 2008) and breakoffs (Peytchev 2009; Heerwegh and Loosveldt 2006), one in particular has gained some attention in recent times: survey completion time. Much of this attention is coming in light of the scrutiny facing online panels today on the issue of panelists participating in numerous surveys in short time periods that suggests their motive for maximizing returns (i.e. incentives) while minimizing survey effort. Termed colloquially as “Speeders,” survey respondents who complete a survey more rapidly compared to the norm are the target of investigation in this study. The data behind this study come from an online mobile phone usage survey sourced from several online sample vendors. In addition to computing an index of speeding, we analyze the relationship between speeding and other data quality measures in order to test the popular notion that speeders provide lower quality data than their non-speeding counterparts. Furthermore, we investigate the variation of the index score for speeding across different samples, taken as a whole or as a random subsample within each sample. Lastly, using a multivariate framework, we analyze the demographic predictors of speeding after controlling for sample source and respondents’ mobile phone service characteristics. In addition to presenting quantitative evidence of the effect of speeding on data quality, this article concludes with a call for further research on the role of speeding in online panel surveys and recommendations for future research.
Recommended Citation:
Gutierrez, C., Wells, T., Rao, K., & Kurzynski, D. (2011). Catch Them When You Can: Speeders and Their Role in Online Data Quality. Paper presented at the Midwest Association for Public Opinion Research, Chicago, IL.
Attached Documents:
- MAPOR 2011 Program (see page #18 for the mention)
- For a copy of this paper/presentation, please send me a comment with your email address in the box below.
Could you please send me a copy of your paper.
Thanks,
Ankush
Can you send me a copy of the speeder paper? Thank you.
Hi Kumar,
Would you send me a copy of this paper? Best, Nancy
Could you please send me the paper/presentation? That would be great.
Thanks a lot.