Evaluation of Alternative Weighting Approaches to Reduce Nonresponse Bias
Abstract:
With declining response rates, surveys increasingly rely on weighting adjustments to correct for potential nonresponse bias. The resulting increased need to improve survey weights faces two key challenges. First, additional auxiliary data are needed to augment the models used to estimate the weights. Depending on the properties of these auxiliary data, nonresponse bias can be reduced, left the same, or even increased. Thus, the second challenge is to be able to evaluate the alternative weights, when the assumption of “different estimates means less bias” may not hold. Ideally, data need to be collected from as many nonrespondents as possible to provide direct estimates of nonresponse bias.
The Nielsen TV Diary, a national probability-based survey, has experienced the ubiquitous trend in declining response rates. In 2012, a nonresponse bias study was conducted in three geographic areas. An abbreviated instrument to evaluate nonresponse bias was mailed to Diary nonrespondents and respondents. A random sample of remaining nonrespondents was selected for in-person interviewing, increasing the three response rates to 58%, 68%, and 73%. We then computed several alternative weighting adjustments to produce estimates for the Diary respondents, and compared them to the estimates based on the Diary respondents and nonrespondents. Two general ways to improve weighting adjustments are to expand the types of auxiliary data and to change how the auxiliary data are modeled. We evaluated four sets of weights: the current approach based on poststratification, adding sample-based nonresponse adjustment based on matched telephone number (Link and Lai, 2011), adding census geocoded data (e.g., Biemer and Peytchev, 2013), and adding interaction effects in poststratification. We describe the ability of each approach to reduce nonresponse bias and suggest avenues for future research. We also note the underutilization of two stage sampling for nonresponse as a cost-effective method to produce direct estimates of nonresponse bias.
Recommended Citation:
Peytchev, A., Rao, K., Link, M. W., & Shagrin, C. (2014). Evaluation of Alternative Weighting Approaches to Reduce Nonresponse Bias. Paper presented at the American Association for Public Opinion Research, Anaheim, CA.
Attached Documents:
- AAPOR 2014 Program (see page #195 for the mention)
- For a copy of this presentation, please send me a comment with your email address in the box below.