Self-selection bias versus nonresponse bias in the Perceptions of Mobility survey

Abstract

In 2005 the Netherlands Institute for Social Research/SCP conducted a survey of people's perceptions of mobility (Harms 2005). This aper reports the results of a follow-up study of the Perceptions of Mobility survey in which both respondents and nonrespondents to the original survey were re-contacted to ask their opinion on six questions central to subjective mobility, while also collecting some other information thought to be relevant to opinion and response. The goal of the follow-up survey was to compare the former respondents and nonrespondents in order to evaluate whether nonresponse bias existed. The context of this comparison is the existence of a survey using the same questionnaire but a convenience sample with self-selection, rather than a random sample of the Dutch population. Although some former nonrespondents did cooperate this time, the follow-up suffered 70% nonresponse in the group of former nonrespondents and only 40% in the group of former respondents. Analysing only the respondents to the follow-up, there appear to be no differences. However, extra information was available about the nonrespondents that could be used to multiply impute their opinions. Multiple imputation does not yield any differences for four out of six study variables, while for one of the variables-attitude towards the bicycle-a clear difference is found if the imputation model is correct. This suggests that the original sample may have been biased concerning this opinion, while for others no problems with the estimates were found. The paper concludes that on the one hand, the random sample with nonresponse bias appears just as bad or good as the self-selected convenience sample in some situations. On the other hand, indications of bias were found in other situations, and without a study such as the one conducted here one cannot know which of the two situations applies.

Furthermore, it was the re-interviewing of the nonrespondents which made it possible to detect and correct for differences. Such knowledge can only be gathered in a sample survey, since in convenience samples the non-participants can usually not be identifi ed. Thus, in a sample survey combatting nonresponse rates and collecting auxiliary data can reduce nonresponse bias, while in a nonrandom survey one can only assume that there is none after correction for background information.