On Mon, Aug 1, 2011 at 10:35, Felipe Contreras <[email protected]> wrote: > After going through all the feedback, here's the second version of the > proposed survey.
Surveys require careful consideration of the phrasing of questions and careful consideration of the selection bias of respondents. It doesn't look like you have considered either, even after the bias in phrasing of the questions was pointed out to you in the first round of emails. For example, here's a classic: Do you approve of the President's job on foreign affairs? versus: How would you rate the President's job on foreign affairs? The first question is leading by punching the survey taker in the face with a statement about their value judgement before you've even finished saying the sentence. The second asks the survey taker to make their own value judgement. Second, you have to carefully consider who is motivated to take a survey. People who are angry are much more likely to respond to a survey versus people who are happy. You can mitigate this problem by offering a survey that appears to have nothing to do with the subject matter that you're really looking for an answer on so that you get a truly random sampling of Linux users. You also must be careful not to recruit people to take the survey from communities which will contain angry people. For example, going to forums to find people to take a survey automatically selectively biases from people who were likely there to solve some kind of problem and are so already in a particular state of mind. There are hundreds of strategies to work around the selection bias problem. It *is* a good idea to run a survey but unless it's done carefully, you could actually do far more damage than good by reporting results which are not accurate or--worse--call in to question the motivations of the study. The survey you're proposing will require months of work to do correctly and is likely best achieved as a part of a large group effort or as part of an academic or non-profit funded venture. _______________________________________________ desktop-devel-list mailing list [email protected] http://mail.gnome.org/mailman/listinfo/desktop-devel-list
