On Tue, Apr 15, 2014 at 10:12 AM, Steven Walling <[email protected]>wrote:
> Are you volunteering to build such a survey tool? ;-) > Will see if I find the time. "Survey" probably gives the wrong idea here, it is really just an overlay with two buttons, more of an interactive A/B test. Could be probably cobbled together from GuidedTours and EventLogging. > When it comes to using a survey to catch problems early and gauging > preferences, a survey still very much suffers from the self-selection bias > that all opt-in options have. It's just the name of the game. When you move > something from opt-in to opt-out you reach a wider audience and encounter > new complaints/questions/bugs. You can survey the opt-out audience before actually enabling any changes; that is a good way of catching those bugs without actually causing them. Of course, that point is moot now, and the refresh seemed like a simple change without the benefit of hindsight. Still, it might be useful to run such a survey (or surveys) even now: - Which fonts users would prefer is mostly based on educating guesses now. Complaints and bugs are much more heavily self-selected than a survey (especially a super-short one-click survey), so even though the results would still be slanted towards more active users, you would get a better picture of severity. - There is a lot of uncertainty about how widespread certain bugs are (e.g. ClearText issues); showing an affected text and asking "Does this look good to you?" is an easy way to get data about that. _______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
