2009/5/7 Brian <[email protected]>: >> >> Based on these criteria, the 2,500 users that responded to our survey were >> filtered down to 500 viable subjects based on their answers to these >> questions. The team, along with B|P, partnered with Davis Recruiting to >> contact, filter, and screen these 500 participants based on their Wikipedia >> contribution history, Wikipedia usage patterns, their given reasons for not >> contributing, and their talkativeness and openness to discuss their thoughts >> and actions. From 2,500 users, we ended up with 10 study participants and >> 3-5 waitlisted participants.
> You went from 2,500 subjects to just 10? The purpose of a study like this is focused observation of the behavior of individual human beings. As David has pointed out, for any study like this there are laws of diminishing returns, and any serious observation of an individual is time-consuming and costly (raw data is worthless if you can't analyze it). That's why usability gurus like Nielsen suggest "5 is enough" for most tests: http://www.useit.com/alertbox/20000319.html - due to our highly diverse audience, we chose a larger group, and we split between remote and lab testing to compensate for biases of both methods. This has worked well to identify plenty of very obvious usability barriers to focus on. There are alternative data collection methods such as large scale quantitative testing where the level of individual engagement is limited; those can give you behavioral patterns etc. They can be useful, too, but are an entirely different thing. -- Erik Möller Deputy Director, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate _______________________________________________ foundation-l mailing list [email protected] Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
