I've been running usability studies since 1974. In 1980-81, the
strategy I settled upon was to continue adding testers (qualified
subjects) until I had learned nothing new from two people in a row.
Depending on the circumstances, that point was usually reached after
4-8 tests, in the same range that other usability professionals have
discovered.
Often, an early tester--sometimes the very first--would encounter a
problem such as ambiguous or misleading copy that would obviously
affect a lot of users. We'd fix such problems immediately, often
before the next tester arrived.
Since that time, I haven't seen many situations where most serious
problems weren't exposed by a well-designed and carefully observed
study of 4-8 qualified subjects, people who would be likely users of
the software with typical levels of prior experience and knowledge.
Problems encountered too rarely to detect with a small number of
testers can still be serious. But most such problems are more
effectively discovered in other ways, such as analysis of customer
service and usage logs.
Errors with profound consequences, such as physical injury or
substantial economic loss, require different approaches, including
careful walkthroughs with security experts, large beta trials in which
adverse consequences are artificially limited, and exposure of test
subjects to intentionally risky scenarios within simulated environments.
Larry Tesler
________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... [email protected]
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help
