I agree absolutely that there comes a time to prioritize problems that
surface in user research.
In my experience, the teams that create the best experiences are not
concerned necessarily with the percentage of users that had the
problem (and I'd argue that in the small tests that most of us do
percentages are bogus).
Instead, they're asking, What do we want the user experience to be?
What are the constraints of the technology? What are the priorities of
the business? Which usability issues prevent us from reaching the
vision we have for this design?
Dana
On Mar 12, 2009, at 11:00 AM, James Page wrote:
@dana
Just get a mix of participants who do and are motivated to do what
you're interested in observing.
agree.....
If one or two participants in your mix have the issue, you want to
fix that because you don't want *anyone* to have it.
Totally agree in the ideal world all issues should be fixed. With
Webnographer some of our clients are so overwhelmed by a long list
of issue, what they are wanting is prioritisation of issues. Many
times there is a very long list of usability issues that have been
waiting for years in some cases. By being able to say X% of users
experienced this issue starts helping the solution to be
prioritised, and fixed. Often there is an argument between one camp
and another camp if a usability issue is real or not.
Prioritisation it seams is very important to Agile operations.
James
http://blog.feralabs.com
2009/3/12 Dana Chisnell <[email protected]>
Ah, I meant with regard to age. If the sample is 8, say, in 99.9% of
cases, age won't matter. Just get a mix of participants who do and
are motivated to do what you're interested in observing.
If you're testing 30 or 50 or 100 participants, you might want to
pay attention to make sure you have participants from all the age
ranges you care about, but you shouldn't be selecting or screening
on age as long as they do and are motivated to do the same kinds of
tasks with your designs.
Most demographics don't matter in usability testing, most of the
time. (Of course, there are exceptions.) Why? Because the purpose of
usability testing is not to generalize preferences to a larger
audience but instead to identify problems with a design that cause
frustration and confusion. If one or two participants in your mix
have the issue, you want to fix that because you don't want *anyone*
to have it.
Dana
On Mar 12, 2009, at 10:21 AM, James Page wrote:
@dana
I am bit confused here by your question "What difference does it
make how many you're testing?"
Surely factors such as "margin of error", and "statistical power"
are important, or are they not?
The point of testing is to find out if your wrong, or right. How do
you know if your wrong or right based on a small sample.
@jenrandolph
On remote usability testing we get more behavioural differences by
machine configuration, then by age. What I mean by machine
configuration is manufacture and screen size. Mac users are
different, why - I don't know. And we get allot of behavioural
differences by culture - (place of birth vs residence). Also
environment seams to have quite a large impact. People in the lab,
and at home spend more time to trying to complete a task before
giving up, then people at work. This is of course impacts success/
failure rates.
We are doing more research here.
James
http://blog.feralabs.com
2009/3/12 Dana Chisnell <[email protected]>
What difference does it make how many you're testing? By breaking
the sample into groups, you're just creating extra work. Are you
going to compare the data by age group? Why would you do that? The
only reason I can think of is if you're creating different sites.
You're not.
Dana
On Mar 12, 2009, at 9:21 AM, James Page wrote:
Out of interest how many participants are you testing with? Could
you break the numbers down?
James
http://blog.feralabs.com
2009/3/12 Dana Chisnell <[email protected]>
Thanks for the prompt, Jared. There's no reason to limit the age
range *at all.* As long as the behaviors are the same -- that is,
the task goals of the users -- across age ranges, then it doesn't
matter a bit how old the participants are.
As members of UPA, people over 65 would very likely have the same
tasks and goals in mind as someone younger: Maintain membership
information, renew memberships, find out what's going on in the
association, get in the consulting directory, find out who is on
the board, find out where the conference is, etc.
Limiting the age range wouldn't benefit the research. In fact,
limiting may be a detriment.
Dana
:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::
Dana Chisnell
desk: 415.392.0776
mobile: 415.519.1148
dana AT usabilityworks DOT net
www.usabilityworks.net
http://usabilitytestinghowto.blogspot.com/
On Mar 12, 2009, at 12:04 AM, Jared Spool wrote:
So? Why limit the age range? How does that benefit the research?
On Mar 11, 2009, at 10:20 PM, Todd Zaki Warfel wrote:
Perhaps because the core audience isn't older than 65? Not to say
that there aren't any, but I'd imagine, based on the meetings and
conferences that I've been to, that the number of people over 65
are statistically quite small.
On Mar 11, 2009, at 11:54 AM, Dana Chisnell wrote:
May I ask why the age range limits to 65?
________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... [email protected]
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help
________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... [email protected]
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help
