James,
You're confused. And you'll have to excuse me if I don't feel like
arguing with your misrepresentations of my viewpoints.
My original comments about final inspection had nothing to do with a
remote testing tool. I suggest you read them again.
I have no problem with remote research. I think it's an exceptionally
valid technique and recommend it to our clients regularly. I've also
just endorsed Nate Bolt and Tony Tulathimutte's awesome upcoming book (http://is.gd/45jxn
) on the subject. Please read what I've written again closely.
I do think that UNMODERATED remote testing is a tool we could do
without. I've never stated anything about unmoderated non-remote
testing, but only because I don't know what that might be. (Actually,
I have seen localized studies where all the results were self
reported, which I'm not a big fan of either.)
To take your list of seven.... my answers in italic
1) Who your users are
Your not going to find that out just by looking locally. If they are
just local then you have an issue that you need to address.
I never said you'd find that out just by looking locally.
2) What they are trying to do with the product
Remote will help you answer the question. As well as identify what
works in which country.
I never said remote wouldn't help.
3) How the product fits into their life
The way people live life differs greatly between cultures. Remote is
a method that can help you answer this question.
I never said that remote wouldn't help this.
4) How they talk about the elements of the application (their
terminology and conceptual models)
This will be different for different cultures. Even between England
and the USA.
I never said that England had the same culture as the USA.
5) What doesn't currently work (and needs attention)
6) What does currently work (and needs to not be broken in a future
release)
7) How you'll create elements that'll migrate people who are happy
with key features of the existing design into the new design (aka
"embraceable change")
For points 5 to 7 remote can answer these question quickly and
cheaply.
I never said remote couldn't help this.
I would love it if, you wanted to argue with me, you'd actually argue
with something I actually said. It really would make my side of the
argument easier.
Thanks,
Jared
On Oct 8, 2009, at 12:26 PM, James Page wrote:
Jared,
Give a small team a chance to come up with something new and
innovative. You seem to be writing it off before even trying it. I
don't know if you have ever used remote, but if you have, then by
the sound of it you must have been using on an old tool, many years
ago. Things have changed. I think you are confused in when and how
remote can be used. Our tool is not a final inspection method, but
can used at the start of a project to inform design teams. I don't
understand how you think it is a final inspection method. Each tool
and method is different, with different trade offs.
This led us to start recommending that teams try to get every team
member exposed to as many hours of observing users throughout the
design process. The minimum we're recommending is 2 hours of
observation every 6 weeks. The best teams have their team members
observing users for several hours every week or so.
If we agree that behavior is not homogeneous. Are you going to fly
your teams around the world? How do you know what you have found out
in North Andover in the USA applies to Andover in the United kingdom?
To take your list of seven.... my answers in italic
1) Who your users are
Your not going to find that out just by looking locally. If they are
just local then you have an issue that you need to address.
2) What they are trying to do with the product
Remote will help you answer the question. As well as identify what
works in which country.
3) How the product fits into their life
The way people live life differs greatly between cultures. Remote is
a method that can help you answer this question.
4) How they talk about the elements of the application (their
terminology and conceptual models)
This will be different for different cultures. Even between England
and the USA.
5) What doesn't currently work (and needs attention)
6) What does currently work (and needs to not be broken in a future
release)
7) How you'll create elements that'll migrate people who are happy
with key features of the existing design into the new design (aka
"embraceable change")
For points 5 to 7 remote can answer these question quickly and
cheaply.
How do you answer these questions when your users are spread all
over the country, if not the world.
I don't believe that data from remote usability testing or data from
eye tracking is flawed, misleading, or made-up.
So we don't disagree with Deming in that "In God we trust, all
others bring data."?
I do believe that the inferences drawn from said data is almost
always flawed, misleading, or made-up.
This depends on the Study design. That is very easy to fix as many
Philosophers of Science have been dealing with this issue for years.
Have a hypothesis before you conduct your test, and don't fish. But
surely this applies to traditional lab testing as well. And is the
reason why Web Stats are great as Hypothesis generating tool, but
not a tool to test a hypothesis.
Give the different tools a chance, please don't write them off
before trying them.
All the best
James
http://blog.feralabs.com
2009/10/8 Jared Spool <[email protected]>
Hi James,
Thanks for telling me what I think. It's good to hear an outsiders
perspective.However, from where I sit, I think I think differently
from where you think I think.
On Oct 8, 2009, at 5:24 AM, James Page wrote:
I think where you are confused is that Deming did not believe in raw
targets.
I never said that Deming didn't believe in raw targets. What I said
was, "You're creating a final inspection mentality, which Demming
and the world of statistical quality control has taught us (since
the 40s) is the most expensive and least reliable way of ensuring
high quality."
I think our argument boils down to if you believe that peoples
behavior is homogeneous across the world.
I never said that either. Nor do I believe it.
Want to try again at telling me what I think?
Jared
________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... [email protected]
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help
