Jared, Ok, so you like some remote methods but don't like UNMODERATED? So Let me add one word to my question before.
Give a small team a chance to come up with something new and innovative. You > seem to be writing it off before even trying it. I don't know if you have > ever used remote UNMODERATED, but if you have, then by the sound of it you > must have been using on an old tool, many years ago. > True or false? All the best James 2009/10/8 Jared Spool <[email protected]> > James, > You're confused. And you'll have to excuse me if I don't feel like arguing > with your misrepresentations of my viewpoints. > > My original comments about final inspection had nothing to do with a remote > testing tool. I suggest you read them again. > > I have no problem with remote research. I think it's an exceptionally valid > technique and recommend it to our clients regularly. I've also just endorsed > Nate Bolt and Tony Tulathimutte's awesome upcoming book ( > http://is.gd/45jxn) on the subject. Please read what I've written again > closely. > > I do think that UNMODERATED remote testing is a tool we could do without. > I've never stated anything about unmoderated non-remote testing, but only > because I don't know what that might be. (Actually, I have seen localized > studies where all the results were self reported, which I'm not a big fan of > either.) > > To take your list of seven.... my answers in italic > > *1) Who your users are* > *Your not going to find that out just by looking locally. If they are just > local then you have an issue that you need to address. > * > > I never said you'd find that out just by looking locally. > > > *2) What they are trying to do with the product* > *Remote will help you answer the question. As well as identify what works > in which country. > * > > I never said remote wouldn't help. > > > *3) How the product fits into their life* > *The way people live life differs greatly between cultures. Remote is a > method that can help you answer this question.* > > I never said that remote wouldn't help this. > > > *4) How they talk about the elements of the application (their terminology > and conceptual models)* > *This will be different for different cultures. Even between England and > the USA. * > > I never said that England had the same culture as the USA. > > > *5) What doesn't currently work (and needs attention) > 6) What does currently work (and needs to not be broken in a future > release) > 7) How you'll create elements that'll migrate people who are happy with key > features of the existing design into the new design (aka "embraceable > change")* > *For points 5 to 7 remote can answer these question quickly and cheaply. * > > I never said remote couldn't help this. > > I would love it if, you wanted to argue with me, you'd actually argue with > something I actually said. It really would make my side of the argument > easier. > > Thanks, > > Jared > > > > On Oct 8, 2009, at 12:26 PM, James Page wrote: > > Jared, > > Give a small team a chance to come up with something new and innovative. > You seem to be writing it off before even trying it. I don't know if you > have ever used remote, but if you have, then by the sound of it you must > have been using on an old tool, many years ago. Things have changed. I think > you are confused in when and how remote can be used. Our tool is not a final > inspection method, but can used at the start of a project to inform design > teams. I don't understand how you think it is a final inspection method. > Each tool and method is different, with different trade offs. > > This led us to start recommending that teams try to get every team member >> exposed to as many hours of observing users throughout the design process. >> The minimum we're recommending is 2 hours of observation every 6 weeks. The >> best teams have their team members observing users for several hours every >> week or so. >> >> If we agree that behavior is not homogeneous. Are you going to fly your > teams around the world? How do you know what you have found out in North > Andover in the USA applies to Andover in the United kingdom? > > To take your list of seven.... my answers in italic > > *1) Who your users are* > *Your not going to find that out just by looking locally. If they are just > local then you have an issue that you need to address. > * > *2) What they are trying to do with the product* > *Remote will help you answer the question. As well as identify what works > in which country. > * > *3) How the product fits into their life* > *The way people live life differs greatly between cultures. Remote is a > method that can help you answer this question.* > > *4) How they talk about the elements of the application (their terminology > and conceptual models)* > *This will be different for different cultures. Even between England and > the USA. * > > *5) What doesn't currently work (and needs attention) > 6) What does currently work (and needs to not be broken in a future > release) > 7) How you'll create elements that'll migrate people who are happy with key > features of the existing design into the new design (aka "embraceable > change")* > *For points 5 to 7 remote can answer these question quickly and cheaply. * > > How do you answer these questions when your users are spread all over the > country, if not the world. > > I don't believe that data from remote usability testing or data from eye >> tracking is flawed, misleading, or made-up. >> > So we don't disagree with Deming in that "In God we trust, all others bring > data."? > > I do believe that the inferences drawn from said data is almost always >> flawed, misleading, or made-up. >> > > This depends on the Study design. That is very easy to fix as many > Philosophers of Science have been dealing with this issue for years. Have a > hypothesis before you conduct your test, and don't fish. But surely this > applies to traditional lab testing as well. And is the reason why Web Stats > are great as Hypothesis generating tool, but not a tool to test a > hypothesis. > > Give the different tools a chance, please don't write them off before > trying them. > > All the best > > James > http://blog.feralabs.com > > > > > > > > 2009/10/8 Jared Spool <[email protected]> > >> Hi James, >> >> Thanks for telling me what I think. It's good to hear an outsiders >> perspective.However, from where I sit, I think I think differently from >> where you think I think. >> >> On Oct 8, 2009, at 5:24 AM, James Page wrote: >> >> I think where you are confused is that Deming did not believe in raw >>> targets. >>> >> >> I never said that Deming didn't believe in raw targets. What I said was, >> "You're creating a final inspection mentality, which Demming and the world >> of statistical quality control has taught us (since the 40s) is the most >> expensive and least reliable way of ensuring high quality." >> >> >>> I think our argument boils down to if you believe that peoples behavior >>> is homogeneous across the world. >>> >> >> I never said that either. Nor do I believe it. >> >> Want to try again at telling me what I think? >> >> Jared >> >> > > ________________________________________________________________ Welcome to the Interaction Design Association (IxDA)! To post to this list ....... [email protected] Unsubscribe ................ http://www.ixda.org/unsubscribe List Guidelines ............ http://www.ixda.org/guidelines List Help .................. http://www.ixda.org/help
