>From Jared Spool > > In the ten years since I first started seeing > these tools on the market. I've never seen > results from a study that the team could > actually interpret and act on.
Ironically, your email arrived in my in-box just as I was taking a break from analysing the results of an online, un-moderated usability test. I don't use them all that often but do so routinely when I'm doing a measurement evaluation of an established web presence. Here in the UK, we have www.usabilityexchange.com They offer a large panel of users with disabilities. I find that it's handy to get a chunk of disabled participants doing the same (or as similar as I can make them) tasks as I'm running with the face-to-face participants. The remote participants are using their own technologies in their usual environment, which is a helpful further insight compared to any disabled participants that I can catch for my face-to-face but who then have to use my technology in my environment. As with the last time I tried this, I find myself turning to the comments made by the remote participants quite often as I write the report. It appeals to me that it's the actual words typed by the participant in their own time. I wouldn't want to run the remote test on its own as indeed it can be a bit misleading. For example, the remote participant might find the correct page on the web site but that doesn't say that they understood it. I also take your point that when using a panel like this, you don't know how much direct interest that they have in the site that you're testing. As it happens, this current job is a large government web site and the sort of tasks that I have designed are things that might spring at you out of a blue sky whether you're interested in them or not. For example, one task was about what you had to do with respect to this particular bit of government after a bereavement. Obviously we were careful to ensure that anyone who felt distress about the task didn't have to do it (no one refused). One face-to-face participant and one remote participant had in fact each been bereaved fairly recently, within the last year. Their task experience was remarkably similar to the participants who had no immediate personal interest in the task - my interpretation is that everyone was able to empathise somewhat with the task. For this evaluation, the remote panel has definitely added useful extra insight (and some helpful extra numbers) at for not much extra cost: Usability exchange charged me GBP 2500 (around USD 3500). One caveat: I'd be very wary of doing the remote unmoderated testing without some face-to-face alongside it. Best Caroline Jarrett www.formsthatwork.com "Forms that work: Designing web form for usability" ________________________________________________________________ Welcome to the Interaction Design Association (IxDA)! To post to this list ....... [email protected] Unsubscribe ................ http://www.ixda.org/unsubscribe List Guidelines ............ http://www.ixda.org/guidelines List Help .................. http://www.ixda.org/help
