I just wanted to add that in the past, as many people know, we tried a few different kinds of testing and even hired a usability testing firm to help us. We conducted research in a lab here in SF and also did some remote testing, compensating participants with gift cards.
We learned that lab testing is very expensive, complicated and slow. It has its own unique filtering qualities that prevent certain kinds of people from participating and encourage others. Participants being in a foreign environment, using someone else's computer and being run through tasks with a giant 2-way mirror behind their back and cameras rolling might distort behavior a bit. Remote testing done with a facilitator and screen-sharing (like what Steven is talking about with Google Hangout) is still time consuming, but far cheaper and easier than lab testing and can be done on shorter notice. It filters out less tech-savvy people or those who use alternative or legacy devices like phones, tablets or older computers. It's interesting that it allows people to use a computer they are already familiar with, but it may not be relevant to the test. Remote testing done using usertesting.com is the cheapest and easiest, but even further filters out less tech-savvy people. I believe, from lots of first-hand experience and some research on the subject, that anytime you can get at least 5 users in front of a product and run them through well written tasks you are going to reveal about 80% of the problems. Getting fancy with the methodology usually only affects the final 20%. I'm really looking forward to having a UX testing person on staff who can facilitate more testing. I find it very valuable and would like to do more in the future. - Trevor On Thu, Mar 6, 2014 at 4:06 PM, David Gerard <dger...@gmail.com> wrote: > On 6 March 2014 23:47, Steven Walling <steven.wall...@gmail.com> wrote: > > > more automated remote testing and is $35/test (this is really cheap since > > the going US rate for an in-person test is something like a $50 Amazon > gift > > card). > > > off-topic on off-topic: Offer swag instead. Wikipedia branded stuff is > presently uncommon enough to *delight* people. I remember doing a > usability test for Ubuntu and accepting some stickers and a £2 USB > stick rather than a £40 cheque ... I could tell it was a £2 USB > because it stopped working 6 months later. > > Anyway. Work the swag angle. Puzzle globes. People LOVE that stuff. > > > - d. > > _______________________________________________ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
