2011/1/10 Eino Keskitalo <eino.keskit...@gmail.com>: [snip] > The testers should be gamers, who have not played roguelike games > (such as Nethack, ADOM, Angband and Crawl itself) before. > > The goal of the testing is to assess how well the new tutorial works. > The previous tutorial was tested in 2010 in the Project 1 and the > Usability Testing courses. We can compare results of those tests to > the new ones and see if the learnability (and also general usability) > of the game has improved during the year. > > Additional requirements: all reports and other material that is > delivered to the Stone Soup project should be in English. Due to the > open development nature of the game, the report will be published in > its wiki. > > interesting and probably beneficial too. I was thinking of suggesting > that the structure of the testing be: have the tester find and start > the tutorial, complete it, and with the remaining time play the > regular game (or hints mode, or whatever the tester wants). This way > we can probably see if the things tutorial teaches the player > translates to actual game-playing skills.
Okay, me and Janne had the meeting with the usability testing team, It was good, there are five people in the group. We presented the above to them, plus that Johanna is overhauling tutorial at the moment. What'll happen is that the team (who've been tasked to think of a spiffy name for themselves[*]) are themselves getting to know the game and will do heuristic evaluation on it. Then they will plan and execute user testing with testers, who'll be gamers with no roguelike experience. The timeframe is on "asap" basis, with absolute latest of mid-March to being the user tests (that's my understanding). However, it looks to me that Johanna has a great pace regarding the tutorial update, so it looks good. Johanna, comments? There'll be an in-team irc channel on ircnet, Me and Janne will be helping them with material and links to dev builds, plus we inviteded the team to ##crawl-dev. State of the tiles interface looked good, with the detached command tab and just four tiles missing. However I noticed that there seemed to be a lack of "rest one turn" button, or if it's done in some other manner, it wasn't immediately obvious. (I tried clicking your own character but that was the pick-up command.) Also I'd like to mention that this testing, and also last years testing (including our project) are part of UKKOSS research project, of Mikko Rajanen from our department (Information Processing Science in the Oulu University). The project researches how usability expertise can be brought into open source projects. We did not mention this last year, because we were supposed to do kind of an "infiltration mission", where the target project did not know they were part of a larger research project, and try to see if/how this affected the outcome. However this wasn't well handled and it leaked on IRC, so we decided that we'd drop that stuff from now on. I hope no-one feels offended by this revelation. :) I'd like to thank Mikko for the unique opportunity to get so much usability work done on the project! I'm in a bit of a messy state of mind, I might have left something out, so if there's something unclear or you have related questions, just ask away! --Eino [*] I realized how important names were during last year, the UP team didn't have a good way of referring to itself until someone coined "UP team", and the name "Immortal Warwalrus" on the other hand has been a subject of much elation on ##crawl-dev ... so names are important! ------------------------------------------------------------------------------ Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d _______________________________________________ Crawl-ref-discuss mailing list Crawl-ref-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/crawl-ref-discuss
