Oh, man. This is hardly solid research. Now you're just begging for a debate. ; )
1. They tested the effectiveness of personas by performing heuristic evaluations. That's like testing the happiness of a cat by determining the ground speed of a duck. Usability and the effectiveness of personas have little to do with each other. Any decent designer can put something together that does well in a heuristic evaluation %u2014 it doesn't mean the app meets the needs of its audience. Not even remotely. 2. They say they set up the teams to be of relatively similar strength, but how was that done, exactly? How do you make sure each team has a relatively equal level of experience, skill, talent, knowledge, and an ability to persuade? How do you set up three teams in a way that no one person on any one team is able to talk the rest of that team into a bad idea based on faulty knowledge? It may seem like I'm nitpicking on this one, but I think this is a pretty important point. Human beings are slippery. 3. The paper doesn't indicate what the control group used instead of personas beyond "image boards" (which are meaningless as research tools). All three of these teams were comprised of people from the same class %u2014 meaning they all had received education on personas by the same instructor. What else did the instructor teach these people that they could put to use to come up with a good design without personas? The only thing this study shows is that 2 out of the 3 teams created a more usable design as measured against heuristics (this assumes, of course (and it's a big assumption), that the evaluators did good evaluations), and that they happened to be the same groups that used personas in the project. At the absolute best, this is a loose correlation. It's absolutely not proof of genuine causation. I could have fared as well as any of them without personas and without a team. 4. Even if you throw out arguments # 2 and # 3 above, # 1 still makes it all a moot point. All that said, I still love you Jared. : ) -r- Hi Robert, Thanks for taking the time to read the paper %u2013 In answer to the 3 points that you made regarding the validity of the paper. Point 1 %u2013 happiness of cats V ground speed of ducks. Everyone knows that cats like to chase ducks. The happiness of the domestic feline is therefore inversely proportional to the ground speed of said duck. But seriously, the research did not use heuristics to evaluate the effectiveness of personas. I used heuristics to evaluate the usability of the resulting designs %u2013 allowing me to compare all of the diverse design solutions consistently with each other. (user testing would have been preferable but not feasible). The measure of effectiveness was based on a number of factors including the usability of the solution, interviews with students, observations from their tutors and group discussions after the project. My initial research question was to see if using personas made any difference %u2013 and based on the research I found that they did. Point 2 %u2013 teams of equal strength. The students had completed 2.5 years of industrial design course when I conducted the experiment. The course is entirely project based. The same tutors work with the students day-in, day-out. After 2.5 years you have a pretty accurate idea of each student%u2019s ability. The fact that all students were working in close proximity was an acknowledged weakness of the study. To work around this unavoidable problem we informed all students of the experimental nature of the project %u2013 not the specific goals, but the fact that each group would be using a different design tool to solve the brief. The importance of secrecy between the groups was stressed and we simply asked for the student%u2019s cooperation in this regard. At the end of the research we gathered feedback on the level of %u2018information leaks%u2019 between groups. While a small degree of information did pass between groups, it was not significant. Point 3 %u2013 The control group All teams received a set of briefing documents outlining the product specifications, the manufacturing constraints, and a market research file outlining the target user demographics. The brief stated that the user-friendliness of the product was of paramount importance %u2013 especially the set-up task as this was seen as a barrier to product sales. The market research outlined the user profile and the personas were created from this same user profile. The image boards were given to the control group covered 3 areas, Product environment, lifestyle and brand landscape. The items shown on the image boards were also mentioned in the personas so that the information provided to all teams were as uniform as possible. Point 4 - conclusion The conclusions of the study find that using personas is an effective tool and did produce more user-focused solutions. It also acknowledges that personas are only one of many design tools %u2013 and does not claim that it is the most effective method. As the saying goes%u2026there is more than one way to skin a cat, or to make a duck happy. :) Regards Frank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Posted from the new ixda.org http://www.ixda.org/discuss?post=42315 ________________________________________________________________ Welcome to the Interaction Design Association (IxDA)! To post to this list ....... [email protected] Unsubscribe ................ http://www.ixda.org/unsubscribe List Guidelines ............ http://www.ixda.org/guidelines List Help .................. http://www.ixda.org/help
