@Jared, Don't get me wrong this is great first step at trying to establish if Personas work, or not.
Give me an example of *any* research that, in your mind, solves the > "inductive issue" and we can talk about it further. We can prove the negative "Some Turkeys don't get fed every day", or that Newton got some of his assumptions wrong, or "Not all swans are white" . And we can say that all your ancestors had children. The null hypothesis is, if personas don't make a differences, then the > control group (the folks w/o personas) will not produce distinguishably different results > from those that do. > As you show here by using a null hypothesis. So Frank Long has shown that "Persona are better than X" The issue is exactly what is X? [OT I believe one issue people have with Personas is that there is no 'null hypothesis' ] Scientific research studies like this are little building blocks. You > disassemble the problem into little problems, evaluate each problem, then reassemble them to build your > case. That works if the studies are replicable. If we don't know how to set up X, we can not replicate it. Sure, there are evaluator effects in heuristic evaluations. But that has to > do with *different* evaluators inspecting the *same* design. So if we agree that it becomes very important to build upon Frank's research, then "evaluator effect" becomes important as the study needs to be replicable. So if we re-ran the study would we get the same results. Then there is the other issues which Joshua Porter points out such as blinding. But I would agree some research is better than no research and puts a line in the sand for further studies. @frank One interesting point from your research is that the effect of the pictures have on the design teams. The teams with a photo seam to do far better, scoring nearly twice as much as the teams with an illustration. Was it the persona's that had the impact or the pictures of users. I wonder if just having random pictures of people handed out to the designers would have the same effect? Against a sample of photos of real users. Is it the pictures that give the designer the empathic tool or is it the narrative. What would the effect be of giving the designers access to the facebook/myspace pages, against using personas. The control group would just be given a picture of one user - Steve Jobs. :-) James http://blog.feralabs.com 2009/5/29 Joshua Porter <[email protected]> > > On May 29, 2009, at 9:28 AM, Jared Spool wrote: > > >> On May 29, 2009, at 6:36 AM, James Page wrote: >> >> I think the issue with using heuristic evaluations is the well known >>> issue >>> with the evaluator effect. >>> >> >> Wow, James. You are *so* missing the point here. >> >> Sure, there are evaluator effects in heuristic evaluations. But that has >> to do with *different* evaluators inspecting the *same* design. >> > > > If I understand James correctly, he is suggesting that if the evaluators > knew which designers used personas then they would be biased from judging > objectively which designs were more user-centered. That is, the so-called > differences uncovered by the heuristic evaluation might only be biases > introduced by the evaluators. > > If this is the case (and it's not clear from the writeup exactly what was > done), then the results of the study are biased. > > Frank, could you shed some more light on this? Did the evaluators know > which designs were done using personas and which were not? > > > Cheers, > > Josh > > > > > Joshua Porter, Founder > Bokardo Design > Interface design & strategy for social web applications > phone: 508-954-1896 > http://bokardo.com > [email protected] > twitter: bokardo > > > > > ________________________________________________________________ > Welcome to the Interaction Design Association (IxDA)! > To post to this list ....... [email protected] > Unsubscribe ................ http://www.ixda.org/unsubscribe > List Guidelines ............ http://www.ixda.org/guidelines > List Help .................. http://www.ixda.org/help > ________________________________________________________________ Welcome to the Interaction Design Association (IxDA)! To post to this list ....... [email protected] Unsubscribe ................ http://www.ixda.org/unsubscribe List Guidelines ............ http://www.ixda.org/guidelines List Help .................. http://www.ixda.org/help
