> Oh, and debate is for women. (Wait, no, I was thinking of something else. > Nevermind,) >
Just on the off chance anyone thinks that continuing this meme would be funny It's not. (and I just got done explaining elsewhere why the first time it came up it didn't bother me so it's not that I don't have a sense of humour about these things) ________________________ Leisa Reichelt Disambiguity.com Contextual Research, User Centred Design & Social Design [email protected] +44 778 071 2129 2009/5/29 Jared Spool <[email protected]> > > On May 28, 2009, at 5:55 PM, Robert Hoekman Jr wrote: > > Oh, man. This is hardly solid research. Now you're just begging for a >> debate. ;) >> > > Trust me. Compared to many of the hundreds of research papers I review each > year, this one is pretty solid. > > And maybe, just maybe, I was begging for debate. > > Oh, and debate is for women. (Wait, no, I was thinking of something else. > Nevermind,) > > 1. ... >> >> 2. ... >> >> 3. ... >> > > I actually think Frank did a nice job of defending his study, so I won't do > that here. > > The only thing this study shows is that 2 out of the 3 teams created a >> more usable design as measured against heuristics (this assumes, of course >> (and it's a big assumption), that the evaluators did good evaluations), and >> that they happened to be the same groups that used personas in the project. >> At the absolute best, this is a loose correlation. It's absolutely not proof >> of genuine causation. I could have fared as well as any of them without >> personas and without a team. >> >> 4. Even if you throw out arguments #2 and #3 above, #1 still makes it all >> a moot point. >> > > No, no, no. You're looking at this wrong. > > Scientific research studies like this are little building blocks. You > disassemble the problem into little problems, evaluate each problem, then > reassemble them to build your case. > > While the bigger problem is, "Do teams that employ personas produce > effective designs for their audiences?" that, as you've correctly pointed > out, is hard to prove in a study. So, you break it down. > > What this study does (in a very sweet, nice way) tackle one small aspect of > the problem: if you take a group of designers, break them up, give some > personas and others not, do you see different results. The null hypothesis > is, if personas don't make a differences, then the control group (the folks > w/o personas) will not produce distinguishably different results from those > that do. > > This study contradicts the null hypothesis, because the teams with personas > produced different results based on the criteria (heuristic evaluation). > > Now, as you rightly point out, we can question the criteria (and should!). > That would be a different study. What criteria would you like Frank or other > researchers to measure against? That's the next building block. > > Also, we could perform the same study with different user research tools. > What tools would you like to see studied? > > Of course, this study can't stand alone. Good research, like this, needs to > be duplicated elsewhere before you can really stand behind it. Other > researchers should try to replicate the experiment to see if they get > similar results. Then, and only then, will we empirically know that the > results were great. Frank did a good job of explaining to another team how > to duplicate the experiment. > > You have to take this type of research for what it is and not expect a > single study to prove everything. I found it fascinating because it > duplicated what we've seen in our research in a controlled setting: teams > that use personas have a different dynamic than teams that don't -- a > dynamic that, in my opinion, leads to better design. > > All that said, I still love you Jared. :) >> > > Oh, Robert, you certainly know how to make a guy swoon. Hugs & Kisses. > > Jared > > > ________________________________________________________________ > Welcome to the Interaction Design Association (IxDA)! > To post to this list ....... [email protected] > Unsubscribe ................ http://www.ixda.org/unsubscribe > List Guidelines ............ http://www.ixda.org/guidelines > List Help .................. http://www.ixda.org/help > ________________________________________________________________ Welcome to the Interaction Design Association (IxDA)! To post to this list ....... [email protected] Unsubscribe ................ http://www.ixda.org/unsubscribe List Guidelines ............ http://www.ixda.org/guidelines List Help .................. http://www.ixda.org/help
