On Thu, May 7, 2009 at 11:58 PM, Brian <brian.min...@colorado.edu> wrote: > Quite frankly the advice that you should only use five subjects makes no > sense. The appeal to Nielsen's authority is not going to work on me or > anyone else who understands why the scientific method exists. It's > unscientific thinking and it's going cause to you waste money. You're going > to draw conclusions based on results that simply aren't valid, and you won't > know it until the study is over and you didn't make progress. > > Careful analysis of site data could allow you to draw some conclusions. I'm > curious how you're planning to go about that. Dependent/independent > variables? >
An exercise in statistical thinking: when everyone or almost everyone cites problem X, how many people does it take to reach statistical significance that X is a problem worth addressing? Even if the results are a statistical fluke and in reality only 20% of new users run into trouble with problem X, that's still a problem worth addressing. The fact that so many of the 15 people had the same problems, and those problems also align with common sense, is a strong indication that the study has found some things worth fixing. There is more than one way to come to reliable conclusions. Any time I see someone invoking "the [singular] scientific method", as if there is only one and it is set in stone and universally agreed upon by all rational people, I have trouble taking them seriously. See [[Talk:Scientific method]]. -Sage (User:Ragesoss) _______________________________________________ foundation-l mailing list foundation-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l