What you say makes sense. The problem is that often "real" and "meaningful" are not distinguished properly, and claims are made about group differences that are misleading based upon the fact of statistical significance.
Paul ----- Original Message ----- From: "Wallace E. Dixon, Jr." <[EMAIL PROTECTED]> To: "Teaching in the Psychological Sciences" <[EMAIL PROTECTED]> Sent: Monday, August 16, 2004 6:59 PM Subject: Re: teaching effect size > Well, I'd guess you'd say the glass is half empty then. In my research, > accounting for any individual differences in language acquisition is hard > enough, so I'll take every 1% variance accounted for I can get. I try to > convey this attitude when I teach effect sizes. My take on it is that when > one starts adding all the 1%s together, pretty soon one get some pretty > sweet variance accounted for. And in any case, science doesn't progress by > leaps and bounds, it progresses incrementally, over time, when all the > little 1%s add up. > > I might add that there is a huge push among funding agencies to fund > projects with large enough sample sizes to detect small effects; so I guess > it's not surprising that there are large N studies and small effect sizes. > But I'm not sure I would regard them as automatically lacking clinical > significance or meaningfulness; I'm too committed to science as a > transgenerational institution to give up on the little bits of variance > accounted for that's sprinkled throughout it. > > Wally Dixon > > > > > On 8/16/04 5:36 PM, "Paul Okami" <[EMAIL PROTECTED]> wrote: > > > I suspect that the opposite is far more often the case: Statistically > > significant results due only to the fact that large samples were used > > represent lack of clinical significance or meaningful group differences. > > > > Paul > > ----- Original Message ----- > > From: "Wallace Dixon" <[EMAIL PROTECTED]> > > To: "Teaching in the Psychological Sciences" <[EMAIL PROTECTED]> > > Sent: Monday, August 16, 2004 5:17 PM > > Subject: Re: teaching effect size > > > > > >> Maybe I was too sloppy in my usage. But what I *meant* to say still > >> applies. The larger the sample, the smaller the effect size that can be > >> detected to be statistically significant. So with extremely large > > samples, > >> extremely small effect sizes can be detected to be statistically > >> significant. But it is not automatically true that small effect sizes due > >> to large samples are not meaningful, they can be very meaningful. So, > >> people may laugh at you if you say "Hey, I accounted for 1% of the > >> variance!" But pragmatically, accounting for 1% of the variance may > > benefit > >> millions of peoples lives in America alone. > >> > >> wedj > >> > >> On 8/16/04 4:38 PM, "Paul Okami" <[EMAIL PROTECTED]> wrote: > >> > >>> Small effect sizes in large populations are not the same as statistical > >>> significance due to large samples. This is what I was referring to. > >>> > >>> Paul Okami > >>> ----- Original Message ----- > >>> From: "Wallace Dixon" <[EMAIL PROTECTED]> > >>> To: "Teaching in the Psychological Sciences" <[EMAIL PROTECTED]> > >>> Sent: Monday, August 16, 2004 12:56 PM > >>> Subject: Re: teaching effect size > >>> > >>> > >>>> But you have to be careful about this. As noted by Rosenthal, small > >>> effects > >>>> sizes in large populations do not mean unimportant. Effect sizes as > > small > >>>> as 1% can still translate into an influence on millions of people in > > the > >>> US, > >>>> as was the case in the aspirin study and the decreased rate of second > >>> heart > >>>> attacks. > >>>> > >>>> Wally Dixon > >>>> > >>>> > >>>> > >>>> > >>>> On 8/16/04 12:25 PM, "Paul Okami" <[EMAIL PROTECTED]> wrote: > >>>> > >>>>> I usually give an example of results that are statistically > > significant > >>>>> because of large samples, but have little meaning in the real world. > > In > >>>>> other words, they're "real" but "not important." > >>>>> > >>>>> That makes it easy to talk about a statistic that reflects magnitude > > of > >>>>> group differences. It's really very intuitive. (I don't discuss the > >>> math > >>>>> involved). > >>>>> > >>>>> Paul Okami > >>>>> ----- Original Message ----- > >>>>> From: "Hatcher, Joe" <[EMAIL PROTECTED]> > >>>>> To: "Teaching in the Psychological Sciences" > > <[EMAIL PROTECTED]> > >>>>> Sent: Monday, August 16, 2004 12:18 PM > >>>>> Subject: teaching effect size > >>>>> > >>>>> > >>>>>> Hi Tipsters, > >>>>>> > >>>>>> I apologize for the cross-posting. > >>>>>> > >>>>>> I teach the first semester of our two-semester Research Design and > >>>>>> Statistics class, my part of which goes through t-tests and requires > > a > >>>>>> completed project. When I first started teaching the course, "effect > >>>>> size" > >>>>>> wasn't one of the topics covered. In trying to update the class, I > >>> would > >>>>>> like to teach this concept, but I'm having some trouble, as it > > doesn't > >>>>> come > >>>>>> as naturally to me as do some of the other topics, and most of the > >>>>>> explanations of effect size seem very complicated. > >>>>>> Can someone point me to a method to teach effect size that is > >>>>> easily > >>>>>> understood by undergraduates? > >>>>>> > >>>>>> Joe Hatcher > >>>>>> Ripon College > >>>>>> Ripon, WI > >>>>>> [EMAIL PROTECTED] > >>>>>> > >>>>>> > >>>>>> > >>>>>> Joe W. Hatcher, Jr., Ph.D. > >>>>>> Psychology > >>>>>> Ripon College > >>>>>> Ripon, WI 54971 > >>>>>> USA > >>>>>> [EMAIL PROTECTED] > >>>>>> > >>>>>> --- > >>>>>> You are currently subscribed to tips as: [EMAIL PROTECTED] > >>>>>> To unsubscribe send a blank email to > >>>>> [EMAIL PROTECTED] > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> --- > >>>>> You are currently subscribed to tips as: [EMAIL PROTECTED] > >>>>> To unsubscribe send a blank email to > >>> [EMAIL PROTECTED] > >>>>> > >>>> > >>>> > >> > >>> ------------------------------------------------------------------------ - > > - > >>> -- > >>>> Wallace E. Dixon, Jr. | > >>>> Chair and Associate Professor | If children grew up according > > to > >>>> of Psychology | early indications, we should > > have > >>>> Department of Psychology | nothing but geniuses. > >>>> East Tennessee State University | -Johann Wolfgang von Goethe > >>>> Johnson City, TN 36714 | > >>>> (423) 439-6656 | > >> > >>> ------------------------------------------------------------------------ - > > - > >>> -- > >>>> > >>>> > >>>> > >>>> > >>>> --- > >>>> You are currently subscribed to tips as: [EMAIL PROTECTED] > >>>> To unsubscribe send a blank email to > >>> [EMAIL PROTECTED] > >>>> > >>> > >>> > >>> > >>> --- > >>> You are currently subscribed to tips as: [EMAIL PROTECTED] > >>> To unsubscribe send a blank email to > > [EMAIL PROTECTED] > >>> > >> > >> > >> ------------------------------------------------------------------------- - > > -- > >> Wallace E. Dixon, Jr. | > >> Chair and Associate Professor | "When I turned two I was really > >> of Psychology | anxious because I'd doubled my > >> Department of Psychology | age in a year." > >> East Tennessee State University | -Steven Wright > >> Johnson City, TN 36714 | > >> (423) 439-6656 | > >> ------------------------------------------------------------------------- - > > -- > >> > >> > >> --- > >> You are currently subscribed to tips as: [EMAIL PROTECTED] > >> To unsubscribe send a blank email to > > [EMAIL PROTECTED] > >> > > > > > > > > --- > > You are currently subscribed to tips as: [EMAIL PROTECTED] > > To unsubscribe send a blank email to [EMAIL PROTECTED] > > > > > --- > You are currently subscribed to tips as: [EMAIL PROTECTED] > To unsubscribe send a blank email to [EMAIL PROTECTED] --- You are currently subscribed to tips as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED]
