Dear Tipsters,
Some comments on how I teach this to undergraduates:
1. I think that the term statistical significance is fine as long we explain it
carefully After all, "significance" is being associated with "statistical"
which only means that the deviation from null is unlikely to occur by chance.
2. The issue of meaningfulness is separate from statistical significance and
can be split into two parts: effect size and clinical significance. I spend
time clearly separating statistical significance and effect size, giving
examples of small effects that are significant and large effects that are not.
3. The idea of just reporting confidence intervals and not p is interesting but
we still have to choose a % for the confidence interval.
For the students in my methods class, I work almost exclusively with 1. and 2.
From time to time a student asks about reporting the confidence interval I
usually make sure that they understand the first two and then give them the
choice if they wish to adopt 3.
Sincerely,
Stuart
_____________________________________________________
Sent via Web Access
"Floreat Labore"
"Recti cultus pectora roborant"
Stuart J. McKelvie, Ph.D., Phone: 819 822 9600 x 2402
Department of Psychology, Fax: 819 822 9661
Bishop's University,
2600 rue College,
Sherbrooke,
Québec J1M 1Z7,
Canada.
E-mail: [email protected] (or [email protected])
Bishop's University Psychology Department Web Page:
http://www.ubishops.ca/ccc/div/soc/psy
" Floreat Labore"
_______________________________________________________
________________________________________
From: Wuensch, Karl L [[email protected]]
Sent: 21 March 2010 13:08
To: Teaching in the Psychological Sciences (TIPS)
Subject: RE: [tips] Odds are, it's wrong
I'm with you, Scott. I find "statistically reliable" every bit as
troublesome as "statistically significant." Perhaps we should just stop
reporting p values and report instead a confidence interval for the effect size
estimator (standardized or not).
Cheers,
Karl W.
________________________________
From: Lilienfeld, Scott O [mailto:[email protected]]
Sent: Sunday, March 21, 2010 12:26 PM
To: Teaching in the Psychological Sciences (TIPS)
Subject: RE: [tips] Odds are, it's wrong
Don - Interesting, because my take is different…I always cross out the term
“reliable” when it’s used in lieu of “significant,” because I fear that the
former term implies that the finding is likely to be consistent or replicable.
In fact, in part because of issues of statistical power, many or most findings
in the social sciences don’t replicate:
See, e.g., Sohn, D. (1998). Statistical significance and replicability: Why the
former does not presage the latter. Theory and Psychology, 8, 291-311.
So I fear that the term “reliable” will lead some students and readers to
conclude that a statistically significant result is likely to be reliable in
the psychometric sense (viz., consistent). But perhaps I’m in a minority on
this score, and I’ll be curious to hear from other TIPSTERs.
….Scott
Scott O. Lilienfeld, Ph.D.
Professor
Editor, Scientific Review of Mental Health Practice
Department of Psychology, Room 473 Psychology and Interdisciplinary Sciences
(PAIS)
Emory University
36 Eagle Row
Atlanta, Georgia 30322
[email protected]
(404) 727-1125
Psychology Today Blog:
http://www.psychologytoday.com/blog/the-skeptical-psychologist
50 Great Myths of Popular Psychology:
http://www.wiley.com/WileyCDA/WileyTitle/productCd-140513111X.html
Scientific American Mind: Facts and Fictions in Mental Health Column:
http://www.scientificamerican.com/sciammind/
The Master in the Art of Living makes little distinction between his work and
his play,
his labor and his leisure, his mind and his body, his education and his
recreation,
his love and his intellectual passions. He hardly knows which is which.
He simply pursues his vision of excellence in whatever he does,
leaving others to decide whether he is working or playing.
To him – he is always doing both.
- Zen Buddhist text
(slightly modified)
From: don allen [mailto:[email protected]]
Sent: Sunday, March 21, 2010 12:14 PM
To: Teaching in the Psychological Sciences (TIPS)
Subject: Re: [tips] Odds are, it's wrong
Hi Stephen-
Thanks for the link. It's a pretty good summary of some of the "dirty little
secrets" of statistical analysis. However I was surprized at the lack of
mention of effect size as an important tool for dealing with the "significant
equals important" isssue. Re this point; I always insited that students use the
term "statistically reliable" rather than "statistically significant". While a
small point I think that it helps people understand what their findings
actually mean.
-Don.
----- Original Message -----
From: [email protected]
Date: Sunday, March 21, 2010 8:24 am
Subject: [tips] Odds are, it's wrong
To: "Teaching in the Psychological Sciences (TIPS)" <[email protected]>
> Alerted by a colleague, I recommend an instructive if
> depressing essay on the problematic use of statistics in science.
>
> http://www.sciencenews.org/view/feature/id/57091/title/Odds_ar
> e,_its_w or http://tinyurl.com/yh7sk7r
>
> Teaser:
>
> "Supposedly, the proper use of statistics makes relying on
> scientific results a safe bet. But in practice, widespread
> misuse
> of statistical methods makes science more like a crapshoot."
>
> Stephen
> --------------------------------------------
> Stephen L. Black,
> Ph.D.
> Professor of Psychology, Emeritus
> Bishop's
> University
> e-mail: sblack at ubishops.ca
> 2600 College St.
> Sherbrooke QC J1M 1Z7
> Canada
> -----------------------------------------------------------------
> ------
>
> ---
> You are currently subscribed to tips as: [email protected].
> To unsubscribe click here:
> http://fsulist.frostburg.edu/u?id=13157.966b795bc7f3ccb35e3da08aebe98f18&n=T&l=tips&o=1419or
> send a blank email to
> leave-1419-13157.966b795bc7f3ccb35e3da08aebe98...@fsulist.frostburg.edu
>
---
You are currently subscribed to tips as:
[email protected]<mailto:[email protected]>.
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13509.d0999cebc8f4ed4eb54d5317367e9b2f&n=T&l=tips&o=1425
(It may be necessary to cut and paste the above URL if the line is broken)
or send a blank email to
leave-1425-13509.d0999cebc8f4ed4eb54d5317367e9...@fsulist.frostburg.edu<mailto:leave-1425-13509.d0999cebc8f4ed4eb54d5317367e9...@fsulist.frostburg.edu>
________________________________
This e-mail message (including any attachments) is for the sole use of
the intended recipient(s) and may contain confidential and privileged
information. If the reader of this message is not the intended
recipient, you are hereby notified that any dissemination, distribution
or copying of this message (including any attachments) is strictly
prohibited.
If you have received this message in error, please contact
the sender by reply e-mail message and destroy all copies of the
original message (including attachments).
---
You are currently subscribed to tips as:
[email protected]<mailto:[email protected]>.
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13060.c78b93d4d09ef6235e9d494b3534420e&n=T&l=tips&o=1426
(It may be necessary to cut and paste the above URL if the line is broken)
or send a blank email to
leave-1426-13060.c78b93d4d09ef6235e9d494b35344...@fsulist.frostburg.edu<mailto:leave-1426-13060.c78b93d4d09ef6235e9d494b35344...@fsulist.frostburg.edu>
---
You are currently subscribed to tips as:
[email protected]<mailto:[email protected]>.
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13510.2cc18398df2e6692fffc29a610cb72e3&n=T&l=tips&o=1429
(It may be necessary to cut and paste the above URL if the line is broken)
or send a blank email to
leave-1429-13510.2cc18398df2e6692fffc29a610cb7...@fsulist.frostburg.edu<mailto:leave-1429-13510.2cc18398df2e6692fffc29a610cb7...@fsulist.frostburg.edu>
---
You are currently subscribed to tips as: [email protected].
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=1432
or send a blank email to
leave-1432-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
