Re: [tips] The Cult of Statistical Significance

2010-10-15 Thread Allen Esterson 
For greater clarity, I should have added to my reference to "Einstein's 
epoch-making 1905 special relativity paper, which takes as a 
fundamental premise the constancy of the speed of light in vacuo": 
"regardless of the motion of the observer".

Allen Esterson
Former lecturer, Science Department
Southwark College, London
allenester...@compuserve.com
http://www.esterson.org

--------------
Re:[tips] The Cult of Statistical Significance

Allen Esterson
Fri, 15 Oct 2010 00:15:04 -0700

It's not psychology, but since it has come up here, I'll have to give
my response here!

Reference Richard Hake's important post "The Cult of Statistical
Significance", which contains a link to AERA-L listserv:
http://listserv.aera.net/scripts/wa.exe?A2=ind1010&L=AERA-L&F=&S=&X=25C042478B751B1DD7&Y=rrhake%40earthlink.net&P=2053

Quote: "Math-Teach's Domenico Rosa (2010), in his post 'The Cult of
Statistical Significance - A Book Review' has called attention to the
review of Ziliak & McCloskey (2008) by Olle Häggström (2010).

"According to Häggström: […]

"Carver (1993) subjected the Michelson and Morley (1887) data to a
simple analysis of variance (ANOVA) AND FOUND STATISTICAL SIGNIFICANCE
ASSOCIATED WITH THE DIRECTION THE LIGHT WAS TRAVELING (p < 0.001 )! He
writes, 'It is interesting to speculate how the course of history might
have changed if Michelson and Morley had been trained to use this
CORRUPT FORM OF THE SCIENTIFIC METHOD, that is, testing the null
hypothesis first. They might have concluded that there was evidence of
SIGNIFICANT differences in the speed of light associated with its
direction and that therefore there was evidence for the luminiferous
ether...' […]"


Carver suggests that "the course of history might have been changed…"
It is arguable that there are two misconceptions here. The first is
that definite conclusions would have been drawn from the results of
*one* experiment. But (hopefully!) science doesn't work like that:

"Other versions of the experiment were carried out with increasing
sophistication. Kennedy and Illingworth both modified the mirrors to
include a half-wave 'step', eliminating the possibility of some sort of
standing wave pattern within the apparatus. Illingworth could detect
changes on the order of 1/300th of a fringe, Kennedy up to 1/1500th.
Miller later built a non-magnetic device to eliminate magnetostriction,
while Michelson built one of non-expanding invar to eliminate any
remaining thermal effects. Others from around the world increased
accuracy, eliminated possible side effects, or both."
http://en.wikipedia.org/wiki/Michelson%E2%80%93Morley_experiment

Second, the allusion to the course of [scientific] history presumably
relates to Einstein's epoch-making 1905 special relativity paper, which
takes as a fundamental premise the constancy of the speed of light in
vacuo. But some eminent historians of physics have convincingly argued
that the Michelson-Morley experiment (1887) did not play a significant
role in Einstein's thought processes during the gestation period and
final development of his theory. See J. Stachel, "How Did Einstein
Discover Special Relativity?","Einstein and Ether Drift Experiments",
"Einstein and Michelson" and "Einstein on the Theory of Relativity",
reprinted in *Einstein From 'B' to 'Z'* (2002), pp. 157-214; G. Holton,
"Einstein, Michelson, and the 'Crucial' Experiment'," chapter 8, *The
Thematic Origins of Scientific Thought: Kepler to Einstein* (1988
edition), pp. 279-370; A. Pais, *Subtle is the Lord: The Science and
the Life of Albert Einstein* (1982), pp. 111-137.

Quote: "[The preceding material] shows that the principal argument
which ultimately led [Einstein] to the special theory was not so much
the need to resolve the conflict between the Michelson-Morley result
and the version of aether theory prevalent in the late nineteenth
century but rather, *independent of the Michelson-Morley experiment*,
the rejection of this nineteenth century edifice as inherently
unconvincing and artificial" (Pais, 1982, p. 117).

Quote: "Yet, the experimenticist fallacy of imposing a logical sequence
must be resisted... [...] The basic achievement of Einstein's theory...
was that at the cost of sacrificing all these [traditional
conceptions], it gave us a new unity in the understanding of nature"
(Holton, 1988, p. 347).

Allen Esterson
Former lecturer, Science Department
Southwark College, London
allenester...@compuserve.com
http://www.esterson.org


---
You are currently subscribed to tips as: arch...@jab.org.
To unsubscribe click here: 
http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=5704
or send a blank email to 
leave-5704-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu

Re:[tips] The Cult of Statistical Significance

2010-10-15 Thread Allen Esterson 
It's not psychology, but since it has come up here, I'll have to give 
my response here!

Reference Richard Hake's important post "The Cult of Statistical 
Significance", which contains a link to AERA-L listserv:
http://listserv.aera.net/scripts/wa.exe?A2=ind1010&L=AERA-L&F=&S=&X=25C042478B751B1DD7&Y=rrhake%40earthlink.net&P=2053

Quote: "Math-Teach's Domenico Rosa (2010), in his post 'The Cult of 
Statistical Significance - A Book Review' has called attention to the 
review of Ziliak & McCloskey (2008) by Olle Häggström (2010).

"According to Häggström: […]

"Carver (1993) subjected the Michelson and Morley (1887) data to a 
simple analysis of variance (ANOVA) AND FOUND STATISTICAL SIGNIFICANCE 
ASSOCIATED WITH THE DIRECTION THE LIGHT WAS TRAVELING (p < 0.001 )! He 
writes, 'It is interesting to speculate how the course of history might 
have changed if Michelson and Morley had been trained to use this 
CORRUPT FORM OF THE SCIENTIFIC METHOD, that is, testing the null 
hypothesis first. They might have concluded that there was evidence of 
SIGNIFICANT differences in the speed of light associated with its 
direction and that therefore there was evidence for the luminiferous 
ether...' […]"


Carver suggests that "the course of history might have been changed…" 
It is arguable that there are two misconceptions here. The first is 
that definite conclusions would have been drawn from the results of 
*one* experiment. But (hopefully!) science doesn't work like that:

"Other versions of the experiment were carried out with increasing 
sophistication. Kennedy and Illingworth both modified the mirrors to 
include a half-wave 'step', eliminating the possibility of some sort of 
standing wave pattern within the apparatus. Illingworth could detect 
changes on the order of 1/300th of a fringe, Kennedy up to 1/1500th. 
Miller later built a non-magnetic device to eliminate magnetostriction, 
while Michelson built one of non-expanding invar to eliminate any 
remaining thermal effects. Others from around the world increased 
accuracy, eliminated possible side effects, or both."
http://en.wikipedia.org/wiki/Michelson%E2%80%93Morley_experiment

Second, the allusion to the course of [scientific] history presumably 
relates to Einstein's epoch-making 1905 special relativity paper, which 
takes as a fundamental premise the constancy of the speed of light in 
vacuo. But some eminent historians of physics have convincingly argued 
that the Michelson-Morley experiment (1887) did not play a significant 
role in Einstein's thought processes during the gestation period and 
final development of his theory. See J. Stachel, "How Did Einstein 
Discover Special Relativity?","Einstein and Ether Drift Experiments", 
"Einstein and Michelson" and "Einstein on the Theory of Relativity", 
reprinted in *Einstein From 'B' to 'Z'* (2002), pp. 157-214; G. Holton, 
"Einstein, Michelson, and the 'Crucial' Experiment'," chapter 8, *The 
Thematic Origins of Scientific Thought: Kepler to Einstein* (1988 
edition), pp. 279-370; A. Pais, *Subtle is the Lord: The Science and 
the Life of Albert Einstein* (1982), pp. 111-137.

Quote: "[The preceding material] shows that the principal argument 
which ultimately led [Einstein] to the special theory was not so much 
the need to resolve the conflict between the Michelson-Morley result 
and the version of aether theory prevalent in the late nineteenth 
century but rather, *independent of the Michelson-Morley experiment*, 
the rejection of this nineteenth century edifice as inherently 
unconvincing and artificial" (Pais, 1982, p. 117).

Quote: "Yet, the experimenticist fallacy of imposing a logical sequence 
must be resisted... [...] The basic achievement of Einstein's theory... 
was that at the cost of sacrificing all these [traditional 
conceptions], it gave us a new unity in the understanding of nature" 
(Holton, 1988, p. 347).

Allen Esterson
Former lecturer, Science Department
Southwark College, London
allenester...@compuserve.com
http://www.esterson.org



---
You are currently subscribed to tips as: arch...@jab.org.
To unsubscribe click here: 
http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=5703
or send a blank email to 
leave-5703-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu

[tips] The Cult of Statistical Significance

2010-10-14 Thread Richard Hake 
Some subscribers to TIPS and TeachEdPsych might 
be interested in a recent post "The Cult of 
Statistical Significance" [Hake (2010)].  The 
abstract reads:

**
ABSTRACT: Math-Teach's Domenico Rosa has called 
attention to the review of Ziliak & McCloskey 
(2008) by Olle Häggström (2010). According to 
Häggström, Ziliak & McCloskey's major point is 
that "many researchers are so obsessed with 
statistical significance that they neglect to ask 
themselves whether the detected discrepancies are 
large enough to be of any subject-matter 
significance."

Consistent with that outlook, in "Lessons from 
the Physics Education Reform Effort" [Hake 
(2002)] I cited the position of many 
psychologists and biologists that the "effect 
size" is a preferred alternative (or at least 
addition) to the usually inappropriate t-tests 
and p values associated with Null Hypothesis 
Statistical Significance Testing (NHSST).

Nevertheless, many educational researchers (even 
some physicists) still utilize *only* NHSST to 
gauge the significance of their research results.
*

To access the complete 14 kB post please click on .


Richard Hake, Emeritus Professor of Physics, Indiana University
Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands
President, PEdants for Definitive Academic References which Recognize the
   Invention of the Internet (PEDARRII)






"After 4 decades of severe criticism, the ritual 
of null hypothesis significance testing - 
mechanical dichotomous decisions around a sacred 
0.05 criterion - still persists. This article 
reviews the problems with this practice, 
including its near-universal misinterpretation of 
p as the probability that Ho . . . .[[the null 
hypothesis]]. . . .  is false, the 
misinterpretation that its complement is the 
probability of successful replication, and the 
mistaken assumption that if one rejects Ho one 
thereby affirms the theory that led to the test. 
Exploratory data analysis and the use of graphic 
methods, a steady improvement in and a movement 
toward standardization in measurement, and 
emphasis on effect sizes using confidence 
intervals, ands the informed use of available 
statistical methods is suggested. FOR 
GENERALIZATION, PSYCHOLOGISTS MUST FINALLY RELY, 
AS HAS BEEN DONE IN THE OLDER SCIENCES, ON 
REPLICATION."  [My CAPS.]
   -Jacob Cohen (1994) in "The earth is round (p < .05)"


REFERENCES [All URL's accessed on 10 October 
2010; some URL's shortened by .]

Cohen, J. 1994. "The earth is round (p < .05)." 
American Psychologist 49: 997-1003; online as a 
1.2 MB pdf at , thanks to 
Christopher Green .

Hake, R.R. 2010. "The Cult of Statistical 
Significance," online on the OPEN AERA-L archives 
at . Post of 10 Oct 2010 
19:34:16-0700 to AERA-L, Math-Teach, & Net-Gold. 
The abstract and link to the complete post are 
being transmitted to various discussion lists and 
are also online on my blog "Hake'sEdStuff" at 
 with a provision for 
comments.
---
You are currently subscribed to tips as: arch...@jab.org.
To unsubscribe click here: 
http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=5695
or send a blank email to 
leave-5695-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu