Nick, et al., I kept flirting with the idea of writing an article for the Journal of Irreproducible Results about the RTQ method of improving test scores for underachieving students. RTQ stands for Re-Test Quickly. The method is well proven. To try it: 1) Create a set of matched multiple choice tests on general relativity. 2) Give it to a group of 2nd graders. 2) Identify the lowest quartile of the class. 3) Immediately retest them with another version. 4) You will find that the students knowledge of general relativity has significantly increased. *Warning, do not try to apply this method with already well performing students.*
The New Yorker might merely be noticing that several scientists fail to heed the warning. On a more serious note (and the previous part was fairly serious already): Given that half the "major discoveries" promoted in psychology are assuredly garbage, how does this surprise you? Are you a "hard-science" snob, and only surprised because this is happening to physicists? There are a million reasons why an initial report of a phenomenon might overestimate the effect size. Some reasons are malicious (i.e., drug company funded studies as to the effectiveness of new drugs), others are benign (i.e. sampling error, unforeseen methodological shortcomings in initial tests, biased acceptance and promotion of "sexy" results). Whole academic industries arise over non-existent effects: Piaget's "A-non-B error", menstrual synchrony, and infant's "innate mathematical abilities." Once the discipline is formed, it is very hard to unform. So, if the NYer is being stupid, it is being stupid either for 1) not understanding what is going on, 2) not recognizing the legitimacy of what is going on, or 3) being selective in reporting by not noticing that some effects raise over time. I suspect a combination of all of those, with #3 being the most damning from a journalistic perspective. Eric P.S. It is also possible that the effect sizes are legitimately changing over time. Lets be honest, doesn't almost everything seem a little less important now than it used to? I mean, just a month or two ago backscatter technology and forced groping seemed like a big deal... and how many people's lives are currently being endangered this week by WikiLeakes... what about Obama's Hope and Change effect... or the way the Republicans would fix Washington... talk about a pervasive drop in effect size! On Sun, Dec 12, 2010 02:46 AM, "Nicholas Thompson" <[email protected]> wrote: > > > >All— > > > > > > > > > > > >Have others seen the article in the New Yorker on the “decline effect”, the alleged tendency for the effect sizes of well documented phenomena to decline with successive years of replication. I kept turning back to the front of the article to reassure myself that it was not one of the “Shouts and Murmurs” series. It is not. The passage that particularly caught my eye: > > > > > > > > > > > >Many scientific theories continue to be considered true even after failing numerous experimental tests. … [This] holds for any number of phenomena, from the disappearing benefits of second-generation antipsychotics to the weak coupling ration exhibited by decaying neutrons, which appears to have fallen by more than ten standard deviations between 1969 and 2001. [NY mag, 15 december 2010, p57] > > > > > > > > > > > >At least until recently, when the NY-er writes about science, they try very hard not to write anything stupid. > > > > > > > > > > > >What gives? > > > > > > > > > > > > > > > > > >Nicholas S. Thompson > > > > > >Emeritus Professor of Psychology and Biology > > > > > >Clark University > > > > > ><http://home.earthlink.net/%7Enickthompson/naturaldesigns/> > > > > > ><http://www.cusf.org/> > > > > > > > > > >
============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org
