Presumably in response to my writing this: > My impression is that this is frequently not the case in psychology, > with results of studies sometimes being widely cited regardless of > whether they have been replicated.
Jim Clark provided a justified corrective: >Is replication that uncommon in psychology? Just a couple of observations >1. the recently cited paper noting that effects get weaker in >subsequent studies would require replications even to conduct the analysis >2. meta-analysis requires multiple replications of some effect (e.g., > gender differences); otherwise again not possible to do a meta-analysis >3. I can think of many, many areas where there are repeated demonstrations >of certain effects (e.g., serial position curve, DRM false memory effect, >caregiver-infant attachment, ...). I agree with virtually everything Jim wrote [see below]. I would just add two comments. I should have made clear that what I had in mind was the limited field of social psychology, where results of studies (typically based on questionnaires) are on occasion publicised and widely accepted without regard to replication. But what I also had in mind was an occasional experimental claim in psychological science that is widely publicised as proven regardless of replication. For instance, the claim by Anderson and Green (2001) to have experimentally validated Freudian repression mechanisms: http://www.nature.com/nature/journal/v410/n6826/abs/410366a0.html A University of Oregon press release (no longer available online) reported: "Our findings are consistent with Freud's notions of suppression and repression, but go a long ways towards demystifying the process," says the paper's co-author Michael Anderson, an assistant professor of psychology at the University of Oregon. "Our work allows Freud's idea to be understood in terms of widely accepted mechanisms of cognitive control that apply in a broader range of circumstances." As a result of the press release this paper was widely heralded in the press as vindication of Freudian repression. Leaving aside the criticism that the kind of innocuous memory suppression claimed in the paper bore little relation to Freudian repression, passing virtually unnoticed was this report of failure to replicate the results: Bulevich et al (2006) Anderson and Green (2001) had subjects learn paired associates and then selectively suppress responses to some of them. They reported a decrease in final cued recall for responses that subjects had been instructed not to think of and explained their data as resulting from cognitive suppression, a laboratory analog of repression. We report three experiments designed to replicate the suppression/repression results. […] None of our experiments showed reliable suppression effects with either the same or independent-probe tests. Suppression is apparently not a robust experimental phenomenon in the think/no-think paradigm. http://www.psych.wustl.edu/coglab/publications/bulevich2006.pdf Allen Esterson Former lecturer, Science Department Southwark College, London [email protected] http://www.esterson.org -------------------------------------------- From: Jim Clark <[email protected]> Subject: Re: Stapel's faking of social psychology data Date: Sat, 05 Nov 2011 18:04:31 -0500 Hi Is replication that uncommon in psychology? Just a couple of observations 1. the recently cited paper noting that effects get weaker in subsequent studies would require replications even to conduct the analysis 2. meta-analysis requires multiple replications of some effect (e.g., gender differences); otherwise again not possible to do a meta-analysis 3. I can think of many, many areas where there are repeated demonstrations of certain effects (e.g., serial position curve, DRM false memory effect, caregiver-infant attachment, ...). No matter what the domain, natural science or psychology, there will always be some lag between the first report of some phenomenon and replications by other people. The problem is that initial reports may get more publicity and credence than they deserve. Indeed, researchers (via their public relations people in the administration) are probably culpable in a lot of the hype that goes on nowadays. Ironically, perhaps the somewhat crude efforts to encourage funding for research and science (i.e., publicity of "interesting" findings) might have the unintended consequence of further undermining people's (already?) weak acceptance of science. People are told one thing one week only to learn later that the effect does not hold up, perhaps even being contradicted in later studies, or was fraudulent. Probably not the best image of science to be projecting to the public? Take care Jim James M. Clark Professor of Psychology 204-786-9757 204-774-4134 Fax [email protected] --- You are currently subscribed to tips as: [email protected]. To unsubscribe click here: http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=13935 or send a blank email to leave-13935-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
