-----Original Message-----
From: Stephen Black [mailto:[EMAIL PROTECTED]]
Sent: Friday, October 22, 1999 4:53 PM
To: TIPS
Subject: Re: the failure to replicate
> ----------
> >From: Michael Sylvester <[EMAIL PROTECTED]>
> >
> > what can be the various explanations why some studies have not
> > been replicated, or attempts at replication were made with inconsistent
> > results?
>
On Fri, 22 Oct 1999, John Serafin wrote:
>
> When two or more studies show different outcomes, there may be any number
of
> reasons <snip>
One reason I haven't seen anyone advance is that a study may not
replicate because it's not replicable. Because of the pressure to
report positive results, analysis is sometimes an exercise in
achieving significance no matter what. There may really be nothing
there. There are numerous examples, including, for one, the infamous
Mozart effect.
A possible new example is the recent paper by Maurer et al (1999).
They reported the startling finding that as little as one hour of
patterned visual stimulation after the birth of a baby with cataracts
improves vision, a result which received wide attention in the press.
Yet the paper is sprinkled with one-tailed tests, without a single
word of justification. I've complained to _Science_ in a
letter-to-the-editor (don't hold your breath, though).
##########
Let's review the bidding here. Replication is purely on the method side
rather than the results side. Although the question communicated and seems
to have been understood to refer to the results side, it _should_ have been
why there has been a failure to _confirm_ some earlier findings. From that
point, we can move on to whether or not the later study replicated, i.e.,
followed the method of the earlier one chapter and verse.
In science we build by replicating (with extensions) on the method side in
order to confirm or disconfirm the earlier findings of others. To say that
someone "failed to replicate" means that researcher number two didn't
duplicate the procedures of researcher number two. It says nothing about the
results. Perhaps I'm being old fashioned in insisting upon this traditional
distinction.
In fact, I'm sorry I opened my yap.
Al
Al L. Cone
Jamestown College <[EMAIL PROTECTED]>
North Dakota 701.252.3467 X 2604
http://www.jc.edu/users/faculty/cone
