Hi, Jay.
You are of course correct, in that the data from Pingu's (aka DN's)
experiment can be analyzed by a one-way ANOVA (aka a completely
randomized design). The thing is, there MAY be noticeable effects on
performance time due to practice -- that is, due to the "repetitions"
factor. Should that be the case, the completely randomized design may
be less sensitive to differences among the eight conditions (which is
what DN really wants to know about, or so I gather) than the randomized-
block design. (It would be entirely appropriate to run both analyses,
and then compute their relative efficiency: see, e.g., Wm.C. Guenther,
"Analysis of Variance", Prentice-Hall, 1964, section 3-11. But I rather
thought this might be more technical than DN actually wanted.)
Whether the putative "less sensitive" is actually true, and whether the
effects of the conditions are subtle enough that this might be a
problem, depend on the data (or the state of nature the data are thought
to reflect, I suppose).
As I wrote earlier, any further comment would entail speculation, in the
absence of more concrete information about the actual enterprise. So
let's speculate a bit. (You can hit the <DELETE> key now, if you like.)
The experiment involves several different conditions under which a task
(nature unspecified, so far) is performed. The measure of interest is
response time -- which may mean "time taken to complete the task",
presumably measured from start to finish; or it may mean "the time
taken to get started" on the task, presumably measured from the time the
task is presented to the person until the time the person responds to
that stimulus; or, imaginably, something else entirely.
That DN is interested in seeing "whether any of the conditions lead to
significantly faster response times than the others" leads me to suppose
that faster response times are desirable, and that he has chosen 10 as a
reasonable number of repetitions suggests (to me, anyway) that DN thinks
that by the time one has done this task ten times, one's speed is about
as fast as it's likely to get. I suspect DN envisions a <response-time>
vs. <repetition-number> graph that declines toward a horizontal
asymptote, which is why I wrote earlier that the decline was probably
not linear.
(I DID say I was speculating! :)
Hence my advice to carry out a two-way ANOVA. In SPSS, or any other of
the usual packages, it's as easy to do a two-way as a one-way; and the
two-way results will include the one-way results, and will also have
partitioned the between-conditions SS (with 72 degrees of freedom) into
two parts with respectively 9 (between repetitions) and 63 ("error")
d.f. If the error MS with 63 d.f. is notably smaller than the between-
groups MS with 72 d.f., the two-way analysis will be more sensitive to
differences among the eight conditions; if it is larger, the explicit
acknowledgement of the "repetitions" (or "practice") factor was not
helpful in detecting any such differences, and one might as well report
the one-way ANOVA you recommend. But one can't tell without looking.
On Thu, 10 Jul 2003, Jay Warner wrote in part:
> Dare I leap in where angels fear to tread?
Do they? I hadn't noticed. <grin>
> Certainly D. Burrell [sic] has beat this question something fierce.
And here I thought I was just being obvious...
> ... Suppose I say that the 8 conditions are categorically different
> - some aren't higher on any scale, they are simply different.
Yep. That's all ANYone has said, so far.
> Then I assert loudly that the 10 repetitions are performed all at
> the same time, with one 'set-up' of the specific condition.
Feel free. But be aware that you are asserting a falsehood. If these
are, as claimed, performance tasks, and all done by one person, they
cannot possibly have been performed all at the same time.
(I had been seriously contemplating elaborating on this with a task of
my choosing [since I'm speculating anyway], namely the playing on a pipe
organ of J.S. Bach's Toccata in d minor, on which one could pretty well
guarantee that performance on the 10th repetition would be superior to
performance on the first; ... but let that pass.)
> If I see no trends (or ignore the ones I see) in the ... repetitions,
> I can ... set up a One-Way Analysis of Variance. ...
Yes. But a two-way ANOVA will provide the one-way information, while
showing whether there ARE trends (or other systematic variation) in the
repetitions that would reduce the sensitivity of the one-way design.
< snip >
> then there is the intriguing possibility that your 8 conditions are
> not simply "8 brands of automobile." Instead, you just might be
> taking on a designed experiment, with three factors. This could be
> interesting!
Yes, "8" IS an interesting number, isn't it? A 2x2x2 design with
repeated measures on the 10-level practice effect would not be at all
unreasonable. It's almost an archetypal design in psychology...
> If you now have 8 conditions, all related but different, your 10
> reps could be seen as a nice way to reduce variation in the
> individual measurements, but you will back it out later in the
> analysis with an estimate.
Another gloss on this is possible. If indeed the "practice" effect
looks like the response times do indeed approach an asymptote, one might
do well to take the last several repetitions, where (if this is indeed
the case) the response times do not differ much from each other, as the
measures for each condition. Variability within conditions might very
well be noticeably greater in the early repetitions, and contribute
visibly to the error variance, so that subtler differences between
conditions might be detectable with the later measurements but not with
the earlier, and perhaps not with all measurements included in the
analysis.
> Could you do say 2 reps, change to a different condition, then do 2
> more, and keep this up until you have 10 sets of measurements
> altogether? Then you could estimate variation due to each
> measurement, and variation due to changing the specific donations of
> a 'condition.'
Definitely preferable. Even better with 1 rep; although the nature of
the task and the repetitions may be such that the repetitions have to be
presented in "massed" rather than "distributed" form. If they can be
distributed, it would also be preferable to randomize the order in which
the eight conditions are encountered.
The impression I had was that data had already been collected, and it is
not clear how handily it might be replicated. In any case, it would
doubtless be salutary to analyze the present data, with an eye toward
detecting any of the possible threats to validity that both of us have
mentioned.
Ciao! -- Don.
-----------------------------------------------------------------------
Donald F. Burrill [EMAIL PROTECTED]
56 Sebbins Pond Drive, Bedford, NH 03110 (603) 626-0816
.
.
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at:
. http://jse.stat.ncsu.edu/ .
=================================================================
