Hi Stephen:
What was the N for the groups?
One justification for using the covariates might be that the N
was very low and that variability within the groups needed
further reduction.
But in that case wouldn't you expect to see an explanation of why
the covariates were chosen and the uniform use of those
covariates across the regressions.
I agree with Don Allen. It looks like data mining until the
magic p=.05 barrier was broken.
Ken
[EMAIL PROTECTED] wrote:
I'm pondering a study which purports to show that a specific early
intervention works in improving school performance and keeping kids from
crime. The design is good, although the results, while positive, are
weak.
But there does appear to be some possibility of fiddle in the way the
results are analyzed. A group of high-risk children were randomly
assigned to intervention and control groups. 15 years after treatment,
two objective measures of success were obtained: high school graduation,
and criminal record.
I'd have gone with a simple independent test of proportions for each
measure. When I did, high school graduation was significant, but criminal
record was not (p=.09).They didn't do this. They used logistic regression
in each case, for graduation controlling both for parental occupation and
disruptiveness, and for criminal record, controlling only for parental
occupation,
The one for graduation was significant, but for criminal record it was p
= .06, which they accepted as significant, "although marginal".
That's not what's bugging me. What I want to know is if it's justifiable
to control for things like parental occupation and disruptiveness in a
randomized study. This is ok for correlational research, but why would
you want to do it in a randomized study where such factors are already
eliminated through randomization?
It seems to me they may have done this because it got them close enough
to the magic p= .05 to claim it anyway. If that's the only reason, I
don't think it's right. Also, once you've controlled in that way,
wouldn't that somehow limit the generality of your findings, that they're
now restricted to an artificial type of homogeneous population resulting
from the controlling? Is there a cost to doing it this way when you don't
have to?
As our Michael would say, send me something.
Stephen
-----------------------------------------------------------------
Stephen L. Black, Ph.D.
Professor of Psychology, Emeritus
Bishop's University e-mail: [EMAIL PROTECTED]
2600 College St.
Sherbrooke QC J1M 1Z7
Canada
-----------------------------------------------------------------------
---------------------------------------------------------------
Kenneth M. Steele, Ph.D. [EMAIL PROTECTED]
Professor
Department of Psychology http://www.psych.appstate.edu
Appalachian State University
Boone, NC 28608
USA
---------------------------------------------------------------
---
