Here is a little bit of answer ...
On 17 Apr 2000 07:39:02 -0700, [EMAIL PROTECTED] (Robert McGrath)
wrote:
< Snip. Concerning predictors based on field studies. ...
predictors were all dichotomous, were related to a series of criteria,
some of which were dichotomous and some of which were quantitative.
Many variables were skewed. Effect sizes were examined as
correlations or as Cohen's d, the standardized difference for two
means.>
> 1. r is more useful here for several reasons:
> a. It is generally applicable to both the dichotomous and quantitative
> criteria.
For dichotomous variables, which might be variously skewed, r is
horrible as a descriptor. Forget about continuous; it cannot compare
fairly between two sets of dichotomies. Epidemiological studies with
Ns of 1000s may have Odds Ratios of 2 or 4 or 8, but they those huge
"effects" are *never* described as correlations, since, indeed, r's
are functionally useless -- and, the Ns have to be huge, because the
r's are so small.
Remember, the 2x2 r-squared times N equals chisquared.
For a 2x2 table, with two huge samples that are Exposed and two small
numbers that are Diseased, the chisquared depends thoroughly on the
two small numbers. That is, 5% vs 1% gives the same test, very
nearly, as .05% vs .01%, if the two pairs of smaller Ns are
equivalent. The Odds Ratio is what has been useful in epidemiology.
(The OR is similar to the Risk Ratio, but the OR is much better
suited, logically and mathematically.)
> 2. d is more useful precisely because it is relatively resistant to the
> impact of skew, unless group SDs are markedly different.
- seems to me like it is going to be confusing and not very
interpretable. I would want to see some good examples before going
for it, over the Odds Ratio.
> 3. A third, less important issue, was raised in response to point 2. If
> effect size measures that are resistant to skew are more desirable, is there
> one that could be applied to both dichotomous and quantitative criteria? If
> not, which would be the "better" effect size measure for dichotomous
> criteria:
> a. the tetrachoric r: one person recommended this on the grounds that it
> is conceptually similar to the Pearson r and therefore more familiar to
> researchers.
- It has strong theoretical assumptions, a big standard error, and I
doubt if it estimates anything very different from what you have from
the phi. Again, I would need good, impressive examples before giving
this one serious consideration.
> b. the odds ratio: recommended because it does not require the
> distributional assumptions of the tetrachoric r.
The distributional assumptions of the Odds Ratio are pretty
reasonable, for a number of different purposes. You did not say
anything about creating your dichotomies from truly continuous
variables; and I have never found a need for tetrachoric.
If you want some limited assumptions for dichotomies, but ones that
are Normal instead of Logistic, you could borrow d-prime from
information theory -- basically, you translate each proportion into a
z, then look at the distance between z's. That would give you a
metric that is pretty much the same for all variables. I think.
>
> The key issue on which I'd like your input, although please feel free to
> comment on any aspect, is this. Given there is real-world skew in the
> occurrence of positives, does r or d present a more accurate picture?
> Should we think of these as small or medium-to-large effect sizes?
Go back and check Cohen and I think you will see that he was careful
not to overgeneralize about r and d . His levels seem to have
*little* to do with your debate, since your studies are a-typical.
He is talking about "the usual studies" in Social Sciences. Those are
ones that do *not* have a rare, dichotomous occurrence as an outcome.
--
Rich Ulrich, [EMAIL PROTECTED]
http://www.pitt.edu/~wpilib/index.html
===========================================================================
This list is open to everyone. Occasionally, less thoughtful
people send inappropriate messages. Please DO NOT COMPLAIN TO
THE POSTMASTER about these messages because the postmaster has no
way of controlling them, and excessive complaints will result in
termination of the list.
For information about this list, including information about the
problem of inappropriate messages and information about how to
unsubscribe, please see the web page at
http://jse.stat.ncsu.edu/
===========================================================================
