Fwd: nonparametric correlations

[David C. Howell]

I received the following message from someone else, and don't have a good answer for him. Does anyone out there have a good suggestion? If so, please write to the original questioner rather than to me. His address is "Todd Bailey" <[EMAIL PROTECTED]>

Thanks,
Dave Howell

X-Mailer: Novell GroupWise Internet Agent 6.5.1
Date: Mon, 10 May 2004 14:54:16 +0100
From: "Todd Bailey" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Subject: nonparametric correlations

Dear Prof. Howell,

In the process of analyzing some recent results of a 2AFC (Two Alternative Forced Choice Task)  task (participants chose whether syllable A or B sounded more similar to T), I wanted to do a non-parametric assessment of how well various models predicted the deviation from chance performance (i.e. 50% A responses).  Failing to find such a test described, either in your excellent book or in other sources I consulted, I wnder whether you could point me to such a test?

In the meanwhile, I "invented" two such tests myself, but I suspect there's a
good chance I have reinvented the wheel.  If not, then I wonder whether either
of these are of potential interest to others, and if so, how best to disseminate
them.  I don't really have the expertise to prove their distributional
properties, etc., myself.

The simpler metric is based on Spearman's rank correlation, but the ranks are
adjusted so that the rank corresponding to chance performance is 0.  Scores
lower than chance have negative ranks and scores greater than chance have
positive ranks.  Then, you compute a "correlation through the origin" instead of
through the means of the two sets of ranks (e.g. regress one set of adjusted
ranks on the other, while constraining the regression constant to be 0).

The second test is based on Kendall's tau, but incorporates an adjustment
penalty for cases in which one variable contains a score below a reference value
(e.g. chance) and the other is above the reference value.  If the scores are
adjusted so that the reference value is 0, then the new metric, tau0 = 1 -
numerator/denominator, where:

numerator = 2(Number of Kendall inversions) + (Number of sign
mismatches)(Number of objects - 1)

denominator = 2(Number of pairs of objects)

This metric ranges from -1 (ranks for the two variables are reversed relative
to each other and corresponding scores are on opposite sides of the reference,
i.e. all signs mismatch after adjusting to make the reference 0) to +1 (all
ranks in matching order and all scores on same side of reference).  The expected
score is 0 if the two variables are independent and the median of each is equal
to the reference.

I'm sorry this is so long, but I would appreciate any comments or advice you
can offer.

Best wishes,
todd bailey

Todd M. Bailey, Ph.D.
Lecturer, School of Psychology, Cardiff University
PO Box 901, Cardiff CF10 3YG, United Kingdom
Email: [EMAIL PROTECTED]
Phone: +44 29 2087 5375 / FAX: +44 29 2087 4858

**************************************************************************

David C. Howell
Professor Emeritus
University of Vermont

New address:
David C. Howell                                 Phone: (970) 871-4556
P.O. Box 770059         
Steamboat Springs, CO  80477            email: [EMAIL PROTECTED]


http://www.uvm.edu/~dhowell/StatPages/StatHomePage.html