On Fri, 29 Jan 2016 08:49:23 -0800, Douglas Peterson wrote:
[snip]
... SDT continues to be applicable in a number of settings,
particularly medical tests, many use a the AUC that Mike mentions
and while this isn't technically SDT (no z transforms) the ROC
method is identical (here is a short and good example
http://www.nature.com/nmeth/journal/v12/n9/fig_tab/nmeth.3482_SF9.html
)
A few points:
(1) As I mentioned in an earlier post, SDT is based on Wald's
statistical theory which serves as the basis for the Neyman-Pearson
framework for statistical testing. The decision matrix originally
developed is a 2 x 2 table where the rows represent the response
("yes" or "no", "present" or "absent", etc.) and the columns
represent the "true state of nature", that is, stimulus was presented
or not presented (this is knows with absolute certainty since they
are selected by the researcher; given that the "true state" is known,
the question that remains is how well do the responses or decision
match the true state -- if the Hit rate is 100% and Correct Rejection
ate is 100%, then there the False Alarm rate = 0.00 and the Miss
rate = 0.00, in other words, performance is perfect which with
weak stimuli in psychophysics rarely/never occurs).
(2) I am puzzled by Peterson's statement that AUC is not really
SDT given that it's equivalent A' was developed by memory
researchers as early as the 1960s and has been shown to be
part of SDT. In the http://opl.apa.org experiment on the "Self
Reference Effect", the dependent variable is a version of A'
that represents the area under "curve" created by the single
pair of Hit and False Alarm rates. One reference on this point
is the following:
Macmillan, N. A., & Creelman, C. D. (1996). Triangles in ROC
space: History and theory of "nonparametric" measures of
sensitivity and response bias. Psychonomic Bulletin & Review,
3(2), 164-170.
Given that the ROC/MOC/AOC is presented in a unit square
-- the x-axis represents the probability of a false alarm is limited
to the range 0.00 to 1.00 and the y-axis represents the prob of
a Hit which also ranges from 0.00 to 1.00 -- chance performance
is represented by the diagonal line representing P(Hit)=P(FA).
In traditional SDT this implies d-prime is zero. It also implies
that the area under the performance curve is 0.50 which can
be interpreted as a measure of accuracy; in this case, it represents
chance performance (hence the term "chance diagonal"). In
most Yes-No recognition memory experiments, only one hit rate
and one false alarm rate is obtained. For nonrandom performance,
this provides a single point above the chance diagonal, forming
a triangle with the chance diagonal as the base. The sum of
the area of the triangle and the area under the chance diagonal
(i.e., 0.50) becomes a measure of accuracy. As the Hit rate
increases and the False Alarm rate decreases, the area in the
triangle increases -- in the limit when the False Alarm rate is zero,
the triangle fills the upper space and A' or AuC is 1.00 or the
entire area of the unit-square. Thus, perfect performance is
represented by A' = AuC = 1.00.
(3) In making a medical diagnosis or interpreting a medical test,
the same reasoning above is employed but the terms differ::
Hit rate becomes True Positive Rate = "Sensitivity"
Correct Rejection become True Negative Rate = "Specificity"
For more on these ideas and how they are used to determine how
good your usual medical test is, see the Wikipedia entry:
https://en.wikipedia.org/wiki/Sensitivity_and_specificity
This entry eventually leads to d-prime but go to the Wikipedia
entry on ROC curves for alternative measures; including AuC:
https://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve
This my third post to TiPS today, so no more till the morrow.
-Mike Palij
New York University
[email protected]
---
You are currently subscribed to tips as: [email protected].
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=48012
or send a blank email to
leave-48012-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
