On Thu, 28 Jan 2016 20:08:38 -0800, Carol DeVolder wrote:
Dear TIPSters,
I am currently teaching about the Theory of Signal Detectability,
Stevens's Power Law, and ROC curves in my Sensation and
Perception course.
I have to admit that I find your lumping Stevens' Power law
with SDT and ROC (or, depending upon the phenomenon
being studied MOC or Memory Operating Characteristic
curves or AOC or Attention Operating Characteristics or
the more general measure AUC or Area under the Curve).
Given that SDT was developed in the context of detecting
weak signals in presence of noise while the Power law is
supposed to represent the relationship of stimulus magnitude
to sensory/subjective magnitude, I find it hard to reconcile
the two theories into a single framework.
Historically, Fechner leads to Stevens (among others)
for relating stimulus energies to sensation -- all above
an "absolute threshold" (if one believes in such a thing).
SDT does away with the concept of threshold in favor of
describing a person's performance in term of sensitivity
(ability to detect a stimulus, usually in a background of
noise of some sort) and bias or willingness to say "Yes"
(in a Yes-No task; other response in multiple alternative
tasks) which if often assumed to be independent of sensitivity
(but may be wrong in certain situations). This is why simple
measures of "accuracy" like "percent correct" are often
misleading indicators of a person's ability to detect or
discriminate stimuli.
Do any of you have any examples that you work on in class
or use to illustrate how to implement them?
You do understand that the types of task you would use with
SDT (ROC is just one way to represent the performance on
SDT tasks) would be different from those used with Power law?
If you put a gun to my head and say you'll blow my brains out
if I don't come with appropriate tasks, I'd suggest:
(1) Showing how the Self-Reference Effect (SRE; typically
a recognition memory task that uses SDT analysis -- see
the Http://opl.apa.org website for their implementation)
and
(2) How to use magnitude estimation procedures for various
social phenomena, such as seriousness of different crimes.
If Hugh Foley is still on Tips, he can provide more information
about this type of research from when he worked with Dave
Cross and others at Stony Brook back when he was in grad
school (a cohort of mine).
I want to do several things. First, I want to be able to
explain the logic of SDT, the power law, and ROCs.
It is probably me but I would have said the following instead
of what you wrote above:
(1) What is SDT, how it is a model of decision-making about
stimuli when they are difficult to detect or discriminate (not
limited to human; animal psychophysics have also used SDT
analysis), and how the ROC provides a convenient representation
of the performance on a SDT task (i.e., it shows the degree of
sensitivity as reflected by d' or a similar measure, the effect of
payoffs and probabilities of stimuli [placement of Beta along the
ROC curve], and accuracy [the area under the ROC curve]).
Second, I want to be able to make the topics relevant and
convince the students that these concepts are active in their
daily lives.
I think you need to be a little bit more specific about which "concepts"
you're referring to. Stevens' power law is just one example of the
"psychophysical law" and it has a number of problems associated
with it -- see the entry on Wikipedia for a brief presentation on the
objections to it:
https://en.wikipedia.org/wiki/Stevens'_power_law
Shepard has shown that what researcher what to do when it comes
to the psychophysical law if show the following relationship:
Sensation = f(stimulus energy)
The problem is that we cannot directly observe sensation so we
typically rely upon the following empirical relationship:
Response = f(stimulus energy)
In both cases, f(stimulus energy) is a mathematical function relation
stimulus energy to sensation or response but the function can take
a variety of form (just ask any Fechnerian ; -). Shepard, however,
has pointed out that this assume that there is a simple relationship
between response and sensation or
Response = f(sensation)
which can be ignored -- it has been ignored or over simplified in
Stevens and other psychophysical functions. So, the equation
that is possibly operating is:
Response = f(sensation[f(stimulus energy]))
That is, the observed response on, say, a magnitude estimation
task is the result of a function of a function, each may differ for
different stimuli.
With respect to SDT, originally it was based on Wald's statistical
decision theory which we are most familiar with whenever we use
the Neyman-Pearson framework for doing statistical analysis in
contrast to classical Fisherian analysis (i.e., it involves the concepts
of Type II errors, statistical power, confidence errors, etc.). So,
SDT represents a model of how (some) people might make decisions
in certain situations (if one were so inclined, one could contrast it
with
Duncan Luce's Choice Axioms approach which Creelman and McMillan
have done; see:
https://books.google.com/books?id=P094AgAAQBAJ&pg=PA201&dq=Creelman+and+McMillan&hl=en&sa=X&ved=0ahUKEwjU3vO6oM_KAhXGrB4KHTMTA8IQ6AEIJjAA#v=onepage&q=luce&f=false
For a more mundane example of how SDT operate which might be
what you're looking for, I suggest you take a look at the following
reference:
Lynn, S. K., & Barrett, L. F. (2014). "Utilizing" signal detection
theory.
Psychological science, 25(9), 1663-1673.
One benefit of the above article is that it uses a task where a person
has
to determine whether a person in a picture is scowling or not, which
may
make the SDT analysis more palatable.
And third, I want to give them some opportunities to practice.
I have to ask: Practice what?
I've already talked about hits, misses, false alarms, and correct
rejections
in class, and using payoffs to manipulate response criteria, now I want
to
make it all applicable.I welcome any and all ideas.
To make SDT relevant to my classes I refer to its application in
mammography
and the detection of breast cancer with seems to be interesting to both
females and males (for obvious reasons). This, of course, lead
naturally
to realization that a radiologist and doctors are making decisions that
can
be modeled by SDT -- I then suggest that when females go breast exams,
they ask their physicians what their d-prime or sensitivity measure is
and
that they should avoid physicians with negative d-primes or d-prime =
zero.
John Swets has done a lot of work in this area and one should look at
his
work. A recent paper that one might want to at is the following:
http://link.springer.com/chapter/10.1007/978-3-319-04831-4_36
If anyone is still reading this, another track to take is how clinical
diagnosis
is made in psychology and I suggest the chapter by McFall & Treat
in the Annual Review of Psychology; see:
http://www.annualreviews.org/doi/abs/10.1146/annurev.psych.50.1.215
The ref is:
McFall, R. M., & Treat, T. A. (1999). Quantifying the information value
of clinical assessments with signal detection theory. Annual review of
psychology, 50(1), 215-241.
HTH.
-Mike Palij
New York University
[email protected]
---
You are currently subscribed to tips as: [email protected].
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=48008
or send a blank email to
leave-48008-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
